Run a container with a host directory mount, and it either leaves root-owned files behind or it runs into "permission denied" errors. Welcome to the dreadful container host filesystem owner matching problem. These issues confuse and irritate people, and they happen because apps in the container run as a different user than the host user.

There are various strategies to solve this issue, but they are all non-trivial (requiring complex logic) and/or have significant caveats (e.g., requiring privileged containers). Here's where my new tool MatchHostFsOwner comes in.

How does MatchHostFsOwner solve container file permission pains?

MatchHostFsOwner implements solution strategy number 1. It ensures that the container runs as the same user (UID/GID) as the host's user. In short, it:

  • modifies a user account inside the container so that the account's UID/GID matches that of the host user.
  • executes the actual container command as the aforementioned user account (instead of, e.g., letting it execute as root).

This strategy is easier said than done, and the article documents the many caveats involved with this strategy. Fortunately, MatchHostFsOwner is here to help because it addresses all these caveats, so you don't have to.

Using MatchHostFsOwner

Here are some core concepts to understand:

  • It's an entrypoint — Install MatchHostFsOwner as the container entrypoint program. It should be the first program to run in the container. When it runs, it modifies the container's environment, then executes the next command with the proper UID/GID.

  • It requires host user input — when starting a container, the host user must tell MatchHostFsOwner what the host user's UID/GID is. How the user passes this information depends on what tool the user uses to start the container (e.g., Docker CLI, Docker Compose, Kubernetes, etc).

  • It requires an extra user account in the container — MatchHostFsOwner tries to execute the next command under a user account in the container whose UID equals the host user's UID. If no such account exists (which is common), then MatchHostFsOwner will take a specific account and modify its UID/GID to match that of the host user.

    The account MatchHostFsOwner will take and modify is called the "app account". MatchHostFsOwner won't create this account for you — you have to supply it. It won't always be used, but often it will.

    By default, MatchHostFsOwner assumes that the app account is named app. But this is customizable.

  • It requires root privileges — MatchHostFsOwner itself requires root privileges to modify the container's environment. It drops these privileges later before executing the next command.

    How exactly MatchHostFsOwner is granted root privileges depends on how one is supposed to start the container. This brings us to the two usage modes.

Usage mode 1: start container without root privileges

This mode is most suitable for starting the container without root privileges. For example:

  • When your Dockerfile sets a default user account using USER.
  • When your container is supposed to be started with docker run --user.
  • When your Kubernetes spec makes use of securityContext's runAsUser/runAsGroup.

In this mode, you must grant MatchHostFsOwner the setuid root bit. MatchHostFsOwner drops its setuid root bit as soon as possible after it has done its work.

This mode has some limitations:

  • The container cannot be started a second time (e.g., using docker stop and then docker start). Upon starting the container for the second time, MatchHostFsOwner no longer has the setuid root bit, so it won't be able to do its job. Thus, mode 1 is only useful for ephemeral containers.
  • Incompatible with Docker Compose because it may start the container a second time.
  • Requires that the container filesystem in which MatchHostFsOwner is located, to be writable. Because MatchHostFsOwner must be able to drop the setuid root bit. Thus, you cannot run the container in read-only mode (e.g., docker run --read-only).

Usage mode 1 in action

Begin by preparing the container.

  • Create an account in your container for running your app. It doesn't matter what you name it (it's customizable), but let's call it "app" in this demo because MatchHostFsOwner assumes by default that that's the name. Set this account up as the default account for the container.
  • Place the MatchHostFsOwner executable in a root-owned directory (e.g., /sbin) and ensure that the executable is owned by root, and has the setuid root bit.
  • Set up the MatchHostFsOwner executable as the container entrypoint.

For example:

FROM ubuntu:22.04

# Install MatchHostFsOwner. Replace X.X.X with an actual version.
# See https://github.com/FooBarWidget/matchhostfsowner/releases
ADD https://github.com/FooBarWidget/matchhostfsowner/releases/download/vX.X.X/matchhostfsowner-X.X.X-x86_64-linux.gz /sbin/matchhostfsowner.gz
RUN gunzip /sbin/matchhostfsowner.gz && \
  chown root: /sbin/matchhostfsowner && \
  chmod +x,+s /sbin/matchhostfsowner
RUN addgroup --gid 9999 app && \
  adduser --uid 9999 --gid 9999 --disabled-password --gecos App app
## Or, on RHEL-based images:
# RUN groupadd --gid 9999 app && \
#   useradd --uid 9999 --gid 9999 app
## Or, on Alpine-based images:
# RUN addgroup -g 9999 app && \
#   adduser -G app -u 9999 -D app
USER app

ENTRYPOINT ["/sbin/matchhostfsowner"]

docker build . -t my-example-image

Next, start the container using a user and group ID that matches the host user's. For example, using the Docker CLI. (See the documentation for a Kubernetes-based example.)

docker run --user "$(id -u):$(id -g)" my-example-image id -a
# Output (assuming host UID/GID is 501/20):
# uid=501(app) gid=20(app) groups=20(app)

Success! Here's what happened under the hood:

  • MatchHostFsOwner (the entrypoint) runs before the container command (id -a) does.
  • MatchHostFsOwner sees the container is running as UID/GID 501/20. So it modifies the "app" account's UID/GID to 501/20. It can do that because it has setuid root privileges.
  • MatchHostFsOwner drops its setuid root privileges, then executes the command id -a under the container's "app" account.

Usage mode 2: start container with root privileges

In this mode, MatchHostFsOwner obtains root privileges through the fact that one starts the container with root privileges. No setuid root privileges required. MatchHostFsOwner drops its root privileges as soon as possible after it has done its work.

This mode is most suitable if any of the following is applicable:

  • You're using Docker Compose.
  • The container could be started a second time, as happens with, e.g., Docker Compose.
  • The container filesystem in which MatchHostFsOwner is located is read-only.

Usage mode 2 in action

Begin by preparing the container:

  • Create an account in your container for running your app. It doesn't matter what you name it (it's customizable), but let's call it "app" in this demo because MatchHostFsOwner assumes by default that that's the name. Set this account up as the default account for the container.
  • Place the MatchHostFsOwner executable in a root-owned directory (e.g., /sbin) and ensure that the executable is owned by root.
  • Set up the MatchHostFsOwner executable as the container entrypoint.
  • Don't set a default user account with USER.

Example:

FROM ubuntu:22.04

# Install MatchHostFsOwner. Replace X.X.X with an actual version.
# See https://github.com/FooBarWidget/matchhostfsowner/releases
ADD https://github.com/FooBarWidget/matchhostfsowner/releases/download/vX.X.X/matchhostfsowner-X.X.X-x86_64-linux.gz /sbin/matchhostfsowner.gz
RUN gunzip /sbin/matchhostfsowner.gz && \
  chown root: /sbin/matchhostfsowner && \
  chmod +x /sbin/matchhostfsowner
RUN addgroup --gid 9999 app && \
  adduser --uid 9999 --gid 9999 --disabled-password --gecos App app
## Or, on RHEL-based images:
# RUN groupadd --gid 9999 app && \
#   useradd --uid 9999 --gid 9999 app
## Or, on Alpine-based images:
# RUN addgroup -g 9999 app && \
#   adduser -G app -u 9999 -D app

ENTRYPOINT ["/sbin/matchhostfsowner"]

docker build . -t my-example-image

Next, start the container while setting the environment variables MHF_HOST_UID and MHF_HOST_GID to the host user's UID/GID like this:

docker run -e "MHF_HOST_UID=$(id -u)" -e "MHF_HOST_GID=$(id -g)" my-example-image id -a
# Output (assuming host UID/GID is 501/20):
# uid=501(app) gid=20(app) groups=20(app)

Here's what happened under the hood:

  • MatchHostFsOwner (the entrypoint) runs before the container command (id -a) does.
  • MatchHostFsOwner sees that MHF_HOST_UID/MHF_HOST_GID is set to 501/20. So it modifies the "app" account's UID/GID to 501/20.
  • MatchHostFsOwner drops its root privileges, then executes the command id -a under the container's "app" account.
MatchHostFsOwner mascot: dog with glasess
MatchHostFsOwner project mascot

Conclusion

MatchHostFsOwner is an excellent way to solve Docker volume permission problems (more precisely: the container host filesystem owner matching problem). Please have a look at its source code (it's written in Rust!) and check out its documentation for customization, advanced usage, and troubleshooting instructions.

Stay cured!