# mpydocker

Launch a container for a given Docker or Apptainer image. This tool provides a
sophisticated wrapper around Docker and Apptainer to create user-friendly
development environments with automatic configuration for common development
needs.

The container setup:

- Automatically creates user matching host UID/GID for seamless file
  permissions (docker container runtime)
- Detects and mounts NVIDIA GPUs when available
- Configures X11/XWayland/XQuartz display support
- Forwards SSH authentication for git operations
- Expands environment variables in volume paths
- Supports Jupyter Lab server integration
- Allows custom build instructions

## Usage

```bash
mpydocker [options] image
```

### Container Runtime

The tool supports two container runtimes, Docker and Apptainer. The latter
support is yet experimental. To switch between the container runtimes, use the
`--engine` option and select either `docker` or `apptainer`. In case the option
is not explicitly set, `docker` will be used as default.

To run a docker container:

```bash
mpydocker --engine docker ubuntu:latest
```

or run a apptainer container:

```bash
mpydocker --engine apptainer docker://ubuntu:latest
```

### Restart and Attach (Docker only)

It is possible to attach interactively to a running container. The default name
of a container corresponds to the name of the parent directory. The command will
check if it finds a docker container, which is already running, and will open a
new bash inside this container. Note that opening a interactive bash session in
a running container works not for remote hosts.

Create a named container that you can reattach to:

```bash
mpydocker --name dev-env myimage:latest
```

Later, restart and attach with the same command. In case you forgot to specify
a name upon start of the container, use `docker container ls` to determine
the name.

### Running on Remote Hosts (Docker only)

It is possible launch a container on a remote host. The only requirement is that
the remote host has to have docker installed and the user has to have permission
to run docker containers. The remote host is specified with the `--host` option.
The syntax is the same as for ssh, e.g., `username@hostname:port`. The remote
host must be reachable via ssh without password, e.g., by using ssh keys. The
remote host must also have access to the docker registry to pull the image.

Using a remote host requires to launch the container in detached mode. This is
possible by using `--extra-args '--detach'`. Furthermore, you have to modify the
command to be executed in the container to something that keeps the container
alive, e.g., `sleep infinity`.

```bash
mpydocker --host <remote> --extra-args '--detach' ubuntu:latest sleep infinity
```

### Customize your Docker Container

It is possible to specify a reduced set of docker build instructions to build a
custom image, e.g., RUN or ENV. The image will get a new tag with the suffix
'MPY'. In case a newer image is pulled from the registry, the custom image will
be build automatically. The custom image is also used to add the host user as a
new user to the docker image. The reason is to enable the dynamic code feature
of the OpenFOAM Foundation software, which prevents running as root in the
docker image. A new user and a new group will be created, with the corresponding
user id and group id of the host system user. If the user id or group id is 1000
or less, the default user named 'ubuntu' will be used. The user will get sudo
permissions.

For example to build a custom image, by adding `x11-apps`
package to docker container for testing X11 capabilities:

```bash
mpydocker --custom-build "RUN sudo apt-get update" --custom-build "RUN sudo apt-get install -qy x11-apps" ubuntu:latest
```

### Customize your Apptainer Container

It is also possible to install additional packages into the apptainer container
upon launch. Therefore, `bash` instructions has to be provided, which will be
injected into the `%post` section of an [apptainer definition file](https://apptainer.org/docs/user/main/build_a_container.html#building-containers-from-apptainer-definition-files)

Make sure that you have setup the
[user namespace mapping](https://man7.org/linux/man-pages/man7/user_namespaces.7.html)
correctly by checking `etc/subuid` and `etc/subgid` for an entry for your
username, e.g., `foo:100000:65536`. Otherwise, the custom build will fail.

For example to build a custom image, by adding `x11-apps`
package to apptainer container for testing X11 capabilities:

```bash
mpydocker --engine apptainer --custom-build "apt-get update" --custom-build "apt-get install -qy x11-apps" docker://ubuntu:latest
```

### Variable Expansion

#### Variable Expansion on the Host

All variables specified as command line parameters are expanded on the host
system. If variables are specified in the configuration file, variable expansion
on the host system is enabled for the source definition in `--volume`
, and for the `--env` option.

#### Variable Expansion in the Image

The `--volume` option enable variable expansion for the target definition by
inspecting the image.  When specifying this options as command line
arguments, the dollar sign has to be escaped with a backslash, otherwise the
variables are expanded on the host system. Alternatively, you can use single
quotes.

For example to mount a directory as `run` directory for OpenFOAM Foundation software:

```bash
mpydocker --volume /path/to/cases:\${WM_PROJECT_USER_DIR}/run <image> # or
mpydocker --volume /path/to/cases:'${WM_PROJECT_USER_DIR}/run' <image>
```

or to mount the current directory to the site directory of OpenFOAM Foundation software:

```bash
mpydocker --volume $PWD:'${WM_PROJECT_SITE}/${WM_PROJECT_VERSION}' <image>
```

### Jupyter Server (Docker only)

In case you need a jupyter server to run jupyter notebooks in the container,
you have to configure `mpydocker` with the following `config.json`

```json
{
  "image": "<image:tag>",
  "extra_args": [
    "--detach",
    "-p",
    "8888:8888"
  ],
  "custom_build": [
    "RUN sudo apt-get update && sudo apt-get install -y python3 python3-pip",
    "RUN sudo pip3 install --break-system-packages Jupyterlab pandas ipywidgets scikit-learn pulp"
  ],
  "cmd": "jupyter",
  "args": [
    "lab",
    "--ServerApp.port=8888",
    "--ServerApp.ip=0.0.0.0",
    "--ServerApp.token=''",
    "--ServerApp.password=''"
  ]
}
```

The jupyter lab server is listening on port 8888. To access the jupyter server
use Visual Studio Code or a web browser with the following url:
<http://127.0.0.1:8888/lab>. If the server is running remotely, use the ip
address or hostname of the remote machine instead:
http://\<ip-address\>:8888/lab. The container will be started as 'detached' in
the background. Use `docker container ls` to obtain the container id. Then you
can use `docker container stop <container id>` to stop the container, or
`docker exec -it <container id>` to attach to the server.

If you install new packages via `pip install --break-system-packages <package>`
you have to restart the jupyter kernel afterwards.

Run a container with jupyter lab server, access at <http://127.0.0.1:8888/lab>:

```bash
mpydocker --config config.json --name server <image>
```

### Display Support

Display support works on Linux systems with X11 and XWayland. On Mac OS enable
in settings for XQuartz 'Allow connections from network clients'. If a Nvidia
runtime environment is available, the tool mounts mount the graphic cards (GPU)
to the container automatically at runtime. If no Nvidia support is found, only
the default display device is added. Note, AMD graphics cards are not supported
at the moment.

For NVIDIA GPUs the propritery NVIDA driver has to be installed. To this end
follow the
[NVIDIA installation instructions](<https://docs.nvidia.com/datacenter/tesla/driver-installation-guide/index.html>)
and make sure that the driver is installed
properly. This can be done by running `nvidia-smi` on the host system.

Once, the driver is installed the
[NVIDIA Container Toolkit](<https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html>)
has to be installed too. To verify that
the toolkit works correctly run

```bash
sudo docker run --gpus=all --runtime=nvidia --rm nvcr.io/nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
```

Note that an update of the NVIDIA driver can cause a update or re-installation
of the NVIDIA Container Toolkit too. Always verify that `nvidia-smi` works on
the host system as well as in a container.

## Configuration File

An example configuration file can be produces by `mpydocker --print-config`.
For convenient usage create `~/.config/mpydocker/config.json`:

```json
{
  "image": "myregistry/myimage:latest",
  "volume": [
    "~/cases:${WM_PROJECT_USER_DIR}/run",
    "~/data:/data"
  ],
  "custom_build": [
    "RUN sudo apt-get update",
    "RUN sudo apt-get install -y vim"
  ]
}
```

Then simply run:

```bash
mpydocker
```