Use Docker Compose Override Files#

This guide explains how to use Docker Compose override files to customize your Dioptra deployment. Override files allow you to extend or modify the base docker-compose.yml configuration without editing it directly.

Prerequisites#

Why Use Override Files?#

The docker-compose.yml file is generated by the cookiecutter template and managed by cruft. When you run cruft update, this file may be regenerated with upstream changes.

If you edit docker-compose.yml directly, your changes will conflict with template updates, requiring manual conflict resolution.

Using docker-compose.override.yml instead keeps your customizations separate from the template-managed file. Docker Compose automatically merges both files when you run docker compose up, and your override file remains untouched during cruft update.

Configuration Steps#

Step 1: Create the Override File#

The deployment template includes a docker-compose.override.yml.template file with commented examples for common customizations.

Copy this template to create your override file:

cp docker-compose.override.yml.template docker-compose.override.yml

Important

Only perform this copy once when first setting up customizations. If you have already created docker-compose.override.yml and added customizations, do not run this command again as it will overwrite your changes.

Step 2: Understand the File Structure#

The override file uses standard Docker Compose syntax. To modify a service, specify the service name and only the properties you want to change or add.

Basic structure:

# Modify existing services
services:
  service-name:
    # Properties here are merged with docker-compose.yml
    volumes:
      - "/host/path:/container/path:ro"
    environment:
      NEW_VAR: value

# Define new named volumes
volumes:
  volume-name:
    driver: local
    driver_opts:
      type: nfs
      o: "addr=<nfs-server-ip>,nfsvers=4"
      device: ":<exported-directory>"

Note

Replace <nfs-server-ip> with your NFS server’s IP address and <exported-directory> with the path to the exported directory on the NFS server.

How merging works:

  • Scalar values (strings, numbers): Override file values replace base values

  • Lists (like volumes: or ports:): Items are appended, not replaced

  • Mappings (like environment:): Keys are merged; override values take precedence

Step 3: Add Your Customizations#

Open docker-compose.override.yml in a text editor and add your customizations.

Note

In the examples below, replace <deployment-name> with your deployment’s slugified name (default: dioptra-deployment) and <host-data-path> with the absolute path to your data directory.

Example: Adding a volume mount to a worker

services:
  <deployment-name>-tfcpu-01:
    volumes:
      - "<host-data-path>:/dioptra/data:ro"

Example: Setting environment variables

services:
  <deployment-name>-tfgpu-01:
    environment:
      NVIDIA_VISIBLE_DEVICES: 0

Example: Using a custom-built image for a specific worker

If you are using downloaded images but need to use a custom-built image for a specific worker (e.g., a modified GPU worker), you can override the image for that service. This is useful because downloaded and locally built images have different registry prefixes. See Understanding Container Registry Prefixes for more details.

services:
  # Override image for a custom-built worker
  <deployment-name>-tfgpu-01:
    image: dioptra/tensorflow2-gpu:dev

Note

Replace dioptra/tensorflow2-gpu:dev with the name and tag of your custom-built image.

Example: Multiple customizations in one file

You can combine multiple customizations in a single override file:

services:
  # Mount data directory to CPU worker
  <deployment-name>-tfcpu-01:
    volumes:
      - "<host-data-path>:/dioptra/data:ro"

  # Configure GPU assignment for GPU worker
  <deployment-name>-tfgpu-01:
    environment:
      NVIDIA_VISIBLE_DEVICES: 0
    volumes:
      - "<host-data-path>:/dioptra/data:ro"

# Define an NFS volume
volumes:
  shared-data:
    driver: local
    driver_opts:
      type: nfs
      o: "addr=<nfs-server-ip>,auto,rw,bg,nfsvers=4,intr,actimeo=1800"
      device: ":<exported-directory>"

Step 4: Apply Your Changes#

After modifying the override file, restart the deployment to apply changes:

docker compose down
docker compose up -d

Verify the configuration was applied:

docker compose config

This command shows the merged configuration that Docker Compose will use.

See Also#

Common Customizations:

Reference: