yesnik/docker-compose.md

Last active March 25, 2025 13:24

Star (2) You must be signed in to star a gist
Fork (1) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/yesnik/46b830afeba0eb3c5fd978dfe33c53b4.js"></script>
Save yesnik/46b830afeba0eb3c5fd978dfe33c53b4 to your computer and use it in GitHub Desktop.

Download ZIP

Docker Guides

Raw

docker-compose.md

Docker Compose

Docker compose is a tool that is used for defining and running multi-container Docker apps in an easy way. It provides docker-compose.yml configuration file that can be used to bring up an app and the suite of services it depends on with just one command.

docker compose up - start all services from docker-compose.yml
docker compose up db - start only service db
docker compose up --build - rebuild all images and run containers
docker compose restart db - restart db service
docker compose kill db - kill db service
docker compose -f docker-compose.prod.yml up - start all services from docker-compose.prod.yml
docker compose -f docker-compose.yml -f docker-compose.staging.yml up -d - run Docker Compose and override config file
docker compose build - build images for all services
docker compose build db - rebuild db service
docker compose down
docker compose down --remove-orphans - remove containers for services not defined in the Compose file.
docker compose down -v - shutdown containers, remove volumes. Docker stores volumes at /var/lib/docker/volumes
docker compose logs db | less - show logs of db container
docker compose ps - show running app's containers
docker compose exec fpm bash - connect to running fpm container and start bash console in it
docker compose exec db mysql -u root -ppassword - connect to running db container and start mysql console
docker compose events - listen for events that happen on the Compose application
docker compose run --rm php-cli sh - run php-cli container, start sh console. --rm - Docker also removes the anonymous volumes associated with the container when the container is removed.
docker compose rm db - remove db container
docker compose rm -fv - remove all containers with volumes
docker compose top - it will run top command in the containers listed in the docker-compose.yaml
docker compose config - show docker-compose.yml content after the substitution step has been performed
docker compose images - show images used by containers

Proxy settings for Docker

If you see an error:

docker compose up
Pulling db (mysql:)...
ERROR: Get https://registry-1.docker.io/v2/: Proxy Authentication Required

you should configure your Docker's proxy settings (see stackoverflow).

Create file /etc/systemd/system/docker.service.d/http-proxy.conf:

[Service]
Environment="HTTP_PROXY=http://kenny:[email protected]:3118"
Environment="HTTPS_PROXY=http://kenny:[email protected]:3118"
Environment="NO_PROXY=localhost,127.0.0.1,localaddress,.localdomain.com"

Reload docker:

sudo service docker restart

Check if it works:

docker pull busybox

Environment variables

Example of docker-compose.yml with env variables:

services:
    database:
        image: postgres:${POSTGRES_VERSION:-15}-alpine
        environment:
            POSTGRES_DB: ${POSTGRES_DB:-app}
            POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-123}
            POSTGRES_USER: ${POSTGRES_USER:-app}
        volumes:
            - database_data:/var/lib/postgresql/data:rw

We can pass environment variables to docker-compose.yml files in these ways:

By exporting the variable to the terminal before running docker compose:
```
POSTGRES_PASSWORD=123 docker compose config
```
By putting the variables inside .env file:
```
POSTGRES_PASSWORD=123
```

The env_file option only passes those extra variables to the containers and not the docker-compose.yml file.

docker-compose.yml

`build`, `image`

If we use build and image, the image will be built and will then be tagged based on the name given using image:

services:
    some-service:
        build: .
        image: some-service:1.5

`healthcheck`

services:
    some-service:
        # ...
        healthcheck:
            test: ["CMD", "curl", "-f", "http://localhost:8080/ping"]
            interval: 20s
            timeout: 10s
            retries: 3
            start_period: 5s

Every 20 seconds after an initial period of 5 seconds, a ping to some-service will be issued using curl with a timeout of 10 seconds. In case of failure, there should be 3 retries before marking the instance unhealthy. We use curl with -f option: fail fast with no output on HTTP errors.

Run docker ps to see if container's status is healty or unhealty.

Links:

Raw

docker_commands.md

Docker Commands

Installation

We can install Docker using official bash script.

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

System

docker system prune - clean up any resources - images, containers, volumes, and networks — that are dangling (not associated with a container)
docker system prune -a - remove ALL stopped containers, unused networks, abandoned images, build cache

Containers

They are created from Docker Images and run the actual app.

docker run busybox echo 123 - find image, load up the container, run a provided command
docker run --rm busybox - flag --rm: remove container when it exits
docker run -it busybox sh - flag --it: attach us to an interactive tty in the container
docker run --name hello -it busybox sh - param --name hello sets name hello for created container
docker run -p 8000:80 nginx - flag -p: publish a container's port to the host
docker run --net foo-network foo-container - flag --net: set network for container
docker run --rm -it -v $(pwd):/app php:8 bash - create volume for current dir
docker run --rm -it redis redis-cli -h host.docker.internal -p 6379 - run redis container, enter redis-cli connected to the port 6379
docker cp nginx-example-nginx-1:/etc/nginx/nginx.conf ./nginx.conf - copy file from working container to host
docker ps - list running containers
docker ps -a - list all containers
docker start foo-container - start existing container by name
docker start 0a8a89d6b4e5 - ... by container ID
docker stop foo-container - stop existing container by name
docker stop 0a8a89d6b4e5 - ... by container ID
docker rm foo-container - remove container
docker kill foo-container - kill container
docker container ls - list contaners
docker container ls -a - list all containers
docker container prune - remove all stopped containers
docker port foo-container - show exposed ports
docker container logs foo-container - see logs of the container
docker exec -it foo-container sh - run console into container
docker logs -f foo-container - show logs of container. Docker's log location: /var/lib/docker/containers/
docker stats - show statistics for running containers
docker stats foo-container - show CPU, Memory Usage, Net, Block I/O of container

Images

They are blueprints of our app which form the basis of containers.

docker build -t kenny/nginx:2.0 . - build an image with repo name kenny/nginx and tag 2.0. And the . lets Docker know where it can find the Dockerfile.
docker image tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG] - Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE (it's like rename image tag)
docker image ls - list images
docker image prune - remove all dangling images (not used by containers)
docker rmi -f foo-image-name - remove image, -f - forced mode
docker save nginx > nginx.tar - save image to .tar archive. Size: 121Mb
docker save nginx:latest | gzip > nginx.tar.gz - save image to .tar.gz archive. Size: 50Mb
docker load < ./nginx.tar.gz - load image from .tar.gz or .tar archive

Volumes

docker volume create my-volume - create volume
docker volume rm my-volume - remove volume
docker volume ls - list all volumes
docker volume inspect my-volume - show info about volume
docker run -it --rm --name example-volume-container --mount source=my-volume,target=/storage bash - mount my-volume to /storage path in the container of bash image
docker run -it --rm --name read-only-volume-container --mount source=my-volume,target=/storage,readonly bash - mount my-volume to /storage path in readonly mode. We can't write data to this volume.

Volumes location:

Windows (open in the Explorer): \\wsl$\docker-desktop-data\data\docker\volumes\my-volume

Volume can be located anywhere, the driver will facilitate making the usage of the volume possible.

Volume drivers can enable interactions with a remote filesystem and persist data. Available drivers include: Azure File Storage, NFS, and AWS EFS.

Characteristics of volumes:

Volume with data can remain after a container has been deleted
We can attach a volume to another container
One volume can be used by many containers
Volumes can be local or remote

Network

docker network ls - list docker's networks
docker network inspect bridge - inspect bridge network of docker
docker network create foo-network - create new bridge network
docker network rm foo-network - remove network
docker network prune - remove all custom networks not used by at least one container

Network types

Networks on Docker can come in various forms. The most used formats include the following:

Bridge

It's the default network driver when using Docker. A bridge network on Docker is a software layer providing connectivity between containers that are connected to the same bridge network. Also, those containers are isolated from containers that are not connected to that network.

Run nginx container: docker run --rm -p 8080:80 --name nginx-my nginx
Check the network that the container is attached to: docker inspect nginx-my. Find value Networks.bridge.NetworkID
Ensure that we have bridge network with the same id: docker network ls

Host

By using the Docker host, the container is not isolated from it. The container does not have its own IP address. The container will be accessed using the host IP.

Run nginx instance using the host network: docker run --rm --network host --name nginx-my nginx
Ensure that the container doesn't have an IP assigned. Check section NetworkSettings.Networks: docker inspect nginx-my.
Run bash container: docker run -it --network host --rm --name nginx-wget bash
Try to get access to localhost: wget -O- localhost

Here, instead of attaching the containers to a network, we mounted them to the host network. As expected, the container doesn't have an IP assigned; therefore, any requests to localhost are mapped to the host.

Overlay

Bridge, host networks centered on Docker Engine installed on one host. Using Docker on a production with an orchestration engine such as Swarm requires the use of multiple hosts. The overlay network creates a distributed network among those hosts. An overlay network will transparently join the host networks and create a unified network on top of them.

Default address pools

By default, docker creates networks in these pools:

172.17.0.0/16 - 172.31.0.0/16
192.168.0.0/20 - 192.168.240.0/20

These IP addresses may conflict with server's IP address. You may lose SSH connection trying to docker-compose up. In this case create file /etc/docker/daemon.json:

{
  "default-address-pools" : [
    {
      "base" : "172.31.0.0/16",
      "size" : 24
    }
  ]
}

In this case created networks will have Subnet: 172.31.1.0/24, 172.31.2.0/24, .., 172.31.255.0/24.

Useful commands

Copy config from working container

docker run -d httpd:2.4
docker ps
docker cp [ID of container]:/usr/local/apache2/conf/httpd.conf ./httpd.conf

docker-compose run php bash
docker ps # See Container ID
docker cp 164b0441bb1c:/usr/local/etc/php/php.ini-development ./

Raw

docker_logging_drivers.md

Docker Logging Drivers

Docker supports many logging drivers.

Graylog Extended Log Format (GELF)

Docker config file /etc/docker/daemon.json:

{
  "log-driver": "gelf",
  "log-opts": {
    "gelf-address": "udp://192.168.110.10:12201"
  }
}

Docker will start to send logs to Graylog.

Raw

docker_swarm.md

Docker Swarm

docker node ls - show virtual machines in the cluster
docker service ls - show services in cluster
docker node inspect jenkins - show detail info for node
docker node update jenkins --label-add db - add label db for node jenkins
docker service ps site_api-php - show replicas' info for one serivce

Raw

dockerfile.md

Dockerfile

See official Dockerfile reference

Each image has base image layer, e.g. FROM debian:12-slim
Each instruction in Dockerfile creates Instruction Layer, e.g. RUN apt update && apt install iputils-ping -y
On subsequent image build Docker use cached layers. You'll see in logs:
```
#5 [2/2] RUN apt update && apt install iputils-ping -y
#5 CACHED
```
Put frequently changed commands at the bottom of Dockerfile to get maximum benefit from cached layers.

To build image run this command in the same directory where Dockerfile is located:

docker build -t myimage .

Run container based on our new image:

docker run -it --rm myimage

Example 1

# base image
FROM python:3-onbuild

# This port the container should expose
EXPOSE 5000

# Tell the container which command it should run when it's started
CMD ['python', './app.py']

Example 2

FROM nginx:latest

ADD ./nginx/default.conf /etc/nginx/conf.d/default.conf
WORKDIR /var/www/frontend

Example 3

FROM debian:12-slim

RUN apt update && apt install iputils-ping -y

CMD ["echo", "Hello!"]

Example 4. Python app

File requirements.txt:

requests==2.31.0

File app.py:

print("Hello!")

File Dockerfile:

FROM python:3.9-alpine

ENV APP_NAME="Hey Python!"

WORKDIR /app

RUN apk update

COPY requirements.txt requirements.txt

COPY app.py app.py

RUN pip install -r requirements.txt

CMD python app.py

Dockerfile commands

RUN - executes command(s) in a new layer and creates a new image. E.g., it is often used for installing software packages.
CMD ["echo", "Hi"] (exec form, preferred), CMD echo "Hi" (shell form) - sets default command and/or parameters, which can be overwritten from command line when docker container runs.
ENTRYPOINT ["python", "app.py"] (exec form, preferred), ENTRYPOINT python app.py (shell form) - configures a container that will run as an executable.
COPY ./index.html /var/www/html/index.html - copy file from current directory to a container

Debug Dockerfile

Place this code in the Dockerfile to prevent further build process:

# ...
RUN exit 1;

# .. Instructions below will not be executed

Look at the last console output:

/usr/local/share/doc/openssl/html/man7/x509.html
Removing intermediate container e535156b723f
 ---> 3402f3207c06
Step 8/21 : RUN exit 1;
 ---> Running in 0c00f6ae64fb
The command '/bin/sh -c exit 1;' returned a non-zero code: 1

You can run saved contaner 3402f3207c06 before an error occured and try to find out what happened:

docker run -it 3402f3207c06 bash

Raw

errors.md

Docker Errors

ERROR: network site_default is ambiguous

docker network ls
docker network rm e2d790be0302