Using Docker-Compose, how to execute multiple commands

x

x-yuri

Figured it out, use bash -c.

Example:

command: bash -c "python manage.py migrate && python manage.py runserver 0.0.0.0:8000"

Same example in multilines:

command: >
    bash -c "python manage.py migrate
    && python manage.py runserver 0.0.0.0:8000"

Or:

command: bash -c "
    python manage.py migrate
    && python manage.py runserver 0.0.0.0:8000
  "

@Pedram Make sure you are using an image that actually has bash installed. Some images may also require a direct path to bash e.g. /bin/bash

If there is no bash installed you could try sh -c "your command"

Alpine-based images actually seem to have no bash installed - do like @Chaoste recommends and use sh instead: [sh, -c, "cd /usr/src/app && npm start"]

Can also use just ash on alpine :)

I use sh -cx so i can also see the commands running. useful for debugging.

P

Pang

I run pre-startup stuff like migrations in a separate ephemeral container, like so (note, compose file has to be of version '2' type):

db:
  image: postgres
web:
  image: app
  command: python manage.py runserver 0.0.0.0:8000
  volumes:
    - .:/code
  ports:
    - "8000:8000"
  links:
    - db
  depends_on:
    - migration
migration:
  build: .
  image: app
  command: python manage.py migrate
  volumes:
    - .:/code
  links:
    - db
  depends_on:
    - db

This helps things keeping clean and separate. Two things to consider:

You have to ensure the correct startup sequence (using depends_on). You want to avoid multiple builds which is achieved by tagging it the first time round using build and image; you can refer to image in other containers then.

This seems like the best option to me, and I would like to use it. Can you elaborate on your tagging setup to avoid multiple builds? I would prefer to avoid extra steps, so if this needs some, I might go with bash -c above.

In the yaml above, the build and tagging happens in the migration section. It's not really obvious at first sight, but docker-compose tags it when you specify the build AND image properties - whereby the image property specifies the tag for that build. That can then be used subsequently without triggering a new build (if you look at web, you see it has no build but only an image property). Here's some more details docs.docker.com/compose/compose-file)

While I like the idea of this, the problem is that depends_on only ensures they start in that order, not that they are ready in that order. wait-for-it.sh may be the solution some people need.

That is absolutely correct and a bit of a shame that docker-compose doesn't support any fine grained control like waiting for a container to exit or start listening on a port. But yes, a custom script does solve this, good point!

This answer gives incorrect and potentially destructive information about how depends_on work.

P

Pang

I recommend using sh as opposed to bash because it is more readily available on most Unix based images (alpine, etc).

Here is an example docker-compose.yml:

version: '3'

services:
  app:
    build:
      context: .
    command: >
      sh -c "python manage.py wait_for_db &&
             python manage.py migrate &&
             python manage.py runserver 0.0.0.0:8000"

This will call the following commands in order:

python manage.py wait_for_db - wait for the DB to be ready

python manage.py migrate - run any migrations

python manage.py runserver 0.0.0.0:8000 - start my development server

Personally this is my favourite, and cleanest, solution.

Mine too. As @LondonAppDev points-out, bash isn't available by default in all containers to optimize on space (e.g., most containers built on top of Alpine Linux)

I had to escape the multiline && with a \

@AndreVanZuydam hmmm that's strange, I didn't need to do that. Did you surround with quotes? What flavour of docker are you running?

@oligofren the > is used to start a multi-line input (see stackoverflow.com/a/3790497/2220370)

P

Pang

This works for me:

version: '3.1'
services:
  db:
    image: postgres
  web:
    build: .
    command:
      - /bin/bash
      - -c
      - |
        python manage.py migrate
        python manage.py runserver 0.0.0.0:8000

    volumes:
      - .:/code
    ports:
      - "8000:8000"
    links:
      - db

docker-compose tries to dereference variables before running the command, so if you want bash to handle variables you'll need to escape the dollar-signs by doubling them...

    command:
      - /bin/bash
      - -c
      - |
        var=$$(echo 'foo')
        echo $$var # prints foo

...otherwise you'll get an error:

Invalid interpolation format for "command" option in service "web":

Hi, mate. I met a problem: ``` unrecognized arguments: /bin/bash -c python3 /usr/local/airflow/__init__.py -C Local -T Windows ``` the command in my docker-compose.yml is: command: - /bin/bash - -c - | python3 /usr/local/airflow/__init__.py -C ${Client} -T ${Types} Do you know how to fix that? I add Client and Types in my .env file.

Here's a doc for you: docs.docker.com/compose/compose-file/#variable-substitution I think what's happening is that your .env file places those variables in the container environment, but docker-compose is looking in your shell environment. Try instead $${Types} and $${Client}. I think this will prevent docker compose from interpreting those variables and looking for their values in whatever shell you invoke docker-compose from, which will mean that they're still around for bash to dereference them (after docker has processed your .env file).

Thanks for your comment. I did what you said in fact. So I got the $(Client) in the error information. I changed the way to read environment variables to use os.getenv in python, which is easier. Thanks anyway.

M

MUHAHA

Cleanest ?

---
version: "2"
services:
  test:
    image: alpine
    entrypoint: ["/bin/sh","-c"]
    command:
    - |
       echo a
       echo b
       echo c

This worked perfectly when the entrypoint of my image was other than sh/bash.

H

Harshad Yeola

You can use entrypoint here. entrypoint in docker is executed before the command while command is the default command that should be run when container starts. So most of the applications generally carry setup procedure in entrypoint file and in the last they allow command to run.

make a shell script file may be as docker-entrypoint.sh (name does not matter) with following contents in it.

#!/bin/bash
python manage.py migrate
exec "$@"

in docker-compose.yml file use it with entrypoint: /docker-entrypoint.sh and register command as command: python manage.py runserver 0.0.0.0:8000 P.S : do not forget to copy docker-entrypoint.sh along with your code.

Note that this will execute also when you do docker-compose run service-name ....

r

rweng

Another idea:

If, as in this case, you build the container just place a startup script in it and run this with command. Or mount the startup script as volume.

Yes, at the end I created a run.sh script: #!/bin/bash \n python manage.py migrate \n python manage.py runserver 0.0.0.0:8000 (ugly oneline)

r

radtek

* UPDATE *

I figured the best way to run some commands is to write a custom Dockerfile that does everything I want before the official CMD is ran from the image.

docker-compose.yaml:

version: '3'

# Can be used as an alternative to VBox/Vagrant
services:

  mongo:
    container_name: mongo
    image: mongo
    build:
      context: .
      dockerfile: deploy/local/Dockerfile.mongo
    ports:
      - "27017:27017"
    volumes:
      - ../.data/mongodb:/data/db

Dockerfile.mongo:

FROM mongo:3.2.12

RUN mkdir -p /fixtures

COPY ./fixtures /fixtures

RUN (mongod --fork --syslog && \
     mongoimport --db wcm-local --collection clients --file /fixtures/clients.json && \
     mongoimport --db wcm-local --collection configs --file /fixtures/configs.json && \
     mongoimport --db wcm-local --collection content --file /fixtures/content.json && \
     mongoimport --db wcm-local --collection licenses --file /fixtures/licenses.json && \
     mongoimport --db wcm-local --collection lists --file /fixtures/lists.json && \
     mongoimport --db wcm-local --collection properties --file /fixtures/properties.json && \
     mongoimport --db wcm-local --collection videos --file /fixtures/videos.json)

This is probably the cleanest way to do it.

* OLD WAY *

I created a shell script with my commands. In this case I wanted to start mongod, and run mongoimport but calling mongod blocks you from running the rest.

docker-compose.yaml:

version: '3'

services:
  mongo:
    container_name: mongo
    image: mongo:3.2.12
    ports:
      - "27017:27017"
    volumes:
      - ./fixtures:/fixtures
      - ./deploy:/deploy
      - ../.data/mongodb:/data/db
    command: sh /deploy/local/start_mongod.sh

start_mongod.sh:

mongod --fork --syslog && \
mongoimport --db wcm-local --collection clients --file /fixtures/clients.json && \
mongoimport --db wcm-local --collection configs --file /fixtures/configs.json && \
mongoimport --db wcm-local --collection content --file /fixtures/content.json && \
mongoimport --db wcm-local --collection licenses --file /fixtures/licenses.json && \
mongoimport --db wcm-local --collection lists --file /fixtures/lists.json && \
mongoimport --db wcm-local --collection properties --file /fixtures/properties.json && \
mongoimport --db wcm-local --collection videos --file /fixtures/videos.json && \
pkill -f mongod && \
sleep 2 && \
mongod

So this forks mongo, does monogimport and then kills the forked mongo which is detached, and starts it up again without detaching. Not sure if there is a way to attach to a forked process but this does work.

NOTE: If you strictly want to load some initial db data this is the way to do it:

mongo_import.sh

#!/bin/bash
# Import from fixtures

# Used in build and docker-compose mongo (different dirs)
DIRECTORY=../deploy/local/mongo_fixtures
if [[ -d "/fixtures" ]]; then
    DIRECTORY=/fixtures
fi
echo ${DIRECTORY}

mongoimport --db wcm-local --collection clients --file ${DIRECTORY}/clients.json && \
mongoimport --db wcm-local --collection configs --file ${DIRECTORY}/configs.json && \
mongoimport --db wcm-local --collection content --file ${DIRECTORY}/content.json && \
mongoimport --db wcm-local --collection licenses --file ${DIRECTORY}/licenses.json && \
mongoimport --db wcm-local --collection lists --file ${DIRECTORY}/lists.json && \
mongoimport --db wcm-local --collection properties --file ${DIRECTORY}/properties.json && \
mongoimport --db wcm-local --collection videos --file ${DIRECTORY}/videos.json

mongo_fixtures/*.json files were created via mongoexport command.

docker-compose.yaml

version: '3'

services:
  mongo:
    container_name: mongo
    image: mongo:3.2.12
    ports:
      - "27017:27017"
    volumes:
      - mongo-data:/data/db:cached
      - ./deploy/local/mongo_fixtures:/fixtures
      - ./deploy/local/mongo_import.sh:/docker-entrypoint-initdb.d/mongo_import.sh


volumes:
  mongo-data:
    driver: local

A

Amritpal Singh

To run multiple commands in the docker-compose file by using bash -c.

command: >
    bash -c "python manage.py makemigrations
    && python manage.py migrate
    && python manage.py runserver 0.0.0.0:8000"

Source: https://intellipaat.com/community/19590/docker-run-multiple-commands-using-docker-compose-at-once?show=19597#a19597

D

Display name

There are many great answers in this thread already, however, I found that a combination of a few of them seemed to work best, especially for Debian based users.

services:
  db:
    . . . 
  web:
    . . .
    depends_on:
       - "db"
    command: >      
      bash -c "./wait-for-it.sh db:5432 -- python manage.py makemigrations
      && python manage.py migrate
      && python manage.py runserver 0.0.0.0:8000"

Prerequisites: add wait-for-it.sh to your project directory.

Warning from the docs: "(When using wait-for-it.sh) in production, your database could become unavailable or move hosts at any time ... (This solution is for people that) don’t need this level of resilience."

Edit:

This is a cool short term fix but for a long term solution you should try using entrypoints in the Dockerfiles for each image.

G

GtdDev

tht's my solution for this problem:

services:
  mongo1:
    container_name: mongo1
    image: mongo:4.4.4
    restart: always
#    OPTION 01:
#    command: >
#      bash -c "chmod +x /scripts/rs-init.sh
#      && sh /scripts/rs-init.sh"
#    OPTION 02:
    entrypoint: [ "bash", "-c", "chmod +x /scripts/rs-init.sh && sh /scripts/rs-init.sh"]
    ports:
      - "9042:9042"
    networks:
      - mongo-cluster
    volumes:
      - ~/mongors/data1:/data/db
      - ./rs-init.sh:/scripts/rs-init.sh
      - api_vol:/data/db
    environment:
      *env-vars
    depends_on:
      - mongo2
      - mongo3

D

Dinko Pehar

Alpine-based images actually seem to have no bash installed, but you can use sh or ash which link to /bin/busybox.

Example docker-compose.yml:

version: "3"
services:

  api:
    restart: unless-stopped
    command: ash -c "flask models init && flask run"

C

Community

If you need to run more than one daemon process, there's a suggestion in the Docker documentation to use Supervisord in an un-detached mode so all the sub-daemons will output to the stdout.

From another SO question, I discovered you can redirect the child processes output to the stdout. That way you can see all the output!

Looking at this again, this answer seems more suited for running multiple commands in parallel instead of serially.

E

Eran

Use a tool such as wait-for-it or dockerize. These are small wrapper scripts which you can include in your application’s image. Or write your own wrapper script to perform a more application-specific commands. according to: https://docs.docker.com/compose/startup-order/

link to wait-for-it.sh. It should be noted (as it was in the docs you linked to) "The problem of waiting for a database (for example) to be ready is really just a subset of a much larger problem of distributed systems. In production, your database could become unavailable or move hosts at any time. Your application needs to be resilient to these types of failures." This solution is for people that "don’t need this level of resilience"

C

Cyril Duchon-Doris

Building on @Bjorn answer, docker has recently introduced special dependency rules that allows you to wait until the "init container" has exited successfully which gives

db:
  image: postgres
web:
  image: app
  command: python manage.py runserver 0.0.0.0:8000
  depends_on:
    db:
    migration:
      condition: service_completed_successfully
migration:
  build: .
  image: app
  command: python manage.py migrate
  depends_on:
    - db

I'm not sure if you still need buildkit or not, but on my side it works with

DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 docker-compose up

J

Jason Anderson

I ran into this while trying to get my jenkins container set up to build docker containers as the jenkins user.

I needed to touch the docker.sock file in the Dockerfile as i link it later on in the docker-compose file. Unless i touch'ed it first, it didn't yet exist. This worked for me.

Dockerfile:

USER root
RUN apt-get update && \
apt-get -y install apt-transport-https \
ca-certificates \
curl \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; 
echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable" && \
apt-get update && \
apt-get -y install docker-ce
RUN groupmod -g 492 docker && \
usermod -aG docker jenkins  && \
touch /var/run/docker.sock && \
chmod 777 /var/run/docker.sock

USER Jenkins

docker-compose.yml:

version: '3.3'
services:
jenkins_pipeline:
    build: .
    ports:
      - "8083:8083"
      - "50083:50080"
    volumes:
        - /root/pipeline/jenkins/mount_point_home:/var/jenkins_home
        - /var/run/docker.sock:/var/run/docker.sock

This seems like the answer to different question.

M

Mihir Bhende

I was having same problem where I wanted to run my react app on port 3000 and storybook on port 6006 both in the same containers.

I tried to start both as entrypoint commands from Dockerfile as well as using docker-compose command option.

After spending time on this, decided to separate these services into separate containers and it worked like charm

J

Jeremy Beale

In case anyone else is trying to figure out multiple commands with Windows based containers the following format works:
command: "cmd.exe /c call C:/Temp/script1.bat && dir && C:/Temp/script2.bat && ..."

Including the 'call' directive was what fixed it for me.

Alternatively if each command can execute without previous commands succeeding, just separate each with semicolons:
command: "cmd.exe /c call C:/Temp/script1.bat; dir; C:/Temp/script2.bat; ... "

c

chanllen

try using ";" to separate the commands if you are in verions two e.g.

command: "sleep 20; echo 'a'"

Using Docker-Compose, how to execute multiple commands

Follow WeChat

Want to stay one step ahead of the latest teleworks?

相似问题

Platform

Support

Contact US