how to get docker-compose to use the latest image from repository

in order to make sure, that you are using the latest version for your :latest tag from your registry (e.g. docker hub) you need to also pull the latest tag again. in case it changed, the diff will be downloaded and started when you docker-compose up again.

so this would be the way to go:

docker-compose stop
docker-compose rm -f
docker-compose pull   
docker-compose up -d

i glued this into an image that i run to start docker-compose and make sure images stay up-to-date: https://hub.docker.com/r/stephanlindauer/docker-compose-updater/

---

To get the latest images use docker-compose build --pull

I use below command which is really 3 in 1

docker-compose down && docker-compose build --pull && docker-compose up -d

This command will stop the services, pulls the latest image and then starts the services.

---

[EDIT] DO NOT USE The below is in the spec but, as pointed out in the comments, has not been implemented yet (docker-compose 1.29.2 and 2).

Since 2020-05-07, the docker-compose spec also defines the "pull_policy" property for a service:

version: '3.7'

services:
  my-service:
    image: someimage/somewhere
    pull_policy: always

The docker-compose spec says:

pull_policy defines the decisions Compose implementations will make when it starts to pull images.

Possible values are (tl;dr, check spec for more details):

always: always pull

never: don't pull (breaks if the image can not be found)

missing: pulls if the image is not cached

build: always build or rebuild

---

I use the following even if my container is running and it updates just fine.

docker-compose pull
docker-compose up -d

---

To close this question, what seemed to have worked is indeed running

docker-compose stop
docker-compose rm -f
docker-compose -f docker-compose.yml up -d

I.e. remove the containers before running up again.

What one needs to keep in mind when doing it like this is that data volume containers are removed as well if you just run rm -f. In order to prevent that I specify explicitly each container to remove:

docker-compose rm -f application nginx php

As I said in my question, I don't know if this is the correct process. But this seems to work for our use case, so until we find a better solution we'll roll with this one.

---

Pull new Image: docker-compose pull

Rebuild docker container with new Image: docker-compose up --force-recreate --build -d

Delete unused Images: docker image prune -f

---

Option down resolve this problem

I run my compose file:

docker-compose -f docker/docker-compose.yml up -d

then I delete all with down --rmi all

docker-compose -f docker/docker-compose.yml down --rmi all

Stops containers and removes containers, networks, volumes, and images
created by `up`.

By default, the only things removed are:

- Containers for services defined in the Compose file
- Networks defined in the `networks` section of the Compose file
- The default network, if one is used

Networks and volumes defined as `external` are never removed.

Usage: down [options]

Options:
    --rmi type          Remove images. Type must be one of:
                        'all': Remove all images used by any service.
                        'local': Remove only images that don't have a custom tag
                        set by the `image` field.
    -v, --volumes       Remove named volumes declared in the `volumes` section
                        of the Compose file and anonymous volumes
                        attached to containers.
    --remove-orphans    Remove containers for services not defined in the
                        Compose file

---

I spent half a day with this problem. The reason was that be sure to check where the volume was recorded.

volumes: - api-data:/src/patterns

But the fact is that in this place was the code that we changed. But when updating the docker, the code did not change.

Therefore, if you are checking someone else's code and for some reason you are not updating, check this.

And so in general this approach works:

docker-compose down docker-compose build docker-compose up -d

---

docker-compose pull
docker-compose up -d
docker image prune -af

It works for me.

Pull the newest image first, Than update the container, Finally clean images that we don't need.

---

I've seen this occur in our 7-8 docker production system. Another solution that worked for me in production was to run

docker-compose down
docker-compose up -d

this removes the containers and seems to make 'up' create new ones from the latest image.

This doesn't yet solve my dream of down+up per EACH changed container (serially, less down time), but it works to force 'up' to update the containers.

---

If the docker compose configuration is in a file, simply run:

docker-compose -f appName.yml down && docker-compose -f appName.yml pull && docker-compose -f appName.yml up -d

---

But

https://docs.docker.com/compose/reference/up/ -quiet-pull Pull without printing progress information

docker-compose up --quiet-pull

not work ?

---

Your docker-compose.yml uses image to refer to ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev. That means that running docker-compose build doesn’t take any action in regards to ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev. There must be commands like the following run somewhere:

docker build -t ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev
docker push ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev

This would put the versioned build into your repository. But it would not update its latest tag. That would still point to the prior version (e.g., 0.1.7-dev).

Then, you ran the other commands on your jenkins:

cd $WORKSPACE && PROJECT_VERSION=$(cat VERSION)-dev
/usr/local/bin/docker-compose rm -f
/usr/local/bin/docker-compose build
docker tag ourprivate.docker.reg:5000/ourcompany/buildchaintest ourprivate.docker.reg:5000/ourcompany/buildchaintest:$PROJECT_VERSION
docker push ourprivate.docker.reg:5000/ourcompany/buildchaintest

As part of docker-compose build, you are effectively running docker pull ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev on your server. However, that does not account for how the tag ourprivate.docker.reg:5000/ourcompany/buildchaintest:latest (which you refer to in the command docker tag ourprivate.docker.reg:5000/ourcompany/buildchaintest ourprivate.docker.reg:5000/ourcompany/buildchaintest:$PROJECT_VERSION) got onto your server. If the latest tag was not defined by something else beforehand, that docker tag command would give this error: No such image: ourprivate.docker.reg:5000/ourcompany/buildchaintest:latest. So, prior to running the commands you showed, your jenkins server must have run something like the following:

docker pull ourprivate.docker.reg:5000/ourcompany/buildchaintest

This command would cause it to pull whatever is set to latest at that time in your repository. Because you gave no proof that you ever pushed your newly build buildchaintest:0.1.8-dev as buildchaintest:latest prior to running the above docker pull command, we have to assume that this is where the old version comes from. As a result, your tag command effectively does this:

docker tag ourprivate.docker.reg:5000/ourcompany/buildchaintest:latest ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev
docker push ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev

Which, if latest is pointing to the same thing at 0.1.7-dev, is the same as doing:

docker tag ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.7-dev ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev
docker push ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev

Then, later, when you run docker-compose pull && docker-compose up -d, of course it will pull what is set in the repository as ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev which should be the old version because you tagged 0.1.7-dev as 0.1.8-dev.

Notes

You seem to be assuming that docker-compose build does something related to building buildchaintest. However, it does not. You set image but not build in your docker-compose.yml. That means docker-compose build will not build the buildchaintest docker image—it will just directly pull and run the docker image tag. Since docker-compose build doesn’t attempt to build buildchaintest, there’s no point to trying to tag or push that image after invoking it.

Also, docker-compose push exists and would do all of the tagging and pushing of things if you specify both build and image (because then image is used to inform docker-compose that it should set a tag for the built thing). This would push the correct tags (and would correctly be a no-op for your docker-compose.yml file).

Your docker-compose has build in it without image. Those things will be built wherever you run docker-compose up. So there is no point (unless you add image to set tags and save those built images and use docker-compose push) in running docker-compose build on the build server (beyond testing that building runs to completion). You’re not saving yourself any time or reducing the work performed by servers during deployment. You’re increasing the number of times things are built and the variability between nodes in your system (as whatever is built on your build server will be different (e.g., file timestamps, updated image tags) than whatever is running on your server.

Unfortunately, your question is not trustworthy. You use commands as if they have different behaviors than they are supposed to have. I am sorry to attack your question, but I have learned some surprising (to me) things about how docker tag and docker-compose work as a result. Thanks!

tl;dr

Make sure that you actually tag and push the correct image into your repository!

---

The docker-compose documentation for the 'up' command clearly states that it updates the container should the image be changed since the last 'up' was performed:

If there are existing containers for a service, and the service’s configuration or image was changed after the container’s creation, docker-compose up picks up the changes by stopping and recreating the containers (preserving mounted volumes).

So by using 'stop' followed by 'pull' and then 'up' this should therefore avoid issues of lost volumes for the running containers, except of course, for containers whose images have been updated.

I am currently experimenting with this process and will include my results in this comment shortly.

---

I am using following command to get latest images

sudo docker-compose down -rmi all

sudo docker-compose up -d

---