[BUG] 2.32.0 unnecessarily recreates containers and loses volumes

[sayhiben]

Description

My team manages a Django application backed by MySQL. In the development environment, each service runs in a separate container orchestrated by Docker Compose on GitHub Codespaces using their Docker-in-Docker devcontainer feature. This devcontainer feature defaults to the latest version of Docker Compose when creating new Codespaces. As of this morning, we have been unable to persist data in MySQL's attached volume, and the container seems to restart and/or be recreated unnecessarily

Unfortunately, I haven't yet had time to come up with a solid explanation or understanding of what's happening, so I don't have good repro steps. That said, the issue is severe enough for my org. that I wanted to report the problem here before I stopped work for the weekend

Here's the MySQL Docker image we pull, where the database volume is defined: https://github.com/docker-library/mysql/blob/090eb25ac69bca920fc5320484bc35aac92a8143/8.0/Dockerfile.debian

Our in-house reproduction steps have been to:

1. Get a shell_plus session in our Django app inside the container
2. Create instances of our models and save them
3. Exit the shell_plus session and container
4. Get another shell_plus session
5. Try to load the previously created instance from MySQL
6. Data miss

After manually downgrading back to Docker Compose 2.31.0, I can no longer reproduce the data persistence issue and see fewer recreates/restarts when running docker compose commands.

I apologize for the lack of specifics in this report, but again, given the instability, I figured that perhaps Docker Compose maintainers would want to hear from a canary sooner than later

Steps To Reproduce

I'm unsure how reproducible this is outside my specific set of Dockerfiles and Docker Compose configurations (which I am disallowed to share), but I'd test with using a MySQL 8 image based on mysql:8-debian and see if you can persist data in the volume it creates using Docker Compose 2.32.0.

I will try to give this some time on Monday and post a minimal reproducible example if I find it

Compose Version

2.32.0

Docker Environment

Client:
 Version:    27.3.1-1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.19.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 11
  Running: 11
  Paused: 0
  Stopped: 0
 Images: 15
 Server Version: 27.3.1-1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 88c3d9bc5b5a193f40b7c14fa996d23532d6f956
 runc version: bc20cb4497af9af01bea4a8044f1678ffca2745c
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-1025-azure
 Operating System: Ubuntu 20.04.6 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 62.78GiB
 Name: codespaces-151e8e
 ID: 6745412a-81e5-4daa-b556-7bf87382b12b
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: codespacesdev
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

• docker-in-docker devcontainer feature

[takicichy]

For me 2.32.0 running on Linux recreates container which has only bind mounts defined every time i do "docker-compose up -d", even when nothing changed.

Sample docker-compose.yml:

services:
  changer:
    image: ealen/echo-server
    container_name: changer-container
    volumes:
      - ./changer-vol:/changervol

I change nothing but each time docker compose up -d is run then the container gets recreated:

a@a~/composetest $ docker compose --progress plain up -d
 Container changer-container  Recreate
 Container changer-container  Recreated
 Container changer-container  Starting
 Container changer-container  Started
a@a~/composetest $ docker compose --progress plain up -d
 Container changer-container  Recreate
 Container changer-container  Recreated
 Container changer-container  Starting
 Container changer-container  Started

[strasharo]

For me 2.32.0 running on Linux recreates container which has only bind mounts defined every time i do "docker-compose up -d", even when nothing changed.

Sample docker-compose.yml:

services:
  changer:
    image: ealen/echo-server
    container_name: changer-container
    volumes:
      - ./changer-vol:/changervol

I change nothing but each time docker compose up -d is run then the container gets recreated:

a@a~/composetest $ docker compose --progress plain up -d
 Container changer-container  Recreate
 Container changer-container  Recreated
 Container changer-container  Starting
 Container changer-container  Started
a@a~/composetest $ docker compose --progress plain up -d
 Container changer-container  Recreate
 Container changer-container  Recreated
 Container changer-container  Starting
 Container changer-container  Started

Same here with docker-compose-2.32.0-1 running on Arch Linux. It keeps on recreating all containers when there are no changes at all. Rolled back to docker-compose-2.31.0-1-x86_64.pkg.tar.zst and the issue is not present there, this appeared with the recent update to 2.32

[ndeloof]

Sounds like some unexpected side effect introduced by #12363
I'll look into a fix ASAP