-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Execute a command after run #1809
Comments
I think these should be steps in the Dockerfile
That way you also get it cached when you rebuild. |
Thanks @dnephin.
|
The point is that putting commands to Alternatively, start your app with a shell script or Makefile that runs the appropriate The functionality isn't worth adding to Compose unless it would add significant value over doing either of those, and I don't think it would for the use cases you've cited. |
So, to manage my docker, you suggest me to use a Script or a Makefile. So why compose was created Ok, I take this example, it's what I used to deploy my application testing environment in the CI process.
After container starts, I must provision mongodb, manage queue and account in rabbitmq What i'm doing today is a script with:
With
|
This would be rather useful and solves a use case. |
👍 |
It will make using |
👍 |
This would be very useful. I don't understand the argument of "oh you could do this with a bash script". Of course we could do it with a bash script. I could also do everything that Docker-compose does with a bash script. But the point is that there is one single YAML file that controls your test environment and it can be spun up with a simple |
It is not the remit of Compose to do everything that could be done with a shell script or Makefile - we have to draw a line somewhere to strike a balance between usefulness and avoiding bloat. Furthermore, one important property of the Compose file is that it's pretty portable across machines - even Mac, Linux and Windows machines. If we enable people to put arbitrary shell commands in the Compose file, they're going to get a lot less portable. |
@aanand To be fair, being able to execute a |
Apologies - I misread this issue as being about executing commands on the host machine. Still, my first point stands. |
I understand your point @aanand. It doesn't seem out of scope to me, since already |
@aanand the main problem for many devs and CI pipelines is to have a data very close to the production env. Like a dump from a DB. I create this ticket 1 year ago and nothing move in docker compose. So you suggest a Makefile or a Bashcript just to run some exec #1809 (comment) What I originally suggest is Finally, in my git repository I will have a compose file, a dockerfile and a makefile with idempotency management (may makefile could create a statefile). Genius! |
There's a big difference between There are already a couple ways to accomplish this with Compose (#1809 (comment)). Regarding the specific problem of dumping or loading data from a database, those are more "workflow" or "build automation" type tasks, that are generally done in a Makefile. I've been prototyping a tool for exactly those use-cases called dobi, which runs all tasks in containers. It also integrates very well with Compose. You might be interested in trying it out if you aren't happy with Makefiles. I'm working on an example of a database init/load use case. |
@dnephin Let's imagine.
If you need more examples:
Thanks for dobi, but if you need to create a tool to enhance compose, compose is bad and it's better to use a more powerfull tool. |
That's like saying "if you need applications to enhance your operating system, your OS is bad". No one tool should do everything. The unix philosophy is do one thing, and do it well. That is what we're doing here. Compose does its one thing "orchestrate containers for running an application". It is not a build automation tool. |
Wow I think that we reached the best bad faith.
So you can guarantee that we will never see "docker compose" wrote in Go inside in the docker monolithic binary to keep the unix philosophy ? https://www.orchardup.com/blog/orchard-is-joining-docker
|
So in short there is no way to do things like loading fixtures with compose..? I have to say I'm surprised.. |
@discordianfish, If, somehow, someone would wake up to the fact that CI/CD engineers need to be able to handle life cycle events and orchestration at least at a very basic level, then who knows docker/docker-compose may actually make its way out of local development pipelines and testing infrastructure and find a place in more production environments. I'm hopeful whoever is working on the stacks will address these issues, but I won't hold my breath. After all what needs to be done at build time may be different than what is needed at runtime, and is needed at runtime often varies by deployment environment... It is kind of annoying work to make my external scripts aware of whether an up is going to create or start containers... And those are things some lifecycle hooks + commands + environment variables could help with. You see it in service management frameworks and other orchestration tools... why not in docker-compose? |
You might be interested in https://github.com/dnephin/dobi , which is a tool I've been working on that was designed for those workflows. |
@dnephin stop spamming this issue with your tools. We see your comment before and the answer is the same. Makefile/bash is probably better than an nth "my tool enhance docker". |
Thank you for your constructive comment. I didn't realize that I had already mentioned dobi on this thread 8 months ago. If you're happy with Makefile/bash that's great! I'm glad your problem has been solved. |
Added a comment related to this topic here: #1341 (comment) |
@dnephin for this one, my comment can be applied:
In #1341, I "only" see a way to write in Use caseI have a custom NodeJS image and I want to run My problem is: the application code is no added in the container, it's mounted in it with a volume, defined in custom-node:
build: ../my_app-node/
tty: true
#command: bash -c "npm install && node"
volumes:
- /var/www/my_app:/usr/share/nginx/html/my_app so I can't run To run command: bash -c "npm install && node" which is not really clean 😞 and which I can't run on Alpine versions (they don't have Bash installed in it). I thought that Docker Compose would provide a way to run exec commands on containers, e.G.: custom-node:
build: ../my_app-node/
tty: true
command: node
volumes:
- /var/www/my_app:/usr/share/nginx/html/my_app
exec:
- npm install But it's not, and I think it's really missing! |
I expected compose is designed for testing, but I'm probably wrong and it's intended more for local development etc. I ran into several other rough edges like orphaned containers and the unclear relation between project name, path and how it's used to identify ownership, what happens if you have multiple compose files in the same directory etc etc. So all in all it doesn't seem like a good fit for CI. |
@lucile-sticky you can use It sounds like what you want is "build automation" which is not the role of docker-compose. Have you looked at dobi ? |
Two questions:
|
k8s != docker-compose. Wrong channel |
Sorry for not being clear, but my point was: Kubernetes supports this, and because Kubernetes and Docker compose have many of the same use cases/purposes, that would be an argument for having it in compose. Sorry if I was unclear. |
Good news!! I think docker has heard us, (on this issue and a few others). https://www.docker.com/blog/announcing-the-compose-specification/ Let's try to work on the specification there to fulfill the community needs. We can try to make this an open and friendly community with this restart. |
Has anyone suggested this change yet? Mailing list isn't available yet so I think the next best place is here: https://github.com/compose-spec/compose-spec I don't see an issue that describes this problem but not sure if that's the right place... Edit: I opened an issue at compose-spec/compose-spec#84. Please upvote it to show your support for the feature! |
You can use the HEALTHCHECK to do something else like following example: Code
|
@reduardo7 |
Many of these solutions are valid workarounds to this problem. For e.g. making an entrypoint script could also resolve this: which will include a more complex logic before running the actual service or process.
I know that not all the above are meaningful but I hope that you get the picture because this is the point.
or even like that:
|
I am unable to overwrite file which requires root permission using hook script or bootstrap script approach, since we start container as non root user |
Wow, such a basic functionality and still not implemented. |
This is workaround from @reduardo7 worked for me.
I have the following Dockerfile:
I need to run the following command (to reindex ElasticSearch) after the container starts:
but couldn't get it working with ENTRYPOINT or CMD. So wrote a bash script (script.sh)
, made it executable and added this block to the Dockerfile
which works. Question is, since HEALTHCHECK will will run the script every 5 s, is there a way to make HEALTHCHECK run only once and quit? |
The HEALTHCHECK runs every 5s, it's a Docker feature. But, with You need to define #/usr/bin/env bash
set -e
FIRST_READY_STATUS_FLAG='/tmp/.FIRST_READY_STATUS_FLAG'
# Health check
echo 'Run command to validate the container status HERE'
# On success
if [ ! -f "${FIRST_READY_STATUS_FLAG}" ]; then
# On first success...
touch "${FIRST_READY_STATUS_FLAG}"
# Run ON_RUN on first health check ok
perl koha/misc/search_tools/rebuild_elasticsearch.pl -d
fi |
Thank you @reduardo7, I'll try this out tomorrow and let you know how it goes. Much appreciated! |
I created a repository with examples about how to implement this required feature: |
Why this has to be so complicated? |
@Luc45 this is the next challenge after landing in Mars |
It will certainly take a similar amount of time ;) |
This would be a nice feature. I have a case where I need to run a single command after starting PHP service and don't want to add a bash script wrapper or anything like that. |
@superswan not sure (and also never tried), but is this working for you: #1510 (comment) |
It's 2022, and yet we don't have an elegant way to do this. |
This is not we want, as #1809 (comment) said, we want something that could only be executed once, just like initializing the database or run |
March 2023 and I just wanted to alter a user in a docker-compose after I up some database... but fine, bash script! |
Those interested with this topic could help us providing feedback on compose-spec/compose-spec#289 |
The solution I've landed at for at least the docker compose up scenarios are task containers that setup my databases etc. Here is an example froma django environment # re-usable yaml anchors
x-task: &task
# docker stack deploy restart policy for tasks
deploy:
restart_policy:
condition: on-failure
replicas: 1
# docker compose stand-alone restart policy for tasks
restart: on-failure
x-task-ensure-database: &task-ensure-database
<<: *task
# image: this task must extend a service with a postgres image.
entrypoint: |
sh -c "
psql -v ON_ERROR_STOP=0 --host postgres --username \"$${POSTGRES_USER}\" <<EOSQL
CREATE ROLE $${SERVICE_NAME} LOGIN PASSWORD '$${SERVICE_PASS}';
CREATE DATABASE $${SERVICE_NAME};
GRANT ALL ON DATABASE $${SERVICE_NAME} TO $${SERVICE_NAME};
EOSQL"
# you must set the following environment variables where you use this anchor.
# environment:
# - POSTGRES_USER
# - PGPASSWORD - superuser password
# USERNAME and DATABASE to create.
# - SERVICE_NAME=UserAndDbName
# - SERVICE_PASS=password
x-service: &service
# docker stack deploy restart policy for tasks
deploy:
restart_policy:
condition: unless-stopped
replicas: 1
# docker compose stand-alone restart policy for tasks
restart: unless-stopped
x-postgres: &postgres
image: postgres:14
environment:
- POSTGRES_USER=example
- POSTGRES_PASSWORD=example
volumes:
postgres-data:
cms-media:
services:
postgres:
<<: *service
<<: *postgres
# we're only exposing here, since this is only called from another service.
expose:
- "5432"
# uncomment below if you wish to access this db directly from the host.
ports:
- "5432:5432"
volumes:
- postgres-data:/var/lib/postgresql
api-ensure-database:
<<: *postgres
<<: *task-ensure-database
environment:
- POSTGRES_USER=example
- PGPASSWORD=example
- SERVICE_NAME=api
- SERVICE_PASS=password
depends_on:
- postgres
api-ensure-database will keep trying to run until it succeeds. The 'task' services fail and respawn a lot while waiting on their dependencies which isn't ideal, but it does get the job done. |
Hi,
It will be very helpful to have something like "onrun" in the YAML to be able to run commands after the run. Similar to moby/moby#8860
After the mongodb start, It will dump db2dump.domain.lan and restore it.
When I will stop and then start the container, onrun part will no be executed to preserve idempotency.
EDIT 15 June 2020
5 years later, Compose wan't to "standardize" specifications,
please check compose-spec/compose-spec#84
The text was updated successfully, but these errors were encountered: