Adds flag modifying pull behavior for running and creating containers #1498

zmackie · 2018-11-05T18:16:20Z

Should close issue moby/moby#34394

- What I did

This PR adds a new --pull flag to docker run and docker create, following the proposal in (Proposal: add explicit "--pull" flag to "docker run" for controlling image pulling behavior moby/moby#34394)
Per this proposal, the flag is tristate:
1. --pull=missing (this is the current behaviour and will be the default.)
2. --pull=never
3. --pull=always

- How I did it

Adds a new config field, createOptions.pull which forks the execution of container.createContainer to either pull the image if it does not exist at all locally --pull=missing, always try and update the image --pull=always, or never try and update the image, only using images that already exist on the local machine --pull=never

Both docker run and docker create can consume this flag.

- How to verify it

Run docker run and docker create with the flag. Notice that pulls images or does not, depending on the behavior specified in the flag. Running those commands without the flag should maintain the current behavior.

- Description for the changelog

Adds flag modifying pull behavior for running and creating containers (Pull if Missing, Pull Always, Pull Never)

- A picture of a cute animal (not mandatory but encouraged)

zmackie · 2018-11-05T18:17:43Z

Hey docker folks. This is my first PR to the project (👋 ). It's likely deficient on tests (I especially wasn't sure about any kind of integration or e2e testing that needed to happen. Let me know what ya'll think.

codecov-io · 2018-11-06T14:16:22Z

Codecov Report

Merging #1498 into master will increase coverage by 0.01%.
The diff coverage is 61.11%.

@@            Coverage Diff             @@
##           master    #1498      +/-   ##
==========================================
+ Coverage   56.75%   56.77%   +0.01%     
==========================================
  Files         309      309              
  Lines       21658    21668      +10     
==========================================
+ Hits        12292    12301       +9     
+ Misses       8469     8468       -1     
- Partials      897      899       +2

lanoxx · 2018-11-12T09:01:27Z

Nice work. I hope the docker folks will merge this.

cli/command/container/create.go

GordonTheTurtle · 2018-11-20T11:54:30Z

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:

$ git clone -b "34394-run-pull-flag" git@github.com:Zanadar/cli.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354135168
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

Using the pull command explicitly until the docker CLI supports this natively. re docker/cli#1498 [#162511937]

svenstaro · 2019-01-22T17:03:39Z

What's the work remaining in order to get this merged?

zmackie · 2019-01-22T17:11:50Z

AFAIK nothing (but it’s been a while since I opened this so I may have forgotten). Maybe project folks have a different opinion?

svenstaro · 2019-01-22T17:50:53Z

@GordonTheTurtle could you take a look?

Morl99 · 2019-02-11T10:58:49Z

@Zanadar can you try to increase the test coverage, you are slightly falling short, as per the above Check. I would love to see this feature merged, it is really useful!

cpuguy83 · 2019-02-14T18:48:16Z

Overall this looks good, but I think we can clean up the implementation a bit.

cpuguy83 · 2019-02-14T18:49:16Z

Also keep in mind pull-policy=always only really works for moving tags (such as latest).

cpuguy83

Should we error out early if pull=always and the pull failed?

cpuguy83 · 2019-02-14T18:51:24Z

cli/command/container/create.go

@@ -194,31 +205,59 @@ func createContainer(ctx context.Context, dockerCli command.Cli, containerConfig
 	}

 	//create the container
-	response, err := dockerCli.Client().ContainerCreate(ctx, config, hostConfig, networkingConfig, opts.name)
+	var response container.ContainerCreateCreatedBody
+	if opts.pull == PullImageMissing { // Pull image only if it does not exist locally. Default.


I think we can do this flow and it will clean up quite a bit:

warning, psuedo-code:

if PullImageAlways { pull() } err := createContainer() if IsNotfound(err) && PullImageMissing { pull() } createContainer()

Latest commit improves the flow per suggestion, but I'm not sure I've got it quite right...will work at it a bit.

One thing I'm not certain on is whether we should error in the case that the image is not found locally and the "never pull" option is selected? In which case the flow becomes more like so:

if PullImageAlways { pull() } err := createContainer() if err { if IsNotfound(err) && PullImageMissing { pull() createContainer() } return err // we won't be pulling so just return whatever err we got }

This also saves trying to create the container twice in the "never pull" case (ie only try to create again in the case where we might pull).

zmackie · 2019-02-16T13:26:41Z

Also keep in mind pull-policy=always only really works for moving tags (such as latest).

That's probably something that would have to be left to users, though, right? We can't necessarily know whether a tag is moving or not.

cpuguy83 · 2019-02-16T14:39:41Z

Right.

zmackie · 2019-02-17T14:46:43Z

Should we error out early if pull=always and the pull failed?

This is now handled in the first block: https://github.com/docker/cli/pull/1498/files#diff-477f60a1b609e77fdca573fd8bd7b0c9R210

zmackie · 2019-02-17T15:04:44Z

@cpuguy83 Okay I cleaned things up a fair bit. I'd love a review pass on the tests and your thoughts on how we handle the error case when "never pull" is selected (see above).

robbaman · 2020-01-21T22:58:13Z

Is there a reason why the latest version of Docker Desktop on Windows still has version 19.03 of the CLI? According to this thread it seems 19.09 should've been worked on/ready for about 6 months already.

jcchavezs · 2020-03-02T20:09:29Z

Could the default behaviour be overriden by an env var? I could come up with a PR.

gitnik · 2020-04-22T10:02:20Z

This is still not available in engine version 19.03.08 in Docker for Mac. Any estimate when this will be available?

thaJeztah · 2020-04-22T14:14:27Z

Next release (20.xx)

nickgrealy · 2020-07-23T02:33:09Z

Two (stupid?) questions:

Is it possible to have this work with docker pull <someimage> commands too?

e.g. docker pull --pull=always nginx

Is it possible to have --pull=newer - to pull the latest version of an image, only if it's newer in the repo?

FernandoMiguel · 2020-07-23T05:34:02Z

Two (stupid?) questions:

Is it possible to have this work with docker pull <someimage> commands too?

e.g. docker pull --pull=always nginx

Is it possible to have --pull=newer - to pull the latest version of an image, only if it's newer in the repo?

I don't understand your first one.
The second one is the behaviour of pull.

robbaman · 2020-09-30T18:30:44Z

Jut got a new version of Docker Desktop for Windows which includes a new version.... 19.03.13

Is 19.09 ever coming to Windows too?

Docker version 19.09 added some new pull logic[1] to the run command that could be used here [1]: docker/cli#1498

- docker-compose doesn't automatically pulll latest container versions, which can result in unexpectedly old containers being used - Despite requests for years to add this support to docker-compose it hasn't been added. see: docker/compose#3574 - Eventually this will make its way in as part of the work in docker/cli#1498

GordonTheTurtle added the status/0-triage label Nov 5, 2018

zmackie force-pushed the 34394-run-pull-flag branch from 0934964 to 58517d0 Compare November 6, 2018 14:16

This was referenced Nov 6, 2018

docker run doesn't pull down latest image if the image exists locally moby/moby#13331

Closed

Proposal: add explicit "--pull" flag to "docker run" for controlling image pulling behavior moby/moby#34394

Closed

shin- mentioned this pull request Nov 14, 2018

docker-compose up doesn't pull down latest image if the image exists locally docker/compose#3574

Closed

zmackie force-pushed the 34394-run-pull-flag branch 2 times, most recently from a43abec to 05ecd71 Compare November 15, 2018 11:59

rsjethani reviewed Nov 20, 2018

View reviewed changes

cli/command/container/create.go Outdated Show resolved Hide resolved

GordonTheTurtle added the dco/no label Nov 20, 2018

zmackie force-pushed the 34394-run-pull-flag branch from f31fec0 to 053b7b4 Compare November 20, 2018 11:58

GordonTheTurtle removed the dco/no label Nov 20, 2018

gerhard added a commit to thechangelog/changelog.com that referenced this pull request Dec 31, 2018

Pull latest thechangelog/bootstrap before running a new container

077238e

Using the pull command explicitly until the docker CLI supports this natively. re docker/cli#1498 [#162511937]

ilyaiqoqo approved these changes Jan 15, 2019

View reviewed changes

henrik242 approved these changes Jan 22, 2019

View reviewed changes

cpuguy83 requested changes Feb 14, 2019

View reviewed changes

zmackie force-pushed the 34394-run-pull-flag branch from f0b476c to a281095 Compare February 17, 2019 14:44

thaJeztah mentioned this pull request Jun 15, 2023

[proposal] implement pull-policies (docker build --pull=missing|never|always) docker/buildx#1889

Open

This was referenced Jan 30, 2020

Add a quiet option to docker pull moby/moby#13588

Closed

Add --pull arg to "up" command docker/compose#7213

Closed

nejch mentioned this pull request Feb 19, 2020

chore: ensure developers use same gitlab image as CI python-gitlab/python-gitlab#1022

Merged

jcchavezs mentioned this pull request Mar 2, 2020

Make the --pull behaviour being configurable over env var #2370

Closed

jamshid mentioned this pull request May 9, 2020

Option on docker-compose up to do a pull even if it already exists docker/compose#7437

Closed

This was referenced Jun 3, 2020

Provide an option to prevent automatic pulling moby/moby#41064

Open

(re)introduce some of the compose v2 attributes compose-spec/compose-spec#78

Merged

tianon mentioned this pull request Sep 11, 2020

Proposal: deprecate "docker run" pulling unknown containers and add explicit "--pull" flag moby/moby#10693

Closed

johanneswuerbach mentioned this pull request Oct 26, 2020

docker run with sha #2815

Closed

gesellix mentioned this pull request Dec 9, 2020

"No such image: testcontainers/ryuk:0.3.0" testcontainers/testcontainers-java#3574

Closed

albers mentioned this pull request Jan 5, 2021

Add bash completion for docker run|create --pull #2912

Merged

daniel-dona mentioned this pull request Feb 1, 2021

Docker "--pull always" option oeg-upm/gtfs-bench#123

Closed

greyltc added a commit to greyltc/valheim-server-docker that referenced this pull request Mar 19, 2021

use --pull=always

333c4fb

Docker version 19.09 added some new pull logic[1] to the run command that could be used here [1]: docker/cli#1498

greyltc mentioned this pull request Mar 19, 2021

use --pull=always lloesche/valheim-server-docker#241

Merged

lloesche pushed a commit to lloesche/valheim-server-docker that referenced this pull request Mar 19, 2021

use --pull=always (#241)

7e9ac29

Docker version 19.09 added some new pull logic[1] to the run command that could be used here [1]: docker/cli#1498

frankfarzan mentioned this pull request May 18, 2021

Support multiple image pull policy kptdev/kpt#1888

Merged

aochagavia mentioned this pull request Jan 17, 2022

Missing documentation for existing docker run flag docker/docs#14135

Closed

thaJeztah mentioned this pull request Mar 1, 2022

docs: add missing documentation for --pull flag #3449

Merged

vogler mentioned this pull request Jan 28, 2023

How does one update the docker container? vogler/free-games-claimer#51

Closed

burak-ok mentioned this pull request Oct 19, 2023

run: Implement --pull argument inspektor-gadget/inspektor-gadget#2101

Closed

smira mentioned this pull request Jan 31, 2024

docs: document the process of building custom kernel packages siderolabs/talos#8220

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds flag modifying pull behavior for running and creating containers #1498

Adds flag modifying pull behavior for running and creating containers #1498

zmackie commented Nov 5, 2018

zmackie commented Nov 5, 2018 •

edited

Loading

codecov-io commented Nov 6, 2018 •

edited

Loading

lanoxx commented Nov 12, 2018

GordonTheTurtle commented Nov 20, 2018

svenstaro commented Jan 22, 2019

zmackie commented Jan 22, 2019

svenstaro commented Jan 22, 2019

Morl99 commented Feb 11, 2019

cpuguy83 commented Feb 14, 2019

cpuguy83 commented Feb 14, 2019

cpuguy83 left a comment

cpuguy83 Feb 14, 2019

zmackie Feb 16, 2019 •

edited

Loading

zmackie Feb 17, 2019

zmackie commented Feb 16, 2019

cpuguy83 commented Feb 16, 2019

zmackie commented Feb 17, 2019

zmackie commented Feb 17, 2019 •

edited

Loading

robbaman commented Jan 21, 2020

jcchavezs commented Mar 2, 2020

gitnik commented Apr 22, 2020

thaJeztah commented Apr 22, 2020

nickgrealy commented Jul 23, 2020 •

edited

Loading

FernandoMiguel commented Jul 23, 2020

robbaman commented Sep 30, 2020

Adds flag modifying pull behavior for running and creating containers #1498

Adds flag modifying pull behavior for running and creating containers #1498

Conversation

zmackie commented Nov 5, 2018

zmackie commented Nov 5, 2018 • edited Loading

codecov-io commented Nov 6, 2018 • edited Loading

Codecov Report

lanoxx commented Nov 12, 2018

GordonTheTurtle commented Nov 20, 2018

svenstaro commented Jan 22, 2019

zmackie commented Jan 22, 2019

svenstaro commented Jan 22, 2019

Morl99 commented Feb 11, 2019

cpuguy83 commented Feb 14, 2019

cpuguy83 commented Feb 14, 2019

cpuguy83 left a comment

Choose a reason for hiding this comment

cpuguy83 Feb 14, 2019

Choose a reason for hiding this comment

zmackie Feb 16, 2019 • edited Loading

Choose a reason for hiding this comment

zmackie Feb 17, 2019

Choose a reason for hiding this comment

zmackie commented Feb 16, 2019

cpuguy83 commented Feb 16, 2019

zmackie commented Feb 17, 2019

zmackie commented Feb 17, 2019 • edited Loading

robbaman commented Jan 21, 2020

jcchavezs commented Mar 2, 2020

gitnik commented Apr 22, 2020

thaJeztah commented Apr 22, 2020

nickgrealy commented Jul 23, 2020 • edited Loading

FernandoMiguel commented Jul 23, 2020

robbaman commented Sep 30, 2020

zmackie commented Nov 5, 2018 •

edited

Loading

codecov-io commented Nov 6, 2018 •

edited

Loading

zmackie Feb 16, 2019 •

edited

Loading

zmackie commented Feb 17, 2019 •

edited

Loading

nickgrealy commented Jul 23, 2020 •

edited

Loading