Expose image cache option in DSL (and other POD/container configuration options).

[gaoning777]

Argo by default reuses images, which might cause unexpected errors. For example, users author a component and update the component while the pipeline is still using the old image cached by argo. Maybe the DSL should provide an option to not use the cached image.
WDYT

[gaoning777]

@Ark-kun @qimingj

[paveldournov]

Can we use strong hashes to avoid image name collisions instead? Assuming mutable images can be dangerous and lead to hard to debug bugs.

[gaoning777]

Sure. For the sample components, we could certainly use the hashes. In fact, This PR(#571) changed the sample components to hashes.
On the other hand, this issue is also about discussing whether the dsl should provide configurations whether to pull image or not in case the users prefer image tags to hashes.

[vicaire]

It makes sense to me to expose POD/container features in the DSL. Is there a way we could expose all these features all at once?

[gaoning777]

we already support the imagepullpolicy:

pipelines/sdk/python/kfp/dsl/_container_op.py

Line 370 in 895645c

def set_image_pull_policy(self, image_pull_policy):

[jameswnl]

I've enabled the set_image_pull_policy('Always') and I can see the compiled yaml now has the imagePullPolicy: Always
However, the image is still not updated in the pipeline run. Here are how I've checked

• docker pull the latest image from docker hub and do a attach to the container image to make sure it has the latest commit
• A git show --summary in the command to dump last commit to log when the component is run and the commit is the older one when the pipeline is first run with.

I am using minikf 20190521.1.0
Here is the excerpt of the compiled dsl yaml:

  - container:
      args:
      - git show --summary && ls -al $0 && ls $1 && python start.py --input_path $0/$2
        --output_path $0 --job_id 1
      - /mnt
      - '{{inputs.parameters.collection-data}}'
      - 1212729.json
      command:
      - sh
      - -c
      image: my_repo/my_image:latest
      imagePullPolicy: Always

[gaoning777]

I'm not exactly following. How did you validate that the image being actually used is not the latest one?

[jameswnl]

I'm not exactly following. How did you validate that the image being actually used is not the latest one?

I have a 'git show --summary which will print the latest commit in the log.

def data_collection():
    return dsl.ContainerOp(
        name='Data collection',
        image='repo/data-collector:next',
        command=['sh', '-c'],
        arguments=['git show --summary && touch /app/results.txt && ls -l $0 ', mount],
        file_outputs={
            'data': '/app/results.txt',
        },
        pvolumes={mount: vop.volume}
    )