Add progress output to oras copy

[sajayantony]

What is the version of your ORAS CLI

1.0.0

What would you like to be added?

1. Currently when executing copy there is no output for quite some time.

   For e.g. when copying a manifest list it would be good to indicate progress that a manifest list has been found and the copy start

2. Verbose gives much more information but these are short hashes and it would good to show fully qualified names so that the user can copy the output and maybe try copying just one manifest incase of failure

3. The statement Exists {sha} seems odd to me. It might better read {digest} exists

4. Before the error the manifest that caused the error for the blob should be shown.

oras copy docker.io/library/hello-world:latest ttl.sh/$IMAGE:2h --verbose
Exists  40d0cfd08617 application/vnd.docker.distribution.manifest.v2+json
Exists  f54a58bc1aac application/vnd.docker.distribution.manifest.v2+json
...
Exists  c4a09ae78232 application/vnd.docker.container.image.v1+json
Error: PUT "https://ttl.sh/v2/745b7f5d-0b42-42fd-878f-61e7900ac774/blobs/uploads/3dbbf735-6a7b-415a-9ace-22224a3b3312?_state=7lO8m8I-zjp5SiHaMB8vsGqVeyDGo7E94XjuaiVO6SZ7Ik5hbWUiOiI3NDViN2Y1ZC0wYjQyLTQyZmQtODc4Zi02MWU3OTAwYWM3NzQiLCJVVUlEIjoiM2RiYmY3MzUtNmE3Yi00MTVhLTlhY2UtMjIyMjRhM2IzMzEyIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIzLTA1LTAzVDIwOjIxOjE1Ljc2Mjc0NDUzNVoifQ%3D%3D&digest=sha256%3Ae06b772d586b58466a653b72002aee7c59496110e9ae402ff58f026e44452506": response status code 413: Request Entity Too Large

For e.g in the last line it makes it hard to try copying just the last manifest. Ideally this output should come out in copy without verbose as well and should show what the last manifest is..

❯ oras copy docker.io/library/hello-world:latest ttl.sh/$IMAGE:2h
Error: PUT "https://ttl.sh/v2/745b7f5d-0b42-42fd-878f-61e7900ac774/blobs/uploads/51c4a02e-da0c-4153-ba9f-a29e42c7191d?_state=eXbH9Stpob5J5RFBg4BP56bcq-k_ivc_qcHYMVfXezl7Ik5hbWUiOiI3NDViN2Y1ZC0wYjQyLTQyZmQtODc4Zi02MWU3OTAwYWM3NzQiLCJVVUlEIjoiNTFjNGEwMmUtZGEwYy00MTUzLWJhOWYtYTI5ZTQyYzcxOTFkIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIzLTA1LTAzVDIwOjE5OjA4LjU2ODk5OTQyNFoifQ%3D%3D&digest=sha256%3Ae06b772d586b58466a653b72002aee7c59496110e9ae402ff58f026e44452506": response status code 413: Request Entity Too Large

Proposed output

The output below shows a possible idea on how the user can see what is being copied with the type, digest and mediaType

$ oras copy docker.io/library/hello-world:latest ttl.sh/$IMAGE:2h
Copying index docker.io/library/hello-world:latest sha256:4e83453afed1b4fa1a3500525091dbfca6ce1e66903fd4c01ff015dbcb1ba33e  application/vnd.docker.distribution.manifest.list.v2+json
Copying manifest sha256:eb11b1a194ff8e236a01eff392c4e1296a53b0fb4780d8b0382f7996a15d5392 
 application\/vnd.docker.distribution.manifest.v2+json
Copying layer ...

Why is this needed for ORAS?

Improve use diagnostics and better UX for a tool that is used to interact with the registry.

Are you willing to submit PRs to contribute to this feature?

• Yes, I am willing to implement it.

[qweeah]

thanks @sajayantony for helping improve the UX, here are some comments

1. Currently when executing copy there is no output for quite some time

There are two options to solve this:

• Show progress bar when downloading or pushing content from/to a registry #893: this only work when stdout is a terminal. We can keep current output format to non-terminal output.
• Incomplete output when copying #916: show more verbose output

2. Verbose gives much more information but these are short hashes and it would good to show fully qualified names so that the user can copy the output and maybe try copying just one manifest incase of failure

Agree, we can print full digest for non-verbose mode, and fully qualified name in verbose mode.

3. The statement Exists {sha} seems odd to me. It might better read {digest} exists

Maybe we can select another verb? Changing the order will break alignment in the output

4. Before the error the manifest that caused the error for the blob should be shown.

I don't quite get this. Do you mean if the operation of pushing a blob fails, we output information of its closest predecessor manifest?

[qweeah]

The proposed output looks good to me, @sajayantony can you help confirm the UX
suppose we are going the index tagged with multi in e2e test data:

current output

$ oras cp localhost:5000/command/images:multi --to-oci-layout root --verbose
Copying fe9dbc99451d application/vnd.oci.image.config.v1+json
Copying 2ef548696ac7 hello.tar
Copying 1fd9a5fc54b6 application/vnd.oci.image.config.v1+json
Copied  2ef548696ac7 hello.tar
Copying a5dfce5d2876 application/vnd.oci.image.config.v1+json
Copied  fe9dbc99451d application/vnd.oci.image.config.v1+json
Copying 9d84a5716c66 application/vnd.oci.image.manifest.v1+json
Copied  9d84a5716c66 application/vnd.oci.image.manifest.v1+json
Copied  a5dfce5d2876 application/vnd.oci.image.config.v1+json
Copying 58efe73e78fe application/vnd.oci.image.manifest.v1+json
Copied  58efe73e78fe application/vnd.oci.image.manifest.v1+json
Copied  1fd9a5fc54b6 application/vnd.oci.image.config.v1+json
Copying 4f9346006188 application/vnd.oci.image.manifest.v1+json
Copied  4f9346006188 application/vnd.oci.image.manifest.v1+json
Copying e2bfc9cc6a84 application/vnd.oci.image.index.v1+json
Copied  e2bfc9cc6a84 application/vnd.oci.image.index.v1+json
Copied [registry] localhost:5000/command/images:multi => [oci-layout] root
Digest: sha256:e2bfc9cc6a84ec2d7365b5a28c6bc5806b7fa581c9ad7883be955a64e3cc034f

proposed output

$ oras cp localhost:5000/command/images:multi --to-oci-layout root --verbose
Copying index    localhost:5000/command/images:multi sha256:e2bfc9cc6a84ec2d7365b5a28c6bc5806b7fa581c9ad7883be955a64e3cc034f application/vnd.oci.image.index.v1+json
Copying manifest linux/amd64                         sha256:9d84a5716c66a1d1b9c13f8ed157ba7d1edfe7f9b8766728b8a1f25c0d9c14c1 application/vnd.oci.image.manifest.v1+json
Copying config                                       sha256:fe9dbc99451d0517d65e048c309f0b5afb2cc513b7a3d456b6cc29fe641386c5 application/vnd.oci.image.config.v1+json
Copying manifest linux/arm64                         sha256:4f93460061882467e6fb3b772dc6ab72130d9ac1906aed2fc7589a5cd145433c application/vnd.oci.image.manifest.v1+json
Copying layer    hello.tar                           sha256:2ef548696ac7dd66ef38aab5cc8fc5cc1fb637dfaedb3a9afc89bf16db9277e1 application/vnd.oci.image.layer.v1.tar
Copied  layer    hello.tar                           sha256:2ef548696ac7dd66ef38aab5cc8fc5cc1fb637dfaedb3a9afc89bf16db9277e1 application/vnd.oci.image.layer.v1.tar
Copying config                                       sha256:1fd9a5fc54b634130102861815e2881f1eec22958d604301904c5353041794c1 application/vnd.oci.image.config.v1+json
Copied  config                                       sha256:1fd9a5fc54b634130102861815e2881f1eec22958d604301904c5353041794c1 application/vnd.oci.image.config.v1+json
Skipped layer    hello.tar                           sha256:2ef548696ac7dd66ef38aab5cc8fc5cc1fb637dfaedb3a9afc89bf16db9277e1 application/vnd.oci.image.layer.v1.tar
Copying manifest linux/arm/v7                        sha256:58efe73e78fe043ca31b89007a025c594ce12aa7e6da27d21c7b14b50112e255 application/vnd.oci.image.manifest.v1+json
Copied  config                                       sha256:fe9dbc99451d0517d65e048c309f0b5afb2cc513b7a3d456b6cc29fe641386c5 application/vnd.oci.image.config.v1+json
Copying config                                       sha256:1fd9a5fc54b634130102861815e2881f1eec22958d604301904c5353041794c1 application/vnd.oci.image.config.v1+json
Copied  config                                       sha256:1fd9a5fc54b634130102861815e2881f1eec22958d604301904c5353041794c1 application/vnd.oci.image.config.v1+json
Copied  manifest linux/amd64                         sha256:9d84a5716c66a1d1b9c13f8ed157ba7d1edfe7f9b8766728b8a1f25c0d9c14c1 application/vnd.oci.image.manifest.v1+json
Copied  manifest linux/arm64                         sha256:4f93460061882467e6fb3b772dc6ab72130d9ac1906aed2fc7589a5cd145433c application/vnd.oci.image.manifest.v1+json
Skipped layer    hello.tar                           sha256:2ef548696ac7dd66ef38aab5cc8fc5cc1fb637dfaedb3a9afc89bf16db9277e1 application/vnd.oci.image.layer.v1.tar
Copied  manifest linux/arm/v7                        sha256:58efe73e78fe043ca31b89007a025c594ce12aa7e6da27d21c7b14b50112e255 application/vnd.oci.image.manifest.v1+json
Copied  index    localhost:5000/command/images:multi sha256:e2bfc9cc6a84ec2d7365b5a28c6bc5806b7fa581c9ad7883be955a64e3cc034f application/vnd.oci.image.index.v1+json

[qweeah]

$ oras copy docker.io/library/hello-world:latest ttl.sh/$IMAGE:2h
Copying index docker.io/library/hello-world:latest sha256:4e83453afed1b4fa1a3500525091dbfca6ce1e66903fd4c01ff015dbcb1ba33e application/vnd.docker.distribution.manifest.list.v2+json
Copying manifest sha256:eb11b1a194ff8e236a01eff392c4e1296a53b0fb4780d8b0382f7996a15d5392
application/vnd.docker.distribution.manifest.v2+json
Copying layer ...

@sajayantony There is a concern from my side: since the underlying oras-go SDK do copying under a bottom-up pattern, although Copying index is put in the very beginning, the actual copy process of index won't start until all index's predecessors are copied, which mean the debug logs of resolving and copying the index will not be shown right after the Copying index log and it might be difficult to correlate them.

[sajayantony]

Overall I like the updated verbose output the best :) and happy if we make the current verbose the default output so that it doesn't look like the tool is hanging.

Personally, I'm not a fan of the short digest and might be worth considering if we just want show full digests. I understand that docker does it but maybe its worth asking what is the value and how can the use use this information.

the actual copy process of index won't start until all index's predecessors are copied,
Yes this makes sense and showing that index copy failed at the end would be good. The Copying index statement atleast gives a user the hint of starting to copy something.

[qweeah]

I understand that docker does it but maybe it's worth asking what is the value and how can the user use this information.

I guess they are trying to save horizontal space for the progress bar? I don't know any folk who designs the short digest output, maybe @FeynmanZhou can help ask

[sajayantony]

@shizhMSFT - Any concerns improving the default output and prioritizing this work. Currently cp looks like it just hangs without any user feedback.

[shizhMSFT]

@shizhMSFT - Any concerns improving the default output and prioritizing this work. Currently cp looks like it just hangs without any user feedback.

@sajayantony Terminal size is the major concern to display lots of content where the terminal size is usually 80x24 (vt100).

The default behavior / design of oras cp follows the unix command cp that no progress bar is shown and copies silently unless it is in a verbose mode.

$ cp hello.txt hello2.txt
$ cp -v hello.txt hello3.txt
'hello.txt' -> 'hello3.txt'

To improve this, it would be better if we have two sets of UIs for standard output and TTY just like docker and buildkit do.

The current oras UI is designed for standard output and is friendly to script or logs. For example,

oras cp localhost:5000/command/images:multi --to-oci-layout root > oras_cp.log

We need another set of UI for TTY that we can actually show a progress bar to humans. For example,

$ oras cp localhost:5000/command/images:multi --to-oci-layout root
Copying 70.3s
+ sha256:e2bfc9cc6a84ec2d7365b5a28c6bc5806b7fa581c9ad7883be955a64e3cc034f 312 KB / 312 KB                   3.4s
  - application/vnd.oci.image.index.v1+json
  - localhost:5000/command/images:multi
  + sha256:9d84a5716c66a1d1b9c13f8ed157ba7d1edfe7f9b8766728b8a1f25c0d9c14c1 42 KB / 42 KB                   5.9s
    - application/vnd.oci.image.manifest.v1+json
    - linux/amd64
    + sha256:fe9dbc99451d0517d65e048c309f0b5afb2cc513b7a3d456b6cc29fe641386c5 371 KB / 462 KB  [ 5.7 KB/s] 60.1s
      - application/vnd.oci.image.config.v1+json
      ...
  + sha256:4f93460061882467e6fb3b772dc6ab72130d9ac1906aed2fc7589a5cd145433c 53 KB / 742 KB     [10.9 KB/s]  5.9s
    - application/vnd.oci.image.manifest.v1+json
    - linux/arm64
    ...
...

Currently, @qweeah is playing with TTYs, and we can prioritize the above work.

[qweeah]

Like @shizhMSFT mentioned, terminal management is the major concern for shipping progressed copying.

Had an offline discussion with @sajayantony on this before and sharing some sights of upcoming changes:

1. The major issue of oras cp is that it may silently work in the background without any output, which confuses our user. So the P0 task is to show human user least progress of undergoing work. This requires introducing a new set of TTY UI targeting terminal output.
2. Making output of oras cp more readable is another todo item but not as urgent as 1. The output should at least

• tell user which blobs are copied, and group them into artifacts
• show exact details of where the error happens instinctively (e.g. failed to copy blob xxx when copying artifact xxx)

3. User needs to see status output of unnamed blobs by default, so --verbose should be set by default. Considering make --debug output to --verbose in 2.0.

I am investigating 1, it should also be applicable to other issue like #893. We can continue discussing the output for 2 and 3 in this issue.

[shizhMSFT]

Closed by #1138