Thoughts on simplified UX

[Wwwsylvia]

Related issue: #102

Thoughts on simplified UX

During the oras-go V2 iteration, we have received several feedbacks from some developer users about the usability of the library. For instance:

• It takes multiple lines of code to download a content identified by a reference to the memory (See Fetch-ManifestByTag example). The steps may include:
  • 1. Resolve the reference
  • 2. Fetch the content by descriptor
  • 3. Read and verify the content
• It also takes multiple lines of code to push a content to a store and tag it (See Push example and Tag example). The steps may include:
  • 1. Generate a descriptor for the content
  • 2. Push the content with the descriptor
  • 3. Tag the descriptor with a reference
• The client.Credential function is often provided in an unsafe way

  • A typical usage is like below:

    Credential: func(ctx context.Context, reg string) (auth.Credential, error) {
                return auth.Credential{
                    Username: username,
                    Password: password,
                }, nil
            },

    This function is unsafe because it will always return the same credential for any registries.

• It is tedious to convert an OCI descriptor to an ORAS Artifact descriptor, and vice versa (See Push-ArtifactReferenceManifest example)

To resolve these pain points and make oras-go handier, we can consider adding the following utility methods:

1. Utility method for getting content

We may combine Resolve, Fetch, reading and verifying into a single method.
The API may look like:

func GetContent(ctx context.Context, target Target, reference string, isManifest bool) (ocispec.Descriptor, []byte, error)

The usage may look like:

ctx := context.Background()
ref := "myregistry.com/myrepo:latest"
target, err := remote.NewRepository(ref) // or any other targets
checkErr(err)
// TODO: possible auth configuration

// the returned content is verified against the size and the digest
desc, data, err := oras.GetContent(ctx, target, ref, true)
checkErr(err)

fmt.Println(desc.Digest)
fmt.Println(string(data))

2. Utility method for putting content

We may combine descriptor generation, Push and Tag into a single method.
The API may look like:

func PutContent(ctx context.Context, target Target, content []byte, mediaType string, reference string) (ocispec.Descriptor, error)

The usage may look like:

ctx := context.Background()
ref := "myregistry.com/myrepo:latest"
target, err := remote.NewRepository(ref) // or any other targets
checkErr(err)
// TODO: possible auth configuration

data := []byte("test manifest")
desc, err := oras.PutContent(ctx, target, data, ocispec.MediaTypeImageManifest, ref)
checkErr(err)

fmt.Println(desc.Digest)

We can have another method or we can leverage the same method for pushing blobs without tagging it, depending on the implementation.

3. Utility method for specifying static credentials

Currently, in order to perform basic auth, users need to provide the client.Credential method like this (See BasicAuth example):

client := &auth.Client{
		Credential: func(ctx context.Context, reg string) (auth.Credential, error) {
			switch reg {
			// expectedHostAddress is of form ipaddr:port
			case expectedHostAddress:
				return auth.Credential{
					Username: username,
					Password: password,
				}, nil
			default:
				// credential not found. Try anonymous access.
				return auth.EmptyCredential, nil
			}
		},
	}

We may simplify this experience by providing some straightforward methods like:

func SingleCredential(registry string, cred Credential) func(ctx context.Context, reg string) (Credential, error) {
	return func(_ context.Context, reg string) (Credential, error) {
		if reg == registry {
			return cred, nil
		}
		return EmptyCredential, nil
	}
}

// or this:
func StaticCredentials(creds map[string]Credential) func(ctx context.Context, reg string) (Credential, error) {
	return func(_ context.Context, reg string) (Credential, error) {
		if cred, ok := creds[reg]; ok {
			return cred, nil
		}
		return EmptyCredential, nil
	}
}

The usage may look like:

ref := "myregistry.com/myrepo:latest"
repo, err := remote.NewRepository(ref)
checkErr(err)
repo.Client = &auth.Client{
    Credential: auth.SingleCredential("myregistry.com", auth.Credential{
        Username: "username",
        Password: "password",
    }),
}

// or this: 
creds := map[string]auth.Credential{
    "myregistry.com": {
        Username: "username",
        Password: "password",
    },
    "localhost:5000": {
        Username: "test",
        Password: "test",
    },
}
repo.Client = &auth.Client{
    Credential: auth.StaticCredentials(creds),
}

4. Utility methods for converting descriptor

We may consider exposing the internal ArtifactToOCI and OCIToArtifact methods as they are simple and commonly-used.

5. Utility methods for generating basic descriptors

We may also consider providing some utility methods for generating basic descriptors, for example:

func OCIDescriptor(content []byte, mediaType string) ocispec.Descriptor {
	return ocispec.Descriptor{
		MediaType: mediaType,
		Digest:    digest.FromBytes(content),
		Size:      int64(len(content)),
	}
}

func ArtifactDescriptor(content []byte, mediaType, artifactType string) artifactspec.Descriptor {
	return artifactspec.Descriptor{
		MediaType:    mediaType,
		ArtifactType: artifactType,
		Digest:       digest.FromBytes(content),
		Size:         int64(len(content)),
	}
}

The usage may look like:

blob := []byte("test content")
ociDesc := OCIDescriptor(blob, ocispec.MediaTypeImageLayer)
artifactDesc := ArtifactDescriptor(blob, ocispec.MediaTypeImageLayer, "test/artifact")

Please leave a comment if you have any thoughts, thanks!

[qweeah]

Thanks for putting these all together! For the PutContent, can we use the below signature?

func PutContent(ctx context.Context, target Target, content io.Reader, mediaType string, tag string) (ocispec.Descriptor, error)

It will be more flexible to use an io.Reader and user don't have to load all the content into memory. For the tag parameter, if it's an non-empty string, we can use it to tag.

[Wwwsylvia]

If user has an io.Reader and a descriptor, do they still need this utility method? They can use Push or PushReference.

[qweeah]

I don't assume user have a descriptor. Converting a string or bytes[] to a io.Reader is just one call away (and without error handling).

[Wwwsylvia]

This is for users who wants to push a content in the memory.

[qweeah]

Yes after some thoughts I think content io.Reader is a bad idea, please ignore it.

[Wwwsylvia]

For the tag parameter, if it's an non-empty string, we can use it to tag.

It makes sense that we don't tag when the tag variable is empty.

[qweeah]

I think 4 and 5 are functionally overlapped, and one upvote for 5.

[binbin-li]

for 1, and 2, I'm just curious that if the original splitted methods might be used independently in the future. And if we always use them together, why did we design those splitted methods initially?

[Wwwsylvia]

We are now adding new convenient methods for common scenarios, the splitted methods will still be there and will still be used in many places.

[shizhMSFT]

I would suggest having a descriptor package for all OCI / Artifact descriptors.

[Wwwsylvia]

Updates

Based on the offline discussions, we will have the following utility methods:

1. Utility method for getting content

// or better name
type ResolveFetcher interface {
	content.Resolver
	content.Fetcher
}

func GetContent(ctx context.Context, target ResolveFetcher, reference string) (ocispec.Descriptor, []byte, error)

Usage:

ctx := context.Background()
target, err := remote.NewRepository("myregistry.com/myrepo") // or any other targets
checkErr(err)
// TODO: possible auth configuration

// get manifest
desc, data, err := oras.GetContent(ctx, target, "latest")
checkErr(err)

// get blob
desc, data, err = oras.GetContent(ctx, target.Blobs(), "sha256:b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9")
checkErr(err)

2. Utility method for putting content

func PutContent(ctx context.Context, storage content.Storage, content []byte, mediaType string) (ocispec.Descriptor, error)

func TagContent(ctx context.Context, target Target, content []byte, mediaType string, reference string) (ocispec.Descriptor, error)

Usage:

ctx := context.Background()
target, err := remote.NewRepository("myregistry.com/myrepo") // or any other targets
checkErr(err)
// TODO: possible auth configuration

// push blob
data := []byte("test blob")
desc, err := oras.PutContent(ctx, target.Blobs(), data, ocispec.MediaTypeImageLayer)

// push and tag manifest
data = []byte("test manifest")
desc, err = oras.TagContent(ctx, target, data, ocispec.MediaTypeImageManifest, "latest")
checkErr(err)

3. Utility method for specifying static credential

func StaticCredential(registry string, cred Credential) func(ctx context.Context, reg string) (Credential, error) {
	return func(_ context.Context, reg string) (Credential, error) {
		if reg == registry {
			return cred, nil
		}
		return EmptyCredential, nil
	}
}

Usage:

ref := "myregistry.com/myrepo"
repo, err := remote.NewRepository(ref)
checkErr(err)
repo.Client = &auth.Client{
    Credential: auth.StaticCredential("myregistry.com", auth.Credential{
        Username: "username",
        Password: "password",
    }),
}

4. Utility methods for converting descriptor

This is not needed anymore, since the ORAS Artifact descriptor will be deprecated since v2.0.0-rc.4.

5. Utility method for manipulating descriptors

func GenerateDescriptor(content []byte, mediaType string) ocispec.Descriptor {
	return ocispec.Descriptor{
		MediaType: mediaType,
		Digest:    digest.FromBytes(content),
		Size:      int64(len(content)),
	}
}

func Equal(a, b ocispec.Descriptor) bool {
	return a.Digest == b.Digest && a.Size == b.Size && a.MediaType == b.MediaType
}

Usage:

blob := []byte("test content")
desc := GenerateDescriptor(blob, ocispec.MediaTypeImageLayer)

isEqual := Equal(desc1, desc2)