Modify conformance tests for single-shot uploads

[jdolitsky]

This is an attempt to implement this suggestion: #230 (comment)

Also updated Go module dependencies, Go version, golanglint-ci version, and removed some dead code

[jzelinskie]

Should we change this to be optional if so many registries don't support it currently?
If nobody is supporting it now, I don't think it's desirable to force them to considering this is basically legacy functionality.

[jdolitsky]

@jzelinskie good call. just added a OCI_SKIP_SINGLE_SHOT_UPLOAD_TEST env var for this

[jonjohnsonjr]

Should we change this to be optional if so many registries don't support it currently?

This is basically testing that the non-support does "the right thing".

If they don't support it, then we would expect a 202 from the POST, and doing a blob GET should result in a 404. That will pass.

If they do support it, then we would expect a 201 from the POST, and doing a blob GET should result in a 200. That will also pass.

I believe any registry that doesn't support this would just not look at the request body (distribution behaves this way), and so they just return a 202 with the expectation that the client would follow up with a PATCH/PUT or just a PUT.

If there is a registry that doesn't support this but also doesn't return a 202 in that case, the OCI_SKIP_SINGLE_SHOT_UPLOAD_TEST would be useful, but I'd be really surprised.

[jonjohnsonjr]

Note that this breaks a few registries, including docker/distribution.

Does it really?

[jdolitsky]

Yes, it does:

The single-shot POST returns a 202

[jdolitsky]

I retract my comments about this breaking distribution.

On further inspection, the same config blob is uploaded during pull test setup. If this is run with the following environment, things pass with docker/distribution (distribution/distribution):

export OCI_NAMESPACE=oci-test
export OCI_TEST_CONTENT_MANAGEMENT=0
export OCI_TEST_PUSH=1
export OCI_TEST_PULL=0
export OCI_ROOT_URL=http://localhost:5000
export OCI_TEST_CONTENT_DISCOVERY=0

So if anything, there is a bug in the tests which disallow running all 4 categories at once

[dmcgowan]

@jdolitsky is the same true for the other registries mentioned? If we can't find a legitimate breakage, we should just remove the skip and get this update in.

[jdolitsky]

@dmcgowan as far as I understand, Harbor (the other registry mentioned), uses distribution as backend. I can't imagine this will break them with that being the case. Also considering 230 was opened by one of the maintainers

Will go ahead and remove the skip ability

[jonjohnsonjr]

I think we need to fix #247 before we can merge this, right?

[jdolitsky]

@jonjohnsonjr probably best to do so - just opened #249 to address

[jdolitsky]

After rebasing, here are the results running against docker/distribution (distribution/distribution):

build conformance binary:

make conformance-binary

create config file with delete enabled:

cat > dist-config.yml << EOF
version: 0.1
log:
  level: debug
storage:
    delete:
      enabled: true
    filesystem:
        rootdirectory: /tmp/registry
http:
    addr: :5000
    headers:
        X-Content-Type-Options: [nosniff]
EOF

start registry at localhost 5000:

docker run -v "$(pwd)/dist-config.yml":/etc/docker/registry/config.yml --rm -p 5000:5000 registry:2.7.1

set environment, all categories enabled:

export OCI_CROSSMOUNT_NAMESPACE=oci-test-cross
export OCI_NAMESPACE=oci-test
export OCI_TEST_CONTENT_MANAGEMENT=1
export OCI_TEST_PUSH=1
export OCI_TEST_PULL=1
export OCI_ROOT_URL=http://localhost:5000
export OCI_TEST_CONTENT_DISCOVERY=1

run conformance tests:

output/conformance.test

results:

Ran 59 of 62 Specs in 0.426 seconds
FAIL! -- 55 Passed | 4 Failed | 0 Pending | 3 Skipped

The 201 vs. 202 is now passing, but failing in other areas previously described here: distribution/distribution#3203

[jdolitsky]

this is the heart of the change

[mikebrow]

LGTM