Add Initial Conformance Test Framework

[robscott]

We want to ensure that controllers using these APIs are conformant. Having a good conformance test suite will help establish consistency across community implementations and help developers identify any missing pieces in their implementation. This should work similarly to the existing Kubernetes conformance tests and be accessible with the same workflows.

[aojea]

is this not related https://github.com/kubernetes-sigs/ingress-controller-conformance?

if affirmative, should we merge it into this repo?

[robscott]

@aojea It is related, but unfortunately it doesn't follow the pattern established by the Kubernetes conformance tests. @aledbf has some work in progress for ingress conformance following that pattern, I'd been thinking about using that as a starting point for work here as well.

[jpeach]

The current conformance suite was proposed and accepted on the mailing list

https://groups.google.com/d/msg/kubernetes-sig-network/dAt78O-JUCo/vvOXt3-XCgAJ
kubernetes/org#1328

Conformance testing needs to test not just the v2 API, but also Ingress v1beta1 and v1. I am not sure that it should live in this repo, since it has a broader scope. I would prefer to avoid building 2 separate suites and ending up with different overlapping tests.

Probably @aledbf and @alexgervais should discuss?

[alexgervais]

When https://github.com/kubernetes-sigs/ingress-controller-conformance started, it was intended to support v1beta1 and v1 Ingress APIs for various ingress-controller implementations outside the core k/k.

It is desirable to have a single conformance test suite for v1beta1, v1 Ingress and v2 Service APIs. We have not committed to using the sonobuoy in kubernetes-sigs/ingress-controller-conformance, but building on a common ground would make total sense. The worst possible thing would be to completely diverge on approaches here and have the 2 suites diverge so early in their inception.

@aledbf , thoughts?

[jpeach]

xref kubernetes-sigs/ingress-controller-conformance#29

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[jpeach]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[robscott]

/remove-lifecycle stale
/lifecycle frozen

[markmc]

I think this issue was resolved by #969

[robscott]

Woops, thanks @markmc!

/close

[k8s-ci-robot]

@robscott: Closing this issue.

In response to this:

Woops, thanks @markmc!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.