Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tags: Adds [Excluded:WindowsDocker] tag #5355

Merged
merged 1 commit into from
Oct 7, 2021
Merged

tags: Adds [Excluded:WindowsDocker] tag #5355

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions contributors/devel/sig-testing/e2e-tests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,13 @@ to be eligible for this tag. This tag does not supersed any other labels.
(e.g.: seLinuxOptions) or is unable to run on Windows nodes, it is labeled
`[LinuxOnly]`. When using Windows nodes, this tag should be added to the
`skip` argument.
- `[Exclude:WindowsDocker]`: Windows Kubelet supports both Docker and Containerd,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feature:NotSupportedByWindowsDocker would mean we don't have to change our existing tag regexes I think? I might be wrong

I think this needs some changes with an eye toward how we use tags with conformance. Maybe just docs, maybe some of the verify code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are quite a lot of test jobs that have \[Feature:.+\] as their skip regex, so that would mean we'd suddenly exclude these tests from all jobs, even though we shouldn't. It would be harder to update all those regexes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is something of a general downside of [Feature] and the whole regex tags thing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I like [Excluded:WindowsDocker] as a bikeshed (As in the linked kubernetes PR) because it implies that it will automatically be excluded somehow (which is also not the sort of logic conformance tests could even have and some of these are conformance tests).

Docker being deprecated, I think it's honestly reasonable to just maintain a list of excluded tests in the windows docker jobs until docker is removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, dockershim is going to be removed in Kubernetes 1.24, so there's still some time until then. This tag would only be temporary until then, and it would be removed afterwards anyways.

Hm, I'd say the purpose of this tag is to extend the test suite we're running on Windows, rather than the other way around. Right now, those tests are being excluded for all Windows runs, by simply having the [LinuxOnly] tag. There are a couple of tests that can easily run and pass on Windows containerd since it has more features, which is why this tag is proposed, so we can run those tests on Windows containerd, while not including them in Windows docker runs.

Indeed, The alternative would be to remove the [LinuxOnly] tag, and add to the skip regex a very long string that would represent that list of tests. But it would be simpler to just exclude [Excluded:WindowsDocker].

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are also some tests that sniff the runtime and auto skip, though not conformance.

I still bikeshed the name at least, the current one sound automatic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's bikesheding, then what about:

  • [SkipWindowsDocker]
  • [PartialWindowsSupport]

Or, since it's related to additional containerd features:

  • [Containerd]
  • [ContainerdOnly] (not a fan of this, it would imply that it wouldn't work on Linux docker, which might not be true)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should start tagging with [Containerd], docker on linux aside there's also cri-o or future runtimes ...

The first two sound ok @spiffxp ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds like you're saying there are things [Windows] nodes cannot do with [Docker] but can do with [Containerd]. Is the situation the same for [Linux] nodes?

Trying to squish:

  • Node os capabilities
  • CRI capabilities
  • Node os implementation
  • CRI implementation

All into a single string tag (or taxonomy) is really showing this system's limits, again.

Thinking through my own bikeshed on this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't think of any other features at the moment, only one comes to mind: single file mounts / mappings. That works in Linux using Docker, but it has been known to not work on Windows using Docker (https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md#windows--linux-considerations , see Storage section). However, that scenario now works with Windows using Containerd (actually, we should update the docs for this as well). We had a couple of PRs that enabled single file mappings for Windows Containerd (example: kubernetes/kubernetes#83057 ). I have sent a PR that would enable some Conformance tests for Windows Containerd as well: kubernetes/kubernetes#97045

It's not a very common scenario, and it's mostly a requirement for Conformance tests since we can't have any Skip logic in them. It might happen more if more tests will be promoted to Conformance.

however, Containerd has a few additional features (e.g.: Single file mapping). This
means that there are tests that can run on Containerd Windows nodes, but they will
fail on Docker Windows nodes. These tests are labeled `[Exclude:WindowsDocker]`, thus,
when using Windows Docker nodes, this tag should be added to the `skip` argument. This
tag is intended to be removed when Containerd becomes the default for Windows nodes in
1.24 (https://github.com/kubernetes/community/issues/6123)

- The following tags are not considered to be exhaustively applied, but are
intended to further categorize existing `[Conformance]` tests, or tests that are
Expand Down