Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: enable integration test run on windows as baseline #4479

Merged
merged 1 commit into from
Dec 14, 2023
Merged

tests: enable integration test run on windows as baseline #4479

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 28 additions & 2 deletions .github/workflows/test-os.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ on:

env:
GO_VERSION: "1.21"
TESTFLAGS: "-v --parallel=6 --timeout=30m"
TESTFLAGS: "-v --parallel=6 --timeout=60m"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not for this PR but if this gets too long we need to split this suite to multiple parallel workers

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, for now we just have those 3.

GOTESTSUM_VERSION: "v1.9.0" # same as one in Dockerfile
GOTESTSUM_FORMAT: "standard-verbose"
DESTDIR: "./bin"
Expand Down Expand Up @@ -48,11 +48,37 @@ jobs:
name: Install gotestsum
run: |
go install gotest.tools/gotestsum@${{ env.GOTESTSUM_VERSION }}
-
name: Install buildkitd.exe binary
run: |
go install .\cmd\buildkitd
-
name: Install containerd.exe binary
run: |
$goModFileContent = Get-Content go.mod -Raw
$pattern = '(?<=github\.com/containerd/containerd v)[\d\.]+'
$version = [regex]::Match($goModFileContent, $pattern).Value
mkdir -p .\bin\installs -f
cd .\bin\installs
git clone https://github.com/containerd/containerd.git
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually, this would need to be done with the Dockerfile. Either directly on windows or cross-compiled on linux. Eg. freebsd task in this file does the cross-compile.

Copy link
Collaborator Author

@profnandaa profnandaa Dec 14, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call out. Eventually, we will need to explore buildkitd running in the container (WCOW).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and same for distribution binaries, can look at this in follow-up 👍

cd containerd
git checkout tags/v$version
go install .\cmd\...
cd ..\..\..
profnandaa marked this conversation as resolved.
Show resolved Hide resolved
-
name: Install registry.exe (distribution/distribution) binary
run: |
mkdir -p .\bin\installs -f
profnandaa marked this conversation as resolved.
Show resolved Hide resolved
cd .\bin\installs
git clone https://github.com/distribution/distribution.git
cd distribution
go install .\cmd\...
cd ..\..\..
-
name: Test
env:
TMPDIR: ${{ runner.temp }}
SKIP_INTEGRATION_TESTS: 1
# SKIP_INTEGRATION_TESTS: 1
run: |
mkdir -p ./bin/testreports
gotestsum \
Expand Down
1 change: 1 addition & 0 deletions client/build_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2254,6 +2254,7 @@ func testClientGatewayNilResult(t *testing.T, sb integration.Sandbox) {
}

func testClientGatewayEmptyImageExec(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureDirectPush)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down
57 changes: 49 additions & 8 deletions client/client_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,14 +246,16 @@ func testIntegration(t *testing.T, funcs ...func(t *testing.T, sb integration.Sa
}),
)

integration.Run(t, integration.TestFuncs(
testBridgeNetworkingDNSNoRootless,
),
mirrors,
integration.WithMatrix("netmode", map[string]interface{}{
"dns": bridgeDNSNetwork,
}),
)
if runtime.GOOS != "windows" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each individual test in testBridgeNetworkingDNSNoRootless should instead declare that it cannot be run in windows, right?

I want to avoid as many explicit runtime.GOOS != "windows" checks as possible, they're a little inflexible, and complicate reading the code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason, even with the line to skip at the beginning of the function, some set up was still being done; so I'd to guard this. Any tips will be appreciated.

Here's the trace:

        	Error Trace:	C:/buildkit/util/testutil/integration/run.go:198
        	Error:      	Received unexpected error:
        	            	process exited: C:\Users\Administrator\go\bin\buildkitd.exe --containerd-worker-gc=false --containerd-worker=true --containerd-worker-addr npipe:////./pipe/containerd-bktest_containerd2165329572 --containerd-worker-labels=org.mobyproject.buildkit.worker.sandbox=true --config=C:\Users\ADMINI~1\AppData\Local\Temp\2\bktest_config1965241481\buildkitd.toml --root C:\Users\ADMINI~1\AppData\Local\Temp\2\bktest_buildkitd2000050298 --addr npipe:////./pipe/buildkitd-bktest_buildkitd2000050298 --debug
        	            	github.com/moby/buildkit/util/testutil/integration.WaitSocket
        	            		C:/buildkit/util/testutil/integration/util.go:111
        	            	github.com/moby/buildkit/util/testutil/workers.runBuildkitd
        	            		C:/buildkit/util/testutil/workers/util.go:93
        	            	github.com/moby/buildkit/util/testutil/workers.(*Containerd).New
        	            		C:/buildkit/util/testutil/workers/containerd.go:228
        	            	github.com/moby/buildkit/util/testutil/integration.newSandbox
        	            		C:/buildkit/util/testutil/integration/sandbox.go:100
        	            	github.com/moby/buildkit/util/testutil/integration.Run.func3.1
        	            		C:/buildkit/util/testutil/integration/run.go:197
        	            	testing.tRunner
        	            		C:/Program Files/Go/src/testing/testing.go:1595
        	            	runtime.goexit
        	            		C:/Program Files/Go/src/runtime/asm_amd64.s:1650
        	            	creating worker
        	            	github.com/moby/buildkit/util/testutil/integration.newSandbox
        	            		C:/buildkit/util/testutil/integration/sandbox.go:102
        	            	github.com/moby/buildkit/util/testutil/integration.Run.func3.1
        	            		C:/buildkit/util/testutil/integration/run.go:197
        	            	testing.tRunner
        	            		C:/Program Files/Go/src/testing/testing.go:1595
        	            	runtime.goexit
        	            		C:/Program Files/Go/src/runtime/asm_amd64.s:1650
        	Test:       	TestIntegration/TestBridgeNetworkingDNSNoRootless/worker=containerd/netmode=dns
    --- FAIL: TestIntegration/TestBridgeNetworkingDNSNoRootless/worker=containerd/netmode=dns (20.22s)
    ```

integration.Run(
t,
integration.TestFuncs(testBridgeNetworkingDNSNoRootless),
mirrors,
integration.WithMatrix("netmode", map[string]interface{}{
"dns": bridgeDNSNetwork,
}),
)
}
}

func newContainerd(cdAddress string) (*containerd.Client, error) {
Expand All @@ -262,6 +264,7 @@ func newContainerd(cdAddress string) (*containerd.Client, error) {

// moby/buildkit#1336
func testCacheExportCacheKeyLoop(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureCacheExport, workers.FeatureCacheBackendLocal)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -327,6 +330,7 @@ func testBridgeNetworking(t *testing.T, sb integration.Sandbox) {
}

func testBridgeNetworkingDNSNoRootless(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureCNINetwork)
if os.Getenv("BUILDKIT_RUN_NETWORK_INTEGRATION_TESTS") == "" {
t.SkipNow()
Expand Down Expand Up @@ -396,6 +400,7 @@ func testHostNetworking(t *testing.T, sb integration.Sandbox) {
}

func testExportedImageLabels(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -525,6 +530,7 @@ func testExportedImageLabels(t *testing.T, sb integration.Sandbox) {

// #877
func testExportBusyboxLocal(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -554,6 +560,7 @@ func testExportBusyboxLocal(t *testing.T, sb integration.Sandbox) {
}

func testHostnameLookup(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
if sb.Rootless() {
t.SkipNow()
}
Expand All @@ -573,6 +580,7 @@ func testHostnameLookup(t *testing.T, sb integration.Sandbox) {

// moby/buildkit#1301
func testHostnameSpecifying(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
if sb.Rootless() {
t.SkipNow()
}
Expand All @@ -597,6 +605,7 @@ func testHostnameSpecifying(t *testing.T, sb integration.Sandbox) {

// moby/buildkit#614
func testStdinClosed(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand All @@ -611,6 +620,7 @@ func testStdinClosed(t *testing.T, sb integration.Sandbox) {
}

func testSSHMount(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -770,6 +780,7 @@ func testSSHMount(t *testing.T, sb integration.Sandbox) {
}

func testExtraHosts(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand All @@ -785,6 +796,7 @@ func testExtraHosts(t *testing.T, sb integration.Sandbox) {
}

func testShmSize(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -816,6 +828,7 @@ func testShmSize(t *testing.T, sb integration.Sandbox) {
}

func testUlimit(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -924,6 +937,7 @@ func testCgroupParent(t *testing.T, sb integration.Sandbox) {
}

func testNetworkMode(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -997,6 +1011,7 @@ func testPushByDigest(t *testing.T, sb integration.Sandbox) {
}

func testSecurityMode(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureSecurityMode)
command := `sh -c 'cat /proc/self/status | grep CapEff | cut -f 2 > /out'`
mode := llb.SecurityModeSandbox
Expand Down Expand Up @@ -1068,6 +1083,7 @@ func testSecurityMode(t *testing.T, sb integration.Sandbox) {
}

func testSecurityModeSysfs(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureSecurityMode)
if sb.Rootless() {
t.SkipNow()
Expand Down Expand Up @@ -1114,6 +1130,7 @@ func testSecurityModeSysfs(t *testing.T, sb integration.Sandbox) {
}

func testSecurityModeErrors(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -1294,6 +1311,7 @@ func testFrontendImageNaming(t *testing.T, sb integration.Sandbox) {
}

func testSecretMounts(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -1367,6 +1385,7 @@ func testSecretMounts(t *testing.T, sb integration.Sandbox) {
}

func testSecretEnv(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -2293,6 +2312,7 @@ func testBuildExportScratch(t *testing.T, sb integration.Sandbox) {
}

func testBuildHTTPSource(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -3084,6 +3104,7 @@ func testSourceDateEpochTarExporter(t *testing.T, sb integration.Sandbox) {
}

func testSourceDateEpochImageExporter(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
cdAddress := sb.ContainerdAddress()
if cdAddress == "" {
t.SkipNow()
Expand Down Expand Up @@ -3369,6 +3390,7 @@ func testTarExporterSymlink(t *testing.T, sb integration.Sandbox) {
}

func testBuildExportWithForeignLayer(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureImageExporter)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -3667,6 +3689,7 @@ func testBuildExportWithUncompressed(t *testing.T, sb integration.Sandbox) {
}

func testBuildExportZstd(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureOCIExporter)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -3763,6 +3786,7 @@ func testBuildExportZstd(t *testing.T, sb integration.Sandbox) {
}

func testPullZstdImage(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureDirectPush)
for _, ociMediaTypes := range []bool{true, false} {
ociMediaTypes := ociMediaTypes
Expand Down Expand Up @@ -4708,6 +4732,7 @@ func testLazyImagePush(t *testing.T, sb integration.Sandbox) {
}

func testZstdLocalCacheExport(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureCacheExport, workers.FeatureCacheBackendLocal)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -4767,6 +4792,7 @@ func testZstdLocalCacheExport(t *testing.T, sb integration.Sandbox) {
}

func testCacheExportIgnoreError(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureCacheExport)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -5153,6 +5179,7 @@ func testBasicLocalCacheImportExport(t *testing.T, sb integration.Sandbox) {
}

func testBasicS3CacheImportExport(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb,
workers.FeatureCacheExport,
workers.FeatureCacheImport,
Expand Down Expand Up @@ -5195,6 +5222,7 @@ func testBasicS3CacheImportExport(t *testing.T, sb integration.Sandbox) {
}

func testBasicAzblobCacheImportExport(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb,
workers.FeatureCacheExport,
workers.FeatureCacheImport,
Expand Down Expand Up @@ -5563,6 +5591,7 @@ func testMultipleRecordsWithSameLayersCacheImportExport(t *testing.T, sb integra

// moby/buildkit#3809
func testSnapshotWithMultipleBlobs(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureOCIExporter)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -5704,6 +5733,7 @@ func testSnapshotWithMultipleBlobs(t *testing.T, sb integration.Sandbox) {
}

func testExportLocalNoPlatformSplit(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureOCIExporter, workers.FeatureMultiPlatform)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -6262,6 +6292,7 @@ func testWhiteoutParentDir(t *testing.T, sb integration.Sandbox) {

// #2490
func testMoveParentDir(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureOCIExporter)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -6329,6 +6360,7 @@ func testMoveParentDir(t *testing.T, sb integration.Sandbox) {

// #296
func testSchema1Image(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand All @@ -6346,6 +6378,7 @@ func testSchema1Image(t *testing.T, sb integration.Sandbox) {

// #319
func testMountWithNoSource(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -6375,6 +6408,7 @@ func testMountWithNoSource(t *testing.T, sb integration.Sandbox) {

// #324
func testReadonlyRootFS(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -6522,6 +6556,7 @@ func testSourceMapFromRef(t *testing.T, sb integration.Sandbox) {
}

func testRmSymlink(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -6555,6 +6590,7 @@ func testRmSymlink(t *testing.T, sb integration.Sandbox) {
}

func testProxyEnv(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -7407,6 +7443,7 @@ func testInvalidExporter(t *testing.T, sb integration.Sandbox) {

// moby/buildkit#492
func testParallelLocalBuilds(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
ctx, cancel := context.WithCancelCause(sb.Context())
defer cancel(errors.WithStack(context.Canceled))

Expand Down Expand Up @@ -9229,6 +9266,7 @@ func testSBOMSupplements(t *testing.T, sb integration.Sandbox) {
}

func testMultipleCacheExports(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
workers.CheckFeatureCompat(t, sb, workers.FeatureMultiCacheExport)
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
Expand Down Expand Up @@ -9346,6 +9384,7 @@ func testMultipleCacheExports(t *testing.T, sb integration.Sandbox) {
}

func testMountStubsDirectory(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -9405,6 +9444,7 @@ func testMountStubsDirectory(t *testing.T, sb integration.Sandbox) {

// https://github.com/moby/buildkit/issues/3148
func testMountStubsTimestamp(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down Expand Up @@ -9811,6 +9851,7 @@ func testSourcePolicy(t *testing.T, sb integration.Sandbox) {
}

func testLLBMountPerformance(t *testing.T, sb integration.Sandbox) {
integration.SkipOnPlatform(t, "windows")
c, err := New(sb.Context(), sb.Address())
require.NoError(t, err)
defer c.Close()
Expand Down
Loading