Tests flake registry #12372

ptabor · 2020-10-08T07:58:45Z

The purpose of this issue is to track different forms of flakes we observe in the tests.
This should allow to identify frequent patterns to fix:

Format:
https://link-to-CI

test command 

Name of the test and line of the assertion

Relevant logs or stacktraces

The text was updated successfully, but these errors were encountered:

ptabor · 2020-10-08T07:59:40Z

https://travis-ci.com/github/etcd-io/etcd/jobs/396572378
https://travis-ci.com/github/etcd-io/etcd/jobs/397103429
https://travis-ci.com/github/etcd-io/etcd/jobs/397449186
https://travis-ci.com/github/etcd-io/etcd/jobs/399152128

  % (cd tests && env go test -timeout=30m -cpu=1 --race=false go.etcd.io/etcd/tests/v3/integration/clientv3)

--- FAIL: TestBalancerUnderNetworkPartitionLinearizableGetWithLongTimeout (14.23s)
    network_partition_test.go:143: #0: expected 'expected error', got 'context deadline exceeded'

{"level":"info","ts":"2020-10-07T20:31:20.647Z","caller":"clientv3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2020-10-07T20:31:21.647Z","caller":"clientv3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2020-10-07T20:31:28.193Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-5346ee46-e62c-49ef-bde1-54171b497bd0/localhost:8405709910289715390","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2020-10-07T20:31:36.369Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-7b98f825-88bb-4ac2-8ea2-45d7f8feb7c9/localhost:54018517413718362640","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2020-10-07T20:31:42.369Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-7b98f825-88bb-4ac2-8ea2-45d7f8feb7c9/localhost:54018517413718362640","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2020-10-07T20:31:44.767Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-9dcb8c71-d322-4822-858d-c6a93998288a/localhost:9824563727653249130","attempt":0,"error":"rpc error: code = Unavailable desc = etcdserver: leader changed"}
{"level":"warn","ts":"2020-10-07T20:31:49.968Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-9dcb8c71-d322-4822-858d-c6a93998288a/localhost:9824563727653249130","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: request timed out"}
{"level":"warn","ts":"2020-10-07T20:31:51.604Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-9dcb8c71-d322-4822-858d-c6a93998288a/localhost:9824563727653249130","attempt":2,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestBalancerUnderNetworkPartitionLinearizableGetWithLongTimeout (14.23s)
    network_partition_test.go:143: #0: expected 'expected error', got 'context deadline exceeded'

ptabor · 2020-10-08T11:29:50Z

https://travis-ci.com/github/etcd-io/etcd/jobs/396886611

% (cd tests && env go test -timeout=30m -cpu=4 --race=false go.etcd.io/etcd/tests/v3/integration/snapshot)

--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (19.03s)
    member_test.go:105: context deadline exceeded

{"level":"warn","ts":"2020-10-08T11:17:46.471Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-c1d96099-a901-45b0-882f-ff709ea4a966/localhost:40494","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (19.03s)
    member_test.go:105: context deadline exceeded

ptabor · 2020-10-08T12:27:15Z

https://travis-ci.com/github/etcd-io/etcd/jobs/396903952
https://travis-ci.com/github/etcd-io/etcd/jobs/397103437
https://travis-ci.com/github/etcd-io/etcd/jobs/399016770

% env go test -short -timeout=3m -cpu=4 --race=true ./... -p=2
FAIL	go.etcd.io/etcd/v3/clientv3/balancer	1.132s

--- FAIL: TestRoundRobinBalancedResolvableFailoverFromRequestFail (0.02s)
    balancer_test.go:308: expected balanced loads for 20 requests, got switches 6

{"level":"info","msg":"updated picker","picker":"picker-roundrobin-balanced","balancer-id":"c67i1lsd2usk","policy":"picker-roundrobin-balanced","subconn-ready":["127.0.0.1:36821 (0xc0001cc280)","127.0.0.1:38179 (0xc0001cc240)","127.0.0.1:39639 (0xc0001cc260)","127.0.0.1:41379 (0xc0001cc1f0)","127.0.0.1:46125 (0xc0001cc210)"],"subconn-size":5}
{"level":"debug","msg":"balancer done","picker":"picker-roundrobin-balanced","address":"127.0.0.1:36821","success":true,"bytes-sent":true,"bytes-received":true}
{"level":"debug","msg":"picked","picker":"picker-roundrobin-balanced","address":"127.0.0.1:36821","subconn-index":0,"subconn-size":5}
{"level":"debug","msg":"balancer done","picker":"picker-roundrobin-balanced","address":"127.0.0.1:36821","success":true,"bytes-sent":true,"bytes-received":true}
{"level":"debug","msg":"picked","picker":"picker-roundrobin-balanced","address":"127.0.0.1:41379","subconn-index":1,"subconn-size":5}
{"level":"debug","msg":"balancer done","picker":"picker-roundrobin-balanced","address":"127.0.0.1:41379","success":true,"bytes-sent":true,"bytes-received":true}
{"level":"debug","msg":"picked","picker":"picker-roundrobin-balanced","address":"127.0.0.1:46125","subconn-index":2,"subconn-size":5}
{"level":"debug","msg":"balancer done","picker":"picker-roundrobin-balanced","address":"127.0.0.1:46125","success":true,"bytes-sent":true,"bytes-received":true}
{"level":"debug","msg":"picked","picker":"picker-roundrobin-balanced","address":"127.0.0.1:38179","subconn-index":3,"subconn-size":5}
{"level":"debug","msg":"balancer done","picker":"picker-roundrobin-balanced","address":"127.0.0.1:38179","success":true,"bytes-sent":true,"bytes-received":true}
{"level":"debug","msg":"picked","picker":"picker-roundrobin-balanced","address":"127.0.0.1:39639","subconn-index":4,"subconn-size":5}
{"level":"debug","msg":"balancer done","picker":"picker-roundrobin-balanced","address":"127.0.0.1:39639","success":true,"bytes-sent":true,"bytes-received":true}
{"level":"debug","msg":"picked","picker":"picker-roundrobin-balanced","address":"127.0.0.1:36821","subconn-index":0,"subconn-size":5}
{"level":"debug","msg":"balancer done","picker":"picker-roundrobin-balanced","address":"127.0.0.1:36821","success":true,"bytes-sent":true,"bytes-received":true}
{"level":"debug","msg":"picked","picker":"picker-roundrobin-balanced","address":"127.0.0.1:41379","subconn-index":1,"subconn-size":5}
{"level":"debug","msg":"balancer done","picker":"picker-roundrobin-balanced","address":"127.0.0.1:41379","success":true,"bytes-sent":true,"bytes-received":true}
--- FAIL: TestRoundRobinBalancedResolvableFailoverFromRequestFail (0.02s)
    balancer_test.go:308: expected balanced loads for 20 requests, got switches 6
FAIL
FAIL	go.etcd.io/etcd/v3/clientv3/balancer	1.132s

ptabor · 2020-10-09T14:43:54Z

https://travis-ci.com/github/etcd-io/etcd/jobs/397424709
https://travis-ci.com/github/etcd-io/etcd/jobs/398575750

 % (cd tests && env go test -timeout=30m -cpu=4 --race=false go.etcd.io/etcd/tests/v3/integration/clientv3)

{"level":"warn","ts":"2020-10-09T14:08:09.969Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-8bbbbe93-1604-4d26-8e94-74d6c035522a/localhost:82250936981450807110","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: request timed out"}
--- FAIL: TestBalancerUnderNetworkPartitionWatchFollower (3.15s)
    network_partition_test.go:266: took too long to detect leader lost

{"level":"warn","ts":"2020-10-09T14:09:17.202Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-c78ce0dd-390f-4561-b0af-7b767d0c4bd9/localhost:57214615783662748290","attempt":0,"error":"rpc error: code = InvalidArgument desc = etcdserver: authentication failed, invalid user ID or password"}
{"level":"warn","ts":"2020-10-09T14:09:17.204Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-e24b0c8f-6f7c-4695-a1e0-401211225adb/localhost:57214615783662748290","attempt":0,"error":"rpc error: code = InvalidArgument desc = etcdserver: authentication failed, invalid user ID or password"}
{"level":"warn","ts":"2020-10-09T14:09:17.374Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-3f5dc931-74f8-41df-acc8-3b23aa12c1b1/localhost:20282868224859228530","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: authentication is not enabled"}

ptabor · 2020-10-09T18:29:23Z

https://travis-ci.com/github/etcd-io/etcd/jobs/397449189
https://travis-ci.com/github/etcd-io/etcd/jobs/397532784
https://travis-ci.com/github/etcd-io/etcd/jobs/397696506
https://travis-ci.com/github/etcd-io/etcd/jobs/403603526

% (cd tests && env go test -short -timeout=3m -cpu=4 --race=true ./... -p=2)

Unexpected goroutines running after all test(s).
1 instances of:
go.etcd.io/etcd/v3/clientv3.(*watchGrpcStream).sendCloseSubstream(...)
	/go/src/go.etcd.io/etcd/clientv3/watch.go:464 +0x204
created by go.etcd.io/etcd/v3/clientv3.(*watchGrpcStream).closeSubstream
	/go/src/go.etcd.io/etcd/clientv3/watch.go:480 +0x21f
FAIL	go.etcd.io/etcd/tests/v3/integration/clientv3/examples	2.111s

ptabor · 2020-10-09T18:31:03Z

https://travis-ci.com/github/etcd-io/etcd/jobs/397449193

% (cd tests && env go test -short -timeout=3m -cpu=1 --race=false ./... -p=4)

Unexpected goroutines running after all test(s).
1 instances of:
syscall.Syscall(...)
	/usr/local/go/src/syscall/asm_linux_386.s:19 +0x5
syscall.Close(...)
	/usr/local/go/src/syscall/zsyscall_linux_386.go:285 +0x3d
internal/poll.(*FD).destroy(...)
	/usr/local/go/src/internal/poll/fd_unix.go:78 +0x30
internal/poll.(*FD).decref(...)
	/usr/local/go/src/internal/poll/fd_mutex.go:213 +0x38
internal/poll.(*FD).Close(...)
	/usr/local/go/src/internal/poll/fd_unix.go:100 +0x43
net.(*netFD).Close(...)
	/usr/local/go/src/net/fd_unix.go:184 +0x49
FAIL	go.etcd.io/etcd/tests/v3/integration/client	0.058s

ptabor · 2020-10-22T07:08:07Z

https://travis-ci.com/github/etcd-io/etcd/jobs/403443329

(cd tests && 'env' 'go' 'test' '-timeout=30m' '-cpu=4' '--race=false' 'go.etcd.io/etcd/tests/v3/integration/clientv3')

{"level":"warn","ts":"2020-10-21T20:29:04.892Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-fd359190-7b13-4d65-b53e-f6a4e7fc15c3/localhost:25843431254586945980","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2020-10-21T20:29:10.120Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-2742a587-8400-4c88-9ea5-8da5c6367c15/localhost:61969052402647477830","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2020-10-21T20:29:10.732Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-c7da41e4-742c-49e0-b478-2c44f9c67587/localhost:1629367234812832150","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: Peer URLs already exists"}
--- FAIL: TestMemberPromote (0.16s)
    cluster_test.go:259: expecting promote not ready learner to fail, got no error

ptabor · 2020-10-22T07:12:02Z

https://travis-ci.com/github/etcd-io/etcd/jobs/403443330

./tests/functional/build

stderr: {"level":"info","ts":1603312264.2554727,"caller":"tester/cluster_run.go:319","msg":"functional-tester FAIL","round":0,"case":2,"case-total":22}

ptabor · 2020-10-22T07:26:33Z

https://travis-ci.com/github/etcd-io/etcd/jobs/403584630

% (cd pkg && 'env' 'go' 'test' '-short' '-timeout=3m' '-cpu=4' '--race=true' './...' '-p=2')

{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-804692469/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-804692469/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-244935888/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-244935888/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-065871855/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-065871855/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-143248514/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-143248514/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-342265337/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-342265337/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-661030148/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-661030148/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-527779987/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-527779987/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-335640278/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-335640278/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-891713085/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-891713085/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-003259000/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-003259000/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-991684983/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-991684983/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-053394538/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-053394538/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-799012545/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-799012545/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-926958892/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-926958892/key.pem"}
{"level":"info","msg":"created cert file","path":"/tmp/tlsdir945893531/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/tlsdir945893531/key.pem"}
--- FAIL: TestWriteReadTimeoutListener (0.02s)
    timeout_listener_test.go:113: err = EOF, want write i/o timeout error
{"level":"info","msg":"created cert file","path":"/tmp/etcd-test-tls-821356350/cert.pem"}
{"level":"info","msg":"created key file","path":"/tmp/etcd-test-tls-821356350/key.pem"}
FAIL
FAIL	go.etcd.io/etcd/pkg/v3/transport	5.611s
ok  	go.etcd.io/etcd/pkg/v3/types	0.030s
ok  	go.etcd.io/etcd/pkg/v3/wait	0.098s
FAIL
FAIL: (code:1):

…goroutines. Should solve the problem of flakes documented here: etcd-io#12372 (comment) ``` % (cd tests && env go test -short -timeout=3m -cpu=4 --race=true ./... -p=2) Unexpected goroutines running after all test(s). 1 instances of: go.etcd.io/etcd/v3/clientv3.(*watchGrpcStream).sendCloseSubstream(...) /go/src/go.etcd.io/etcd/clientv3/watch.go:464 +0x204 created by go.etcd.io/etcd/v3/clientv3.(*watchGrpcStream).closeSubstream /go/src/go.etcd.io/etcd/clientv3/watch.go:480 +0x21f FAIL go.etcd.io/etcd/tests/v3/integration/clientv3/examples 2.111s ``` The goroutine finishes automatically with timeout of 250ms.The change makes the test wait for it - if it still exists. Examples: https://travis-ci.com/github/etcd-io/etcd/jobs/397449189 https://travis-ci.com/github/etcd-io/etcd/jobs/397532784 https://travis-ci.com/github/etcd-io/etcd/jobs/397696506 https://travis-ci.com/github/etcd-io/etcd/jobs/403603526

ptabor · 2020-11-05T11:31:25Z

https://semaphoreci.com/etcd-io/etcd/branches/pull-request-12451/builds/1
https://semaphoreci.com/etcd-io/etcd/branches/pull-request-12588/builds/3

%(cd tests && 'env' 'go' 'test' '-timeout=30m' 'go.etcd.io/etcd/tests/v3/e2e')

--- FAIL: TestIssue6361 (14.64s)
    ctl_v3_snapshot_test.go:261: [../../bin/etcdctl --endpoints http://localhost:20030 --dial-timeout 10s get foo1] read /dev/ptmx: input/output error (expected "val1", got ["{\"level\":\"warn\",\"ts\":\"2020-11-04T14:30:12.769Z\",\"caller\":\"v3/retry_interceptor.go:62\",\"msg\":\"retrying of unary invoker failed\",\"target\":\"endpoint://client-454c2db3-3870-45ea-9fd0-63e3c0dad8c8/localhost:20030\",\"attempt\":0,\"error\":\"rpc error: code = DeadlineExceeded desc = context deadline exceeded\"}\r\n" "Error: context deadline exceeded\r\n"]). Try EXPECT_DEBUG=TRUE

ptabor · 2021-01-08T15:30:26Z

https://travis-ci.com/github/etcd-io/etcd/jobs/469558178

% (cd tests && 'env' 'go' 'test' '-timeout=30m' '-cpu=2' '--race=false' 'go.etcd.io/etcd/tests/v3/integration/clientv3')


{"level":"warn","ts":"2021-01-07T16:16:59.575Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-e4561738-ccf5-44bb-a6c7-6474fae9a872/localhost:88046767240650977590","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: Peer URLs already exists"}
{"level":"warn","ts":"2021-01-07T16:17:00.319Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-83afc6be-5f87-4773-ad51-6a4d0708b651/localhost:23908988034716734710","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: can only promote a learner member which is in sync with leader"}
{"level":"warn","ts":"2021-01-07T16:17:00.998Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-5e193a06-ada4-4419-8eef-8ce58e91bb1a/localhost:65056302181640319310","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: can only promote a learner member"}
{"level":"warn","ts":"2021-01-07T16:17:06.198Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-5e193a06-ada4-4419-8eef-8ce58e91bb1a/localhost:65056302181640319310","attempt":0,"error":"rpc error: code = Unavailable desc = etcdserver: request timed out"}
--- FAIL: TestMemberPromoteMemberNotLearner (5.37s)
    cluster_test.go:326: expect error to contain can only promote a learner member, got etcdserver: request timed out

``` Unexpected goroutines running after all test(s). 1 instances of: syscall.Syscall(...) /usr/local/go/src/syscall/asm_linux_386.s:19 +0x5 syscall.Close(...) /usr/local/go/src/syscall/zsyscall_linux_386.go:285 +0x3d internal/poll.(*FD).destroy(...) /usr/local/go/src/internal/poll/fd_unix.go:77 +0x30 internal/poll.(*FD).decref(...) /usr/local/go/src/internal/poll/fd_mutex.go:213 +0x38 internal/poll.(*FD).Close(...) /usr/local/go/src/internal/poll/fd_unix.go:99 +0x43 net.(*netFD).Close(...) /usr/local/go/src/net/fd_posix.go:37 +0x49 FAIL go.etcd.io/etcd/tests/v3/integration/client 0.039s ``` ``` --- FAIL: TestServer_TCP_Secure_DelayTx (0.20s) server_test.go:110: took 128.026085ms with no latency server_test.go:125: took 62.980988ms with latency 50ms��5ms server_test.go:133: expected took1 128.026085ms < took2 62.980988ms (with latency) ``` etcd-io#12372

stale · 2021-04-09T09:10:36Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

ptabor mentioned this issue Oct 22, 2020

./pkg/testutil: wait for: (*watchGrpcStream).sendCloseSubstream(...) goroutines. #12418

Merged

ptabor mentioned this issue Nov 5, 2020

Documentation: add generated metrics docs #12451

Merged

ptabor mentioned this issue Mar 1, 2021

tests: Fixes a few recently spotted test-flakes #12736

Merged

stale bot added the stale label Apr 9, 2021

stale bot closed this as completed Apr 30, 2021

chaochn47 mentioned this issue Mar 20, 2023

Flaky Test TestMemberPromoteMemberNotLearner #15528

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests flake registry #12372

Tests flake registry #12372

ptabor commented Oct 8, 2020 •

edited

Loading

ptabor commented Oct 8, 2020 •

edited

Loading

ptabor commented Oct 8, 2020

ptabor commented Oct 8, 2020 •

edited

Loading

ptabor commented Oct 9, 2020 •

edited

Loading

ptabor commented Oct 9, 2020 •

edited

Loading

ptabor commented Oct 9, 2020

ptabor commented Oct 22, 2020

ptabor commented Oct 22, 2020

ptabor commented Oct 22, 2020

ptabor commented Nov 5, 2020 •

edited

Loading

ptabor commented Jan 8, 2021

stale bot commented Apr 9, 2021

Tests flake registry #12372

Tests flake registry #12372

Comments

ptabor commented Oct 8, 2020 • edited Loading

ptabor commented Oct 8, 2020 • edited Loading

ptabor commented Oct 8, 2020

ptabor commented Oct 8, 2020 • edited Loading

ptabor commented Oct 9, 2020 • edited Loading

ptabor commented Oct 9, 2020 • edited Loading

ptabor commented Oct 9, 2020

ptabor commented Oct 22, 2020

ptabor commented Oct 22, 2020

ptabor commented Oct 22, 2020

ptabor commented Nov 5, 2020 • edited Loading

ptabor commented Jan 8, 2021

stale bot commented Apr 9, 2021

ptabor commented Oct 8, 2020 •

edited

Loading

ptabor commented Oct 8, 2020 •

edited

Loading

ptabor commented Oct 8, 2020 •

edited

Loading

ptabor commented Oct 9, 2020 •

edited

Loading

ptabor commented Oct 9, 2020 •

edited

Loading

ptabor commented Nov 5, 2020 •

edited

Loading