Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ToolStrip tests is flaky in arm64 and x86 CI build #6610

Closed
Tracked by #6690
, #8607
...
vladimir-krestov opened this issue Feb 3, 2022 · 5 comments · Fixed by #8864
Closed
Tracked by #6690
, #8607
...

ToolStrip tests is flaky in arm64 and x86 CI build #6610

vladimir-krestov opened this issue Feb 3, 2022 · 5 comments · Fixed by #8864
Labels
test-bug Problem in test source code (most likely)

Comments

@vladimir-krestov
Copy link
Contributor

vladimir-krestov commented Feb 3, 2022 •
edited
Loading

  • .NET Core Version: 7.0

  • Have you experienced this same bug with .NET Framework?: No

Problem description:

  • ToolStrip_WndProc_InvokeMouseActivateWithHandle_Success unit test fails sometime in arm64 CI build for successful PRs, that blocking deployment of fixes into sdk. Stack trace (System.Windows.Forms.Tests_net7.0_arm64.log):
System.Windows.Forms.Tests.ToolStripTests.ToolStrip_WndProc_InvokeMouseActivateWithHandle_Success [FAIL]
Assert.Equal() Failure
      Expected: 0
      Actual:   2
      Stack Trace:
        /_/src/System.Windows.Forms/tests/UnitTests/System/Windows/Forms/ToolStripTests.cs(7139,0): at System.Windows.Forms.Tests.ToolStripTests.ToolStrip_WndProc_InvokeMouseActivateWithHandle_Success()
  • ToolStrip_WndProc_InvokeMouseActivate_Success:
    image

Expected behavior:
All correct tests pass every time.

Minimal repro:

  • Run unit tests on arm64 environment (or create a PR with changes, CI build will do it)
@vladimir-krestov vladimir-krestov mentioned this issue Feb 3, 2022
Merged
@dreddy-work dreddy-work added the test-bug Problem in test source code (most likely) label Feb 3, 2022
@RussKie RussKie added this to the .NET 7.0 milestone Feb 4, 2022
@vladimir-krestov vladimir-krestov changed the title ToolStrip_WndProc_InvokeMouseActivateWithHandle_Success test is flaky in arm64 CI build ToolStrip tests is flaky in arm64 and x86 CI build Feb 9, 2022
vladimir-krestov added a commit to vladimir-krestov/winforms that referenced this issue Feb 9, 2022
@vladimir-krestov
Adding Skip attributes for flaky tests
897a520 
Related Issue dotnet#6610 and Issue dotnet#6654
These tests fail sometimes in x86 CI build, that blocks deployment of fixes into sdk.
@vladimir-krestov vladimir-krestov mentioned this issue Feb 9, 2022
Merged
dreddy-work pushed a commit that referenced this issue Feb 9, 2022
@vladimir-krestov
Adding Skip attributes for flaky tests (#6655)
0007003 
Related Issue #6610 and Issue #6654
These tests fail sometimes in x86 CI build, that blocks deployment of fixes into sdk.
@vladimir-krestov vladimir-krestov mentioned this issue Feb 14, 2022
Closed
9 tasks
@dmitrii-drobotov
Copy link
Contributor

ToolStrip_WndProc_InvokeMouseActivateWithHandle_Success also fails on x64
image

dmitrii-drobotov pushed a commit to dmitrii-drobotov/winforms that referenced this issue Apr 8, 2022
Add Skip attribute or adjust skip condition for flaky tests
7f50efb 
These tests sometimes fail in CI and block deployment.
Related issues: dotnet#6610, dotnet#6714, dotnet#6739, dotnet#6926
dmitrii-drobotov pushed a commit to dmitrii-drobotov/winforms that referenced this issue Apr 8, 2022
Add Skip attribute or adjust skip condition for flaky tests
293a211 
These tests sometimes fail in CI and block deployment.
Related issues: dotnet#6610, dotnet#6714, dotnet#6739, dotnet#6926
dmitrii-drobotov pushed a commit to dmitrii-drobotov/winforms that referenced this issue Apr 8, 2022
Add Skip attribute or adjust skip condition for flaky tests
607ccc8 
These tests sometimes fail in CI and block deployment.
Related issues: dotnet#6610, dotnet#6714, dotnet#6739, dotnet#6926
@dmitrii-drobotov dmitrii-drobotov mentioned this issue Apr 8, 2022
Merged
dreddy-work pushed a commit that referenced this issue Apr 8, 2022
@dmitrii-drobotov
Add Skip attribute or adjust skip condition for flaky tests (#6991)
c402e2a 
These tests sometimes fail in CI and block deployment.
Related issues: #6610, #6714, #6739, #6926
@Tanya-Solyanik
Copy link
Member

ToolStrip_WndProc_InvokeMouseActivateWithHandle_Success does not show among failures in https://runfo.azurewebsites.net/search/tests/?q=started%3A%7E14+definition%3A267. Thus closing

@ghost ghost removed this from the .NET 7.0 milestone Jul 14, 2022
@Tanya-Solyanik
Copy link
Member

Never mind, they don't show up in the stats because they are disabled.

@elachlan
Copy link
Contributor

related: #7333

@elachlan
Copy link
Contributor

Also have seen failures on x64.

The test is supposed to return zero, but the docs don't say anything about that value.
https://learn.microsoft.com/en-us/windows/win32/inputdev/wm-mouseactivate

dreddy-work added a commit that referenced this issue Feb 9, 2023
@dreddy-work
Skipping test related to #6610
8eaceb6
@dreddy-work dreddy-work mentioned this issue Feb 9, 2023
Merged
dreddy-work added a commit that referenced this issue Feb 9, 2023
@dreddy-work
Skip or fix flaky tests blocking CI (#8603)
4d2a8ad 
* Skip or fix flaky tests

Related to:

#8602
#8601
#8598

* Skipping test related to #6610
@dreddy-work dreddy-work mentioned this issue Feb 9, 2023
Closed
54 tasks
@lonitra lonitra mentioned this issue Mar 1, 2023
Closed
@dreddy-work dreddy-work mentioned this issue Mar 14, 2023
Closed
@ghost ghost added the 🚧 work in progress Work that is current in progress label Mar 14, 2023
@dreddy-work dreddy-work mentioned this issue Mar 16, 2023
Merged
@ghost ghost removed the 🚧 work in progress Work that is current in progress label Mar 16, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Apr 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
test-bug Problem in test source code (most likely)
Projects
None yet
Milestone
No milestone
6 participants
@elachlan @RussKie @Tanya-Solyanik @dreddy-work @vladimir-krestov @dmitrii-drobotov