Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Use static macOS pool for nightly emulator tests #8206

Merged
merged 1 commit into from
Aug 3, 2023

Conversation

pjcollins
Copy link
Member

Attempts to improve the reliability and performance of our nightly
emulator tests by using a machine pool containing static mac minis that
are more performant than the hosted VMs.

@pjcollins
Copy link
Member Author

@pjcollins pjcollins force-pushed the static-macos-nightly branch 2 times, most recently from f3b39b3 to 9041b46 Compare July 21, 2023 19:20
@pjcollins
Copy link
Member Author

The few test runs I've done here definitely seem more reliable, though the localization test category is still pretty flaky. My intuition is leaning towards the fact that trying to "soft restart" the emulator ~80 times per test run - https://github.com/xamarin/xamarin-android/blob/69b4ab056badcb5d25a5faf39d08baeb1b8b7010/tests/MSBuildDeviceIntegration/Tests/LocalizationTests.cs#L230 increases the likelihood of us eventually losing the connection to the emulator. I'm going to see if there are any pools with attached Android devices that we may be able to test with.

Attempts to improve the reliability and performance of our nightly
emulator tests by using a machine pool containing static mac minis that
are more performant than the hosted VMs.
@pjcollins
Copy link
Member Author

It doesn't seem any agents in the shared pool have an Android device demand defined, this may be the best we can do for now.

@pjcollins
Copy link
Member Author

@pjcollins pjcollins marked this pull request as ready for review August 2, 2023 16:20
Copy link
Contributor

@jpobst jpobst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be a large pool of real mac machines. Can we also look at using them for our PR build tests?

@pjcollins
Copy link
Member Author

pjcollins commented Aug 3, 2023

This seems to be a large pool of real mac machines. Can we also look at using them for our PR build tests?

I have been looking at that in #8192. The owners would prefer we use the arm64 machines for PR builds and higher volume jobs, but I've been having performance issues there that I am trying to better understand. It's surprising because the arm64 pool should have m1 and m2 machines and generally be running better...

@pjcollins pjcollins merged commit c22b2a8 into dotnet:main Aug 3, 2023
64 of 68 checks passed
@pjcollins pjcollins deleted the static-macos-nightly branch August 3, 2023 15:01
@github-actions github-actions bot locked and limited conversation to collaborators Jan 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants