-
Notifications
You must be signed in to change notification settings - Fork 525
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ci] Use static macOS pool for nightly emulator tests #8206
Conversation
6b119a1
to
74a59dc
Compare
Latest nightly test run: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=8118415&view=results |
f3b39b3
to
9041b46
Compare
The few test runs I've done here definitely seem more reliable, though the localization test category is still pretty flaky. My intuition is leaning towards the fact that trying to "soft restart" the emulator ~80 times per test run - https://github.com/xamarin/xamarin-android/blob/69b4ab056badcb5d25a5faf39d08baeb1b8b7010/tests/MSBuildDeviceIntegration/Tests/LocalizationTests.cs#L230 increases the likelihood of us eventually losing the connection to the emulator. I'm going to see if there are any pools with attached Android devices that we may be able to test with. |
Attempts to improve the reliability and performance of our nightly emulator tests by using a machine pool containing static mac minis that are more performant than the hosted VMs.
9041b46
to
a8aeb8c
Compare
It doesn't seem any agents in the shared pool have an Android device demand defined, this may be the best we can do for now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems to be a large pool of real mac machines. Can we also look at using them for our PR build tests?
I have been looking at that in #8192. The owners would prefer we use the arm64 machines for PR builds and higher volume jobs, but I've been having performance issues there that I am trying to better understand. It's surprising because the arm64 pool should have m1 and m2 machines and generally be running better... |
Attempts to improve the reliability and performance of our nightly
emulator tests by using a machine pool containing static mac minis that
are more performant than the hosted VMs.