Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: retry all soft-fail Android E2E steps, add logging to iOS E2E #4084

Merged
merged 9 commits into from
Aug 14, 2020

Conversation

mikehardy
Copy link
Collaborator

Description

A bit more de-flaking, I think this will handle everything remaining seen on the Android side

  • emulator start up hangs
  • network install hangs
  • codecov report hangs

Added iOS logging but haven't attempted anything there - not sure what the failure modes are yet

The commits are all small separate changes and bear useful titles, they should be reviewed one at a time

Related issues

FIxes #4058

Release Summary

Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
    • Yes
  • My change supports the following platforms;
    • Android
    • iOS
  • My change includes tests;
    • e2e tests added or updated in packages/\*\*/e2e
    • jest tests added or updated in packages/\*\*/__tests__
  • I have updated TypeScript types that are affected by my change.
  • This is a breaking change;
    • Yes
    • No

Test Plan


Think react-native-firebase is great? Please consider supporting the project with any of the below:

It is quite large and managing emulator internal space is important
…s to 29

Includes output of `./gradlew wrapper` to regenerate everything
This should speed things up a bit, increasing success rate vs timeouts
Should help diagnose iOS flakiness there
It is the default on macOS 10.15 runner, so special setup unnecessary
This is the default, so special configuration not needed, but
still need to extract the Xcode version for Detox build cache
@mikehardy
Copy link
Collaborator Author

Follow-on ideas are to make it a matrix so multiple device APIs are handled

  • do Android API16 and API30 instead of API28 (get the full spread)
  • do iOS 10 and iOS 13 vs just 13 (full spread, again)

@mikehardy mikehardy requested a review from Salakar August 14, 2020 06:30
@Salakar
Copy link
Contributor

Salakar commented Aug 14, 2020

LGTM! :) Changes make sense

@Salakar Salakar changed the title Retry all soft-fail Android E2E steps, add logging to iOS E2E chore: retry all soft-fail Android E2E steps, add logging to iOS E2E Aug 14, 2020
@mikehardy
Copy link
Collaborator Author

Great! I'll merge it then and start accumulating false-negative data with this then

The "step chaining / outcome aggregation" trick is ugly to my eyes but it's the github-recommended way at the moment and dang if it doesn't work....

@mikehardy mikehardy merged commit d98b80e into invertase:master Aug 14, 2020
@mikehardy mikehardy deleted the issue4058/flaky-e2e branch August 14, 2020 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

E2E Github Action is flaky
2 participants