Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RUMM-1821 Fix choosing UI tests plan on CI #684

Merged

Conversation

ncreated
Copy link
Member

@ncreated ncreated commented Dec 9, 2021

What and why?

⚙️🐞 This PR is a fix to CI ignoring tests plans when running UITests. Because of this, each CI job was running the same set of integration tests twice and none of them was including integration tests for Crash Reporting:

+---+---------------------------------------------------------------+----------+
| ✓ | Run integration tests for RUM, Logging and Tracing (on iOS ...| 9.4 min  |
+---+---------------------------------------------------------------+----------+

+---+---------------------------------------------------------------+----------+
| ✓ | Run integration tests for Crash Reporting (on iOS Simulator)  | 8.2 min  |
+---+---------------------------------------------------------------+----------+

How?

We had a typo in bitrise.yml when passing test plan name to xcodebuild. Now, each test plan runs distinct set of tests and we should save ~7min in each CI build:

Screenshot 2021-12-09 at 16 24 05

Review checklist

  • Feature or bugfix MUST have appropriate tests (unit, integration)
  • Make sure each commit and the PR mention the Issue number or JIRA reference

@ncreated ncreated self-assigned this Dec 9, 2021
Comment on lines +40 to +45
"RUMMobileVitalsScenarioTests",
"RUMModalViewsScenarioTests",
"RUMNavigationControllerScenarioTests",
"RUMResourcesScenarioTests",
"RUMScrubbingScenarioTests",
"RUMSwiftUIScenarioTests",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two were not ignored in DatadogCrashReportingIntegrationTests.xctestplan test plan. Here I exclude them. Both remain part of DatadogIntegrationTests.xctestplan.

@ncreated ncreated marked this pull request as ready for review December 9, 2021 15:30
@ncreated ncreated requested a review from a team as a code owner December 9, 2021 15:30
Copy link
Member

@maxep maxep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌 Thanks for dealing with flakiness, it's a fastidious chore..

Base automatically changed from ncreated/RUMM-1821-fix-flakiness-in-RUMResourcesScenarioTests to master December 9, 2021 17:16
@ncreated ncreated merged commit fc969b2 into master Dec 9, 2021
@ncreated ncreated deleted the ncreated/RUMM-1821-fix-selecting-tests-plan-for-UI-tests branch December 9, 2021 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants