Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore: Removing integration tests #2916

Merged
merged 16 commits into from
Apr 18, 2023
Merged

Chore: Removing integration tests #2916

merged 16 commits into from
Apr 18, 2023

Conversation

brustolin
Copy link
Contributor

@brustolin brustolin commented Apr 17, 2023

Both Alamofire and Home Assistant integration tests are causing more trouble then helping, we will remove this CI tests.

#skip-changelog

@brustolin brustolin changed the title Chore: Fix integration tests Chore: Removing integration tests Apr 17, 2023
@brustolin brustolin marked this pull request as ready for review April 17, 2023 13:50
Copy link
Member

@philipphofmann philipphofmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before we can merge this, I think we should

  1. Add an entry to the internal decision log
  2. Add critical files we shouldn't change by mistake to no-changes-in-high-risk-files.sh.
  3. The failure message in no-changes-in-high-risk-files.sh should point you to make test-alamofire and make test-homekit.

Maybe doing some of these steps in subsequent PRs is also okay. Up to you.

@armcknight
Copy link
Member

I'll admit I haven't really looked at their results; what issues do they cause? Is it something we could run only after merging to main instead of for every PR push?

@brustolin
Copy link
Contributor Author

Is it something we could run only after merging to main instead of for every PR push?

TBH, I cant figure it out the problem, this tests went from working flawlessly to failing every time in CI with no apparent reason.
I don't think the headache worth keeping this tests.

@philipphofmann
Copy link
Member

@armcknight, we have had these integration tests for roughly a year, and they didn't surface any bugs in our code. We have them to ensure we are not breaking the functionality of UIViewControllers, NSURLSession, NSData, after swizzling. Right now, we know that the swizzling is working fine, and we didn't have to touch the swizzling logic for a year or so. Instead of maintaining these integration tests, we decided to lock these files with no-changes-in-high-risk-files. The next time we have to do some changes, we can add the integration tests again or run them manually.

Copy link
Member

@philipphofmann philipphofmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this @brustolin 👏

develop-docs/README.md Show resolved Hide resolved
scripts/no-changes-in-high-risk-files.sh Outdated Show resolved Hide resolved
scripts/no-changes-in-high-risk-files.sh Show resolved Hide resolved
scripts/no-changes-in-high-risk-files.sh Outdated Show resolved Hide resolved
scripts/test-alamofire.sh Show resolved Hide resolved
scripts/test-homekit.sh Show resolved Hide resolved
Copy link
Member

@philipphofmann philipphofmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @brustolin 👍. LGTM

@brustolin brustolin merged commit f168340 into main Apr 18, 2023
@brustolin brustolin deleted the chore/test-alamofire branch April 18, 2023 08:23
philipphofmann added a commit that referenced this pull request Oct 11, 2024
We don't run the test-alamofire.sh and test-homekit.sh anymore in CI, see GH-2916 and decision log .
Both scripts don't even work anymore. We can remove them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants