Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Take snapshots without Percy #407

Merged
merged 6 commits into from
Nov 30, 2023
Merged

Take snapshots without Percy #407

merged 6 commits into from
Nov 30, 2023

Conversation

nmanu1
Copy link
Contributor

@nmanu1 nmanu1 commented Nov 29, 2023

Update the visual regression snapshots to use the Storybook add-on jest-image-snapshot rather than Percy. We can now remove Percy from the repo entirely. The workflow now checks snapshots and commits any that have changed. Also, the play functions for stories are fixed so async actions are properly awaited.

The loading state for the Geolocation component includes animation, so the snapshot doesn't match perfectly each time. To account for this, the threshold for failure for this story is increased to a 0.005% difference in the image rather than an exact match. We also use this threshold for MapboxMap stories because the map canvas can have slight changes.

The MapboxMap component takes some time to load. I wasn't able to find a good way to detect the correct time to take a snapshot, so I added a 7500ms wait for these stories to give the map time to load before taking the snapshot. Adding more robust detection logic will be part of another item. This item will also address the snapshot for the loading story for LocationBias. Right now, we skip taking a snapshot for this story because the snapshot is blank. This is likely because geolocation permissions are disabled for the test runner, so the request for the location is automatically denied.

J=BACK-2832
TEST=auto

@coveralls
Copy link

coveralls commented Nov 29, 2023

Coverage Status

coverage: 85.155% (+0.05%) from 85.101%
when pulling 7bd3b01 on dev/snapshots
into 74c591f on develop.

Copy link
Contributor

Current unit coverage is 92.1765295887663%
Current visual coverage is 78.32699619771863%
Current combined coverage is 92.68170426065163%

@nmanu1 nmanu1 marked this pull request as ready for review November 30, 2023 15:30
@nmanu1 nmanu1 requested a review from a team as a code owner November 30, 2023 15:30
Copy link
Contributor

@EmilyZhang777 EmilyZhang777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Glad we could finally fix this

customSnapshotsDir,
customSnapshotIdentifier: context.id,
failureThreshold: context.id === 'geolocation--loading' || isMapboxMapStory
? 0.005
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity, does this mean the difference threshold between two snapshots?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exactly, this means that if the image that's currently taken is more than 0.005% pixels different from the image saved in the repo, then it'll fail and trigger a snapshot update

@nmanu1 nmanu1 merged commit 182fc28 into develop Nov 30, 2023
16 of 18 checks passed
@nmanu1 nmanu1 deleted the dev/snapshots branch November 30, 2023 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants