Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Puppeteer API to simulate reduced motion in e2e tests #18453

Closed
wants to merge 1 commit into from

Conversation

youknowriad
Copy link
Contributor

This new API in Puppeteer 2.0 removes the need for the disable animations plugin and the env variable we were using.

@youknowriad youknowriad added the [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. label Nov 12, 2019
@youknowriad youknowriad self-assigned this Nov 12, 2019
@@ -213,7 +212,7 @@ beforeAll( async () => {

await trashExistingPosts();
await setupBrowser();
await activatePlugin( 'gutenberg-test-plugin-disables-the-css-animations' );
await page.emulateMediaFeatures( [ { name: 'prefers-reduced-motion', value: 'reduce' } ] );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How nice 🎉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still waiting for the confirmation from Travis :P

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should move it to each page load.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, this might be required for every page load 🙃 It doesn't work at the moment according to Travis.

Copy link
Member

@gziolo gziolo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool usage of new capabilities of Puppeteer, nice one 👍

@youknowriad youknowriad force-pushed the update/reduced-motion-e2e-tests branch 2 times, most recently from 9f73a0d to dd184cd Compare November 12, 2019 10:09
@youknowriad
Copy link
Contributor Author

For some reason, this is not working as intended, I'll need to debug this a bit later.

@@ -206,6 +206,7 @@ describe( 'Preview with Custom Fields enabled', () => {
beforeEach( async () => {
await createNewPost();
await toggleCustomFieldsOption( true );
await page.emulateMediaFeatures( [ { name: 'prefers-reduced-motion', value: 'reduce' } ] );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it only take effect in specific test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toggleCustomFieldsOption called before reloads the page. My thinking was maybe we need this on each page load, but Travis doesn't not agree with me anyway.

I'm not sure yet why this PR is not working as intended.

@youknowriad youknowriad force-pushed the update/reduced-motion-e2e-tests branch from dd184cd to 9e62143 Compare December 6, 2019 10:12
@youknowriad
Copy link
Contributor Author

My difficulty here is that I'm having a hard time reproducing the travis issues locally. I tried CPU throttling without success. @aduth since you worked on simulation APIs too, maybe you have an idea about the issues here.

@aduth
Copy link
Member

aduth commented Jan 2, 2020

Are you doing a full run of tests locally, or just specific test files? I wonder if there's something we need to be doing as "clean-up" where we enact the emulation, otherwise risk lingering effects bleeding between tests?

@youknowriad
Copy link
Contributor Author

Ideally, this is enabled for the whole suite of tests though. So even if it lingers, that's the desired behavior.

@aduth
Copy link
Member

aduth commented Jan 2, 2020

At least for how the pull request is currently implemented, it's only targeting specific tests.

I'm not saying it should be one way or the other, but I'm wondering if that might explain the differences between what you're seeing locally and in Travis.

@youknowriad
Copy link
Contributor Author

youknowriad commented Jan 2, 2020

originally, I tried activating this in the setup before each test but I had the same failures, I thought that maybe it's because every time a page is refreshed the simulation resets, so I tried doing it after every page refresh (like the current state), but none of these worked.

@aduth
Copy link
Member

aduth commented Jan 2, 2020

Oh, I missed that it's now in createNewPost. Are some of the other calls in beforeEach now redundant with this? I'm not sure whether this could be a source of potential conflict, since we seem to be calling emulateMediaFeatures twice for some of these tests, but at the very least the duplication shouldn't be necessary.

If it's really intended to take effect for the whole test, then I think it should be in the beforeEach for the test setup, not in createNewPost. If it stays there, I think there ought to be an explicit tear-down, otherwise I might worry about some unanticipated breakage later on.

@youknowriad
Copy link
Contributor Author

Right now, neither of these options work though. I moved it to createNewPage because I assumed that maybe the "page" instance change when we navigate to a new page, forcing us to call the emulation again but this is was a wrong assumption I think, since it doesn't work anyway. So there must be something else we're missing.

@youknowriad
Copy link
Contributor Author

Unfortunately, it's not clear how to fix this, I'm going to close this PR for now.

@youknowriad youknowriad deleted the update/reduced-motion-e2e-tests branch January 29, 2020 10:46
@aduth
Copy link
Member

aduth commented Jan 29, 2020

I might like to pick this up again at some point.

Some rough thoughts off the top of my head, for my future self or whoever looks at it again:

  • Any difference if we consistently page.emulateMediaFeatures as part of the global test setup beforeEach ? (I think it should be done this way regardless)
  • Any difference if updating to a more recent version of Puppeteer? (possible upstream bug?)
  • Possibly emulateMediaFeatures is throwing the browser into the "simulation view" where there's a backdrop, heading, etc. added by the browser? (could explain some of the test failures where nodes aren't on-screen to interact with)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants