Migrate 'autosave' e2e tests to Playwright #58171
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Mamaduka
added
the
[Type] Automated Testing
Mamaduka
requested review from
kevin940726,
ntwb,
nerrad and
ajitbohra
as code owners
Mamaduka
changed the title
Mamaduka
requested review from
WunderBart
and removed request for
ntwb,
ajitbohra and
nerrad
|
Size Change: 0 B Total Size: 1.71 MB ℹ️ View Unchanged
|
Mamaduka
commented
Jan 24, 2024
| await page.evaluate( () => window.sessionStorage.clear() ); | ||
| } ); | ||
|
|
||
| test( 'should save to sessionStorage', async ( { page, pageUtils } ) => { |
There was a problem hiding this comment.
I have removed reliance on the autosave monitor in this test. It is covered by unit tests. See: packages/editor/src/components/autosave-monitor/test/index.js.
Mamaduka
commented
Jan 24, 2024
| } ); | ||
|
|
||
| // See https://github.com/WordPress/gutenberg/pull/17501. | ||
| test( "shouldn't conflict with server-side autosave", async ( { |
There was a problem hiding this comment.
I wasn't able to run this test successfully. Either there's a bug in the core, or the test itself is flaky. Currently, it only passes because of a conditional early return.
Do you think we should skip it until the issue is resolved?
|
Flaky tests detected in 5b8c51e. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/8016500115
|
Mamaduka
commented
Jan 24, 2024
| ); | ||
| } ); | ||
|
|
||
| test.skip( 'should clear sessionStorage upon user logout', async ( { |
There was a problem hiding this comment.
The actual logout action is causing the follow-up test to fail. The core uses a new browser context hack to test the logout state. I am not sure if it is suitable here.
cc @swissspidy
There was a problem hiding this comment.
You should be able to log the user out by clearing the storageState for that test only:
test( 'should clear sessionStorage upon user logout', async ( {
test.use( {
storageState: {}, // User will be logged out.
} );
// ...
} );There was a problem hiding this comment.
Oh, that might not be applicable here - we need a logged-out state mid-test.
I think that to make a self-contained logout test, we need a copy of the current storageState that we can use (and ideally dispose of after the test).
The current storage state path should be under process.env.STORAGE_STATE_PATH:
const tmpStorageStatePath = tmpClone( process.env.STORAGE_STATE_PATH );
test( 'should clear sessionStorage upon user logout', async ( { browser } ) => {
test.use( {
storageState: tmpStorageStatePath;
} );
// ...
} );This way the logout should only affect the tmp storage state file. Does that make sense?
cc: @kevin940726
There was a problem hiding this comment.
I thought we could only use this at the test file or test.describe level. I'm not sure if I can emulate the logout action with it.
There was a problem hiding this comment.
I'm not sure if I understand the problem exactly. But could we create a temporary new browser context (and page) and log in "manually"? Then we can follow the normal flow to log out too. I think it might be easier than messing with the storage state?
There was a problem hiding this comment.
Okay, I'm still stuck on this one. If any of you fine folks want to give it a try, it would be much appreciated 🙇
There was a problem hiding this comment.
It's a tough one. Now that I think about it, we might be facing a design issue here. IMO, logging a user out in one test shouldn't mean it stays logged out for the subsequent tests - we should be able to test a logged-out state whenever necessary. Should we move the login logic from the global setup to the page fixture setup instead, something like in this worker fixture example? I guess it'd mean we need to reinitialize the RequestUtils for each page, or something to that effect, which might come with some additional time cost. What do you think?
There was a problem hiding this comment.
That could work; we should probably have a separate file for the logout state test and use custom fixtures.
I'm also wondering if significant refactoring like this should be a part of the migration.
There was a problem hiding this comment.
Sorry for the late reply!
I don't have strong preference about this, and I think we can fix it in a follow-up PR if necessary. Do you think it should be a blocker for this migration PR? I'd prefer to open a new PR exploring other ideas and merge this. Given that it's the last puppeteer PR too! 😆
There was a problem hiding this comment.
I don't think it's a blocker. I'll open a new issue/PR, and we can continue our explorations there.
Mamaduka
force-pushed
the
migrate/autosave-e2e-tests
branch
from
dc3a791 to
a2fc903
Compare
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
WunderBart
reviewed
Feb 6, 2024
WunderBart
reviewed
Feb 6, 2024
Mamaduka
force-pushed
the
migrate/autosave-e2e-tests
branch
from
a2fc903 to
27d5a99
Compare
kevin940726
approved these changes
Feb 23, 2024
Mamaduka
force-pushed
the
migrate/autosave-e2e-tests
branch
from
48d5957 to
5b8c51e
Compare
WunderBart
reviewed
Feb 23, 2024
| - name: Running the tests | ||
| run: | | ||
| npx wp-scripts test-e2e --config=./packages/e2e-tests/jest.config.js --cacheDirectory="$HOME/.jest-cache" | ||
|
|
There was a problem hiding this comment.
I've not removed it entirely. Requires some settings updates for the repo. I'll do a complete clean-up follow-up PR.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Part of #38851.
Supersedes and closes #49026.
PR migrates
autosave.test.jse2e tests to Playwright.Why?
See #38851.
Testing Instructions