Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[playwright] Add support for handling option preferences and reactivate skipped test #10924

Merged
merged 1 commit into from
May 10, 2022

Conversation

planger
Copy link
Contributor

@planger planger commented Mar 25, 2022

What it does

Due to the Monaco uplift, we had to temporarily skip a test, because the Monaco change affected the default preference regarding auto-save.

To re-enable the skipped test, we now explicitly set the auto save preference to off before the test. This way the preference value is ensured to be consistent. In order to do that conveniently, this change adds general support for preferences that are set by HTML select/options. In the course of introducing support for select/option preferences, we harmonized the code for handling different types of preferences in the TheiaPreferenceView page object.

See also #10736
Fixes #10891

How to test

This change adds additional tests for the new support for option preferences and reactivates the test that had to be skipped in #10736.

Review checklist

Reminder for reviewers

@vince-fugnitto vince-fugnitto added the playwright issues related to playwright tests label Mar 25, 2022
Copy link
Contributor

@colin-grant-work colin-grant-work left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few of the tests failed for me (MacOS) when I ran this PR alone, but they passed when I rebased this work on #10925. Let's merge that one, rebase this one, and confirm that everything is working once that's done.

@planger
Copy link
Contributor Author

planger commented Mar 30, 2022

Thanks for testing, @colin-grant-work! #10925 is now merged and I rebased this PR. Build is still running, but the Playwright Tests and one CI/CD Build is green already.

@planger
Copy link
Contributor Author

planger commented Mar 30, 2022

Hm, one build reported two failing unit tests:

root INFO   821 passing (3m)
root INFO   2 failing
root INFO   1) Launch Preferences
       Valid Compound Launch Configuration
         ".theia"
           delete launch:
     AssertionError: expected null to deeply equal undefined
      at Context.<anonymous> (/home/runner/work/theia/theia/examples/api-tests/src/launch-preferences.spec.js:658:24)

root INFO   2) Launch Preferences
       Valid Compound Launch Configuration
         ".vscode"
           delete launch:
     AssertionError: expected null to deeply equal undefined
      at Context.<anonymous> (/home/runner/work/theia/theia/examples/api-tests/src/launch-preferences.spec.js:658:24)

Seems unrelated on a first sight. Is this a flaky test as it passes on node 12 and node 16 if I'm not mistaken?

@colin-grant-work
Copy link
Contributor

@planger, that's on the list of known flaky tests, so intermittent failures in the Launch Preferences suite can mostly be ignored. The undefined !== null complaint is relatively recent, but I've also made note of that specifically, so it's definitely nothing to worry about. I'll take a look at some of the flaky tests after the release.

@planger
Copy link
Contributor Author

planger commented Apr 29, 2022

I had to adjust the preference editor page object as we don't use select elements anymore for options, but render them with a div and a popup div on latest master. Now the tests are green again and the previously skipped test is re-enabled.

Can you please take a look if those changes are fine with you? Thanks!

Copy link
Member

@vince-fugnitto vince-fugnitto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirm that the tests pass during CI and locally 👍

Copy link
Member

@vince-fugnitto vince-fugnitto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 The playwright tests pass successfully.

@JonasHelming
Copy link
Contributor

@planger Could you squash please, I am not sure how the final commit message should look like

Due to the Monaco uplift, we had to temporarily skip a test, because
it affected the default preference regarding auto-save.

To re-enable the test, we now explicitly set the auto save preference to
`off` before the test. This way the preference value is ensured to be
consistent.

In order to do so, we add support for option preferences

See also eclipse-theia#10736

Fixes eclipse-theia#10891

Change-Id: I6ab22a71848e174313a30f9121cb4b3dec363f12
@JonasHelming
Copy link
Contributor

@colin-grant-work : Should I wait merging this?

@colin-grant-work
Copy link
Contributor

@JonasHelming, no; it's good to go. I was enqueuing my reviews this morning, and I hadn't noticed that Vince had already approved. I'm fine with what I've seen with the changes, and his is good enough for me.

@JonasHelming JonasHelming merged commit fe4204f into eclipse-theia:master May 10, 2022
@JonasHelming JonasHelming added this to the 1.26.0 milestone May 10, 2022
@planger planger deleted the issues/10891 branch May 10, 2022 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
playwright issues related to playwright tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update theia-text-editor close without saving playwright test and re-enable
4 participants