E2e Test Fixes #25058

kdumontnu · 2023-06-02T19:59:20Z

Run e2e tests in series so that DB + fixtures can reinitialize between each test
Run only a single test each test loop iteration 🤦‍♂️
Reenable firefox test (Closes Firefox E2E tests are failing with "Page closed" #21355)
Use locator instead of query for playwright functions

To Do:

Add more tests?
~~Bump playwright~~ and review test setup

silverwind · 2023-06-02T20:08:21Z

review test setup

should check if Firefox can be re-enabled or if it's still failing.

tests/e2e/README.md

Co-authored-by: delvh <dev.lh@web.de>

kdumontnu · 2023-06-05T21:29:24Z

review test setup

should check if Firefox can be re-enabled or if it's still failing.

I've reenabled. I've been running it on my fork for a while and haven't seen it fail yet. 👍

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

silverwind · 2023-06-08T10:49:50Z

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

Nothing really comes to mind, but if you are looking for something to test, I'd say the PR flow is something that would be nice to have, e.g.

Create branch
Edit a file in Monaco, commit it
Create PR
Review the PR on a second account
Merge the PR
Delete the branch

Part of this flow is already covered by backend integration tests, but it should really be moved to E2E instead and out of backend integration tests.

silverwind · 2023-06-20T22:56:32Z

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

Nothing really comes to mind, but if you are looking for something to test, I'd say the PR flow is something that would be nice to have, e.g.
* Create branch

* Edit a file in Monaco, commit it

* Create PR

* Review the PR on a second account

* Merge the PR

* Delete the branch
Part of this flow is already covered by backend integration tests, but it should really be moved to E2E instead and out of backend integration tests.

Maybe also start simpler:

Open issue
Comment on it
Close it

silverwind · 2023-07-07T20:27:18Z

tests/e2e/e2e_test.go

+						fmt.Printf("%v", stderr.String())
+						log.Fatal("Playwright Failed: %s", err)
+					} else {
+						fmt.Printf("%v", stdout.String())


I'm surposed ths fmt.Printf passes the linter.

silverwind · 2023-07-07T20:27:42Z

tests/e2e/issue.test.e2e.js

@@ -0,0 +1,56 @@
+// @ts-check


What's this for?

silverwind · 2023-07-07T20:28:47Z

tests/e2e/utils_e2e.js

@@ -53,7 +47,7 @@ export async function save_visual(page) {
      timeout: 20000,
      mask: [
        page.locator('.dashboard-navbar span>img.ui.avatar'),
-        page.locator('.ui.dropdown.jump.item span>img.ui.avatar'),
+        page.locator('.ui.dropdown.jump.item.tooltip span>img.ui.avatar'),


No better selector available? Maybe add a class name? This will break too easily.

silverwind · 2023-07-07T20:29:30Z

tests/e2e/example.test.e2e.js

-  await page.waitForLoadState('networkidle');
+  await page.locator('input#user_name').fill('user2');
+  await page.locator('input#password').fill('password');
+  await page.locator('form button.ui.green.button:visible').click();


Also a rather suboptimal selector. Afraid we have to add some class names to make tests more robust against class name changes.

silverwind · 2023-07-07T20:29:46Z

tests/e2e/example.test.e2e.js

+  await page.locator('input#email').fill(`e2e-test-${workerInfo.workerIndex}@test.com`);
+  await page.locator('input#password').fill('test123');
+  await page.locator('input#retype').fill('test123');
+  await page.locator('form button.ui.green.button:visible').click();


Remove DISPLAY env setting and add debug option

8fd961c

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jun 2, 2023

pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 2, 2023

delvh reviewed Jun 2, 2023

View reviewed changes

tests/e2e/README.md Outdated Show resolved Hide resolved

Update tests/e2e/README.md

7182dc9

Co-authored-by: delvh <dev.lh@web.de>

pull-request-size bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 5, 2023

kdumontnu added 4 commits June 5, 2023 17:13

Use playwright locator and login form

eef1e5b

Reenable Firefox testing

40c02fc

Merge remote-tracking branch 'upstream/main' into kd/fix-e2e_tests

aedd0e1

Add firefox to e2e_test

9d5280d

kdumontnu marked this pull request as ready for review June 5, 2023 21:29

kdumontnu changed the title ~~Remove DISPLAY env setting and add debug option~~ E2e Test Fixes Jun 5, 2023

kdumontnu added 2 commits June 8, 2023 07:28

Merge branch 'main' into kd/fix-e2e_tests

14dc954

Merge branch 'main' into kd/fix-e2e_tests

6abecd1

kdumontnu added 3 commits July 7, 2023 14:51

Minor e2e cleanup

1ade3aa

Add issues test

daf8251

Merge branch 'main' into kd/fix-e2e_tests

ee2e1d4

kdumontnu requested a review from silverwind July 7, 2023 18:57

silverwind reviewed Jul 7, 2023

View reviewed changes

tests/e2e/issue.test.e2e.js

@@ -0,0 +1,56 @@

// @ts-check

Copy link

Member

silverwind Jul 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this for?

silverwind reviewed Jul 7, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

E2e Test Fixes #25058

E2e Test Fixes #25058

kdumontnu commented Jun 2, 2023 •

edited

Loading

silverwind commented Jun 2, 2023

kdumontnu commented Jun 5, 2023

silverwind commented Jun 8, 2023 •

edited

Loading

silverwind commented Jun 20, 2023

silverwind Jul 7, 2023

silverwind Jul 7, 2023

silverwind Jul 7, 2023

silverwind Jul 7, 2023

silverwind Jul 7, 2023

E2e Test Fixes #25058

Are you sure you want to change the base?

E2e Test Fixes #25058

Conversation

kdumontnu commented Jun 2, 2023 • edited Loading

silverwind commented Jun 2, 2023

kdumontnu commented Jun 5, 2023

silverwind commented Jun 8, 2023 • edited Loading

silverwind commented Jun 20, 2023

silverwind Jul 7, 2023

Choose a reason for hiding this comment

silverwind Jul 7, 2023

Choose a reason for hiding this comment

silverwind Jul 7, 2023

Choose a reason for hiding this comment

silverwind Jul 7, 2023

Choose a reason for hiding this comment

silverwind Jul 7, 2023

Choose a reason for hiding this comment

kdumontnu commented Jun 2, 2023 •

edited

Loading

silverwind commented Jun 8, 2023 •

edited

Loading