[ci][playwright] Solidify flaky Playwright tests

[marcdumais-work]

What it does

This is an attempt at solidifying the playwright test suite, that has had testcases intermitently failing. I have been lucky to be able to observe some race conditions rather consistently locally, and so be able to debug them and solidify against them.

The race conditions are very slight - the Playwright test suite takes a screenshot at every failure, and consistently I see the correct state on those, meaning that the state is what it should be, by the time the screenshot is taken.

In consequence, I wound that, for a given flaky scenario, addressing one race consition would often mask a second one, present in the same scenario. It's possible that under a different execution context, more race conditions would occur. Here are the race coditions I have found:

• When the Theia explorer view is created (at frontend reload or when explicitly closed and then re-opened in a test case), it can take some time for its file nodes to appear in the UI. Some tests were assuming that once the view was visible, they would be too [1]
• Similarly, it can take some time for the Explorer's view tab to be created, after the view is. Some test cases were checking immediately for the view tab being visible, and failing when it was not [2]
• Also related to the explorer view, in at least one testcase, method "selectTreeNode" was used to select a file node and assumed that it would be selected immediately after clicking on it [3]

closes #12063

[1] Example of a testcase failing because of the expectation that file tree nodes should be immediately visible after opening the explorer view:

1) ../../src/tests/theia-text-editor.test.ts:38:9 › Theia Text Editor › should be visible and active after opening "sample.txt" 

    Error: TheiaExplorerView is empty.

       at ../../src/theia-app.ts:114

      112 |             const fileStatElements = await explorer.visibleFileStatNodes(DOT_FILES_FILTER);
      113 |             if (fileStatElements.length < 1) {
    > 114 |                 throw Error('TheiaExplorerView is empty.');
          |                       ^
      115 |             }
      116 |         }
      117 |         const fileNode = await explorer.fileStatNode(filePath);

        at TheiaApp.openEditor (/home/<user>/theia/examples/playwright/src/theia-app.ts:114:23)
        at /home/<user>/theia/examples/playwright/src/tests/theia-text-editor.test.ts:39:34

[2] Example of a testcase failing because the Explorer view tab is not immediately visible after Explorer view creation:

  1) ../../src/tests/theia-explorer-view.test.ts:49:9 › Theia Explorer View › should be possible to close and reopen it 

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: true
    Received: false

      52 |
      53 |         explorer = await app.openView(TheiaExplorerView);
    > 54 |         expect(await explorer.isTabVisible()).toBe(true);
         |                                               ^
      55 |         expect(await explorer.isDisplayed()).toBe(true);
      56 |         expect(await explorer.isActive()).toBe(true);
      57 |     });

        at /home/<user>/theia/examples/playwright/src/tests/theia-explorer-view.test.ts:54:47

[3] Example of failed testcase where explorer tree node selection was tested before it became effective:

  1) ../../src/tests/theia-explorer-view.test.ts:57:9 › Theia Explorer View › should show one folder named "sampleFolder" and one file named "sample.txt" 

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: true
    Received: false

      58 |         await explorer.waitForTreeNodesVisible();
      59 |         await explorer.selectTreeNode('sampleFolder');
    > 60 |         expect(await explorer.isTreeNodeSelected('sampleFolder')).toBe(true);
         |                                                                   ^
      61 |         const fileStatElements = await explorer.visibleFileStatNodes(DOT_FILES_FILTER);
      62 |         expect(fileStatElements.length).toBe(2);
      63 |

        at /home/<user>/theia/examples/playwright/src/tests/theia-explorer-view.test.ts:60:67

How to test

Make sure the playwright test suite passes

Review checklist

• As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• As a reviewer, I agree to behave in accordance with the review guidelines

[planger]

Thank you very much! 👍
Code looks reasonable and I ran the tests a few times locally and didn't observe any intermittent failures.

[marcdumais-work]

Thank you very much! +1 Code looks reasonable and I ran the tests a few times locally and didn't observe any intermittent failures.

Thanks for the quick review!