Skip to content

Commit

Permalink
[ci][playwright] Solidify flaky Playwright tests
Browse files Browse the repository at this point in the history
This is an attempt at solidifying the playwright test suite, that has had testcases
intermitently failing. I have been lucky to be able to observe some race conditions
rather consistently locally, and so be able to debug them and solidify against them.

The race conditions are very slight - the Playwright test suite takes a screenshot
at every failure, and consistently I see the correct state on those, meaning that
the state is what it should be, by the time the screenshot is taken.

In consequence, I wound that, for a given flaky scenario, addressing one race
consition would often mask a second one, present in the same scenario. It's
possible that under a different execution context, more race conditions would
occur. Here are the race coditions I have found:

- When the Theia explorer view is created (at frontend reload or when explicitly
closed and then re-opened in a test case), it can take some time for its file nodes
to appear in the UI. Some tests were assuming that once the view was visible, they
would be too.
- Similarly, it can take some time for the Explorer's view tab to be created, after
the view is. Some test cases were checking immediately for the view tab being
visible, and failing when it was not.
- Also related to the explorer view, method "selectTreeNode" was used to select a file
node and assumed that it would be selected imemdiately after clicking on it.

closes #12063

Signed-off-by: Marc Dumais <marc.dumais@ericsson.com>
  • Loading branch information
marcdumais-work committed Apr 4, 2023
1 parent 943dd91 commit 96ba4c7
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 0 deletions.
1 change: 1 addition & 0 deletions examples/playwright/src/tests/theia-explorer-view.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ test.describe('Theia Explorer View', () => {
const ws = new TheiaWorkspace(['src/tests/resources/sample-files1']);
app = await TheiaApp.load(page, ws);
explorer = await app.openView(TheiaExplorerView);
await explorer.waitForVisibleFileNodes();
});

test('should be visible and active after being opened', async () => {
Expand Down
1 change: 1 addition & 0 deletions examples/playwright/src/theia-app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ export class TheiaApp {
throw Error('TheiaExplorerView could not be opened.');
}
if (expectFileNodes) {
await explorer.waitForVisibleFileNodes();
const fileStatElements = await explorer.visibleFileStatNodes(DOT_FILES_FILTER);
if (fileStatElements.length < 1) {
throw Error('TheiaExplorerView is empty.');
Expand Down
18 changes: 18 additions & 0 deletions examples/playwright/src/theia-explorer-view.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,10 @@ export class TheiaExplorerView extends TheiaView {
await treeNode.focus();
} else {
await treeNode.click({ modifiers: ['Control'] });
// make sure the click has been acted-upon before returning
while (!await this.isTreeNodeSelected(filePath)) {
console.debug('Waiting for clicked tree node to be selected: ' + filePath);
}
}
}

Expand Down Expand Up @@ -231,4 +235,18 @@ export class TheiaExplorerView extends TheiaView {
await this.refresh();
}

override async waitForVisible(): Promise<void> {
await super.waitForVisible();
await this.page.waitForSelector(this.tabSelector, { state: 'visible' });
}

/**
* Waits until some file nodes are visible, for non-dot files
*/
async waitForVisibleFileNodes(): Promise<void> {
while ((await this.visibleFileStatNodes(DOT_FILES_FILTER)).length === 0) {
console.debug('Awaiting for tree nodes to appear');
}
}

}

0 comments on commit 96ba4c7

Please sign in to comment.