bug: v4.12.2 breaks E2E tests

[nioe]

Prerequisites

• I have read the Contributing Guidelines.
• I agree to follow the Code of Conduct.
• I have searched for existing issues that already report this problem, without success.

Stencil Version

4.12.2

Current Behavior

We use an async global script, which worked totally fine for us until we updated from v4.12.1 to v4.12.2. After the update, nearly all our E2E Tests started to fail. After some digging I stumbled upon this PR #5158. Seems that there was a bug which caused the initialization of the components not to wait for an async global script. Since this was fixed, I assume that our E2E tests try to access the components too early now, which results in errors like Node is either not clickable or not an Element or shadow root does not exist for element because the components are not actually loaded yet.

Expected Behavior

First, even the PR was marked as breaking. So personally, I do not understand why it was published as a bug-fix release...

IMHO the await newE2EPage() call should wait for the async global script too.

System Info

      System: node 20.10.0
    Platform: windows (10.0.19045)
   CPU Model: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 cpus)
    Compiler: .\e2e-bug\node_modules\@stencil\core\compiler\stencil.js
       Build: 1707755243
     Stencil: 4.12.2
  TypeScript: 5.3.3
      Rollup: 2.56.3
      Parse5: 7.1.2
      jQuery: 4.0.0-pre
      Terser: 5.27.0

Steps to Reproduce

• Create a new Stencil project with npm init stencil@latest
• Create async global script with some delay and add it to stencil.config.ts
• Run tests npm run test
• Default E2E Test fails with shadow root does not exist for element: my-component since the component is not yet ready

   my-component › renders changes to the name data
  
    shadow root does not exist for element: my-component
  
      15 |     await page.setContent('<my-component></my-component>');
      16 |     const component = await page.find('my-component');
    > 17 |     const element = await page.find('my-component >>> div');
         |                     ^
      18 |     expect(element.textContent).toEqual(`Hello, World! I'm `);
      19 |
      20 |     component.setProperty('first', 'James');

Code Reproduction URL

https://github.com/nioe/stencil-e2e-bug

Additional Information

No response

[christian-bromann]

Thanks @nioe for reporting the issue. I will close this as

duplicate of #5457

And asked to add this issue into our next sprint for us to take a look. In the meantime please pin the stencil dependency to v4.12.1. At the time where I worked on the PR I didn't recognised this would cause an issue like this.

[nioe]

Thanks @christian-bromann for the quick reply. I came across the other issue but figured that even if it's possibly the same root cause, it's a different problem which needs to be addressed.
It's ok for me to close my issue, as long as both problems get solved eventually.

[christian-bromann]

@nioe yes, I will make sure to run your reproduction case as well.

[nioe]

Thanks a lot for your effort @christian-bromann.

[christian-bromann]

@nioe sorry for the delayed response. I was able to find the root cause of this issue while understanding the details of similar issues reported to Stencil. In your reproduction case you have a global script that is being invoked when the application is loaded. This means that the component rendering is also delayed for the time being until the global script has finished. You can fix this problem using one of the following two options:

1. delay test execution after you set the content until the global script is done, e.g.

diff --git a/src/components/my-component/my-component.e2e.ts b/src/components/my-component/my-component.e2e.ts
index d7ff311..fd3545b 100644
--- a/src/components/my-component/my-component.e2e.ts
+++ b/src/components/my-component/my-component.e2e.ts
@@ -5,6 +5,7 @@ describe('my-component', () => {
     const page = await newE2EPage();

     await page.setContent('<my-component></my-component>');
+    await new Promise(resolve => setTimeout(resolve, 1100));
     const element = await page.find('my-component');
     expect(element).toHaveClass('hydrated');
   });
@@ -13,6 +14,7 @@ describe('my-component', () => {
     const page = await newE2EPage();

     await page.setContent('<my-component></my-component>');
+    await new Promise(resolve => setTimeout(resolve, 1100));
     const component = await page.find('my-component');
     const element = await page.find('my-component >>> div');
     expect(element.textContent).toEqual(`Hello, World! I'm `);

2. keep your global script synchronous and run the asynchronous part in an extra function:

diff --git a/src/global/global-script.ts b/src/global/global-script.ts
index 05d71d8..f156bf8 100644
--- a/src/global/global-script.ts
+++ b/src/global/global-script.ts
@@ -1,9 +1,13 @@
-const globalScript = async (): Promise<void> => {
-  console.log('Global script started');
+const asyncGlobalScript = async (): Promise<void> => {
   return new Promise<void>(resolve => setTimeout(() => {
-    console.log('Global script ended');
+    console.log('Async Global script ended');
     resolve();
   }, 1000));
+}
+
+const globalScript = async (): Promise<void> => {
+  console.log('Global script started');
+  asyncGlobalScript();
 };

-export default globalScript;
\ No newline at end of file
+export default globalScript;

I will check in with the team to see how we can be more explicit in our docs about this behavior.

[nioe]

Hi @christian-bromann

Thank you for the detailed answer.
Both solutions seem kind of hacky to me.

Since the E2E Testing approach is baked into Stencil and an async global script is clearly mentioned as a valid case, I would assume that the testing framework handels this case automatically.

That said, our components do not function correctly as long as the async part of the global script has finished and waiting for an arbitrary amount of time in our tests will slow down our builds...

Both solutions are therefore not really practical for us.

Is there no way the the E2EPage could wait for the async global script to finish?

[christian-bromann]

Is there no way the the E2EPage could wait for the async global script to finish?

Unfortunately not, once the file with the component source code is loaded, everything comes to an halt. I would strongly advise to use any global scripts with async procedures as these will also delay the rendering of the components in your application. Therefor I suggested to run an asyncGlobalScript function without awaiting it.

[nioe]

Hi @christian-bromann

Since the components add hydrated to its class list once they are completely loaded, you could at least provide a helper method on the E2EPage:

const waitForComponentToBeReady = (selector: string) => this.waitForSelector(`${selector}.hydrated`);

Like this it would be a rather simple change within our tests. After setting the page content we could call the wait function:

const page = await newE2EPage();
await page.setContent(`<my-component></my-component>`);
await page.waitForComponentToBeReady('my-component');

// test logic

What do you think?

[christian-bromann]

@nioe that seems a viable idea to look into. I raised a PR but need to investigate why some end-to-end tests are failing.

[christian-bromann]

@nioe unfortunately this can't be build into Stencil because it is not guaranteed that the element is hydrated with a .hydrated class as the user can customize this behavior. I will raise a PR in the docs to better document this.

[nioe]

Hi @christian-bromann

Thanks for your investigation and your answer.
As we did not customize the behaviour we came up with a solution that works for us:

We extended the E2EPage object and added a new function called findComponent. It's similar to the original find function but it waits for the given selector to have the hydrated class.

Like this, the changes in our tests are rather small and it seems to work for us in all scenarios.