Long task with synchronous pieces does not time out

[bensaufley]

Describe the bug

Basically I've got a synchronous function that I want to test is reasonably fast with a large amount of data. I understand the issue of not being able to interrupt the process for the synchronous task, and saw for example another issue mentioning the problem with an infinite loop—but this function does eventually resolve. If I've set a timeout for 150ms and it takes 5 seconds to resolve, I would expect the test to fail as timed out, but it doesn't, no matter how long it takes.

I suppose this may be "behaving as expected" as timeout is only intended to interrupt asynchronous tasks if it can and does nothing if it can't interrupt, but it's definitely unintuitive to me that a test with a 150ms timeout that resolves in 5 seconds is considered to have passed.

For my particular use case I did a const start = Date.now(); … expect(Date.now() - start).toBeLessThan(150); and I'm sure that will probably be the response here, but I think it's unintuitive that the distinction between synchronous and asynchronous code (even synchronous code within a promise) will change whether the same behavior is considered to have timed out.

Reproduction

Times out as I'd expect, because (as far as I understand, and I admit I'm a little fuzzy) setTimeout does not monopolize ticks and allows the test process to intervene:

it('times out', async () => {
  await new Promise((resolve) => {
    setTimeout(resolve, 5_000);
  });
}, 150);

Does not time out, and does not fail once completed:

it('does not time out', async () => {
  const start = Date.now();
  await new Promise((resolve) => {
    while (Date.now() - start < 5_000) {}
    resolve();
  });
}, 150);

Even this, which occurred to me while writing this issue, does not result in a timeout:

it('does not time out even with a promise after completion allowing the test runner to intervene', async () => {
  const start = Date.now();
  while (Date.now() - start < 5_000) {}
  await new Promise((resolve) => {
    setImmediate(resolve); // setTimeout(resolve, 0 or even 10) has no effect either
  });
}, 150);

System Info

System:
    OS: macOS 13.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 85.13 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 18.10.0 - ~/.nodenv/versions/18.10.0/bin/node
    npm: 8.19.2 - ~/.nodenv/versions/18.10.0/bin/npm
  Browsers:
    Firefox: 109.0.1
    Safari: 16.2
  npmPackages:
    @vitest/coverage-c8: ^0.24.5 => 0.24.5 
    vite: ^3.2.4 => 3.2.4 
    vitest: ^0.24.5 => 0.24.5

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[bensaufley]

If it helps, here's a repo that I just threw together. Vite/vitest versions are absolute latest https://github.com/bensaufley/vitest-timeout-example

[bensaufley]

Found that putting the synchronous task inside the promise, with setImmediate, produces a timeout:

  it("times out", async () => {
    const start = Date.now();
    await new Promise<void>((resolve) => {
      while (Date.now() - start < 5_000) {}
      setImmediate(resolve);
    });
  }, 150);

I'm 100% certain there are system-level reasons for why there are distinctions here, but from a usage standpoint, the distinction between longRunningSync(); promise(setImmediate()) and promise(longRunningSync(); setImmediate()) for the purpose of testing and determining whether to time out seems … unexpected?

[bensaufley]

^ just discovered that this last strategy does not work for my actual use case, which … actually makes no sense, because it's the same thing except instead of the while loop it's calling a function with a for loop inside. Completes in ~3 seconds, and shows it:

[Yokozuna59]

Hi there! Is there any update on this issue? I came across a similar issue: an infinite loop inside a synchronous function, and the testTimeout didn't kill it.

[sheremet-va]

This is just how JavaScript engine works, I don't what else to update here. while loop doesn't empty the event queue so setTimeout cannot be run to stop the test.

[bensaufley]

The originally reported issue is not that the task isn't canceled (which I agree is just the nature of JS); it's that when the task completes, it isn't marked as timed out, despite taking much longer than the specified time.

[sheremet-va]

The originally reported issue is not that the task isn't canceled (which I agree is just the nature of JS); it's that when the task completes, it isn't marked as timed out, despite taking much longer than the specified time.

As I said, setTimeout cannot be called while the test is running synchronously, it „messes“ up the timers. To JS it did not took 5 seconds.

This is the nature of js. Test time is not checked after the test has finished, the fail is scheduled before test starts with setTimeout to timeout early.

[bensaufley]

Test time is not checked after the test has finished

And this is the core of the issue. I explained all of this above:

I suppose this may be "behaving as expected" as timeout is only intended to interrupt asynchronous tasks if it can and does nothing if it can't interrupt, but it's definitely unintuitive to me that a test with a 150ms timeout that resolves in 5 seconds is considered to have passed.

If we're considering timeout to only mean "canceled before execution could finish" then timeout may not be the best way to frame that.

To JS it did lot took 5 seconds.

This is just incorrect. JS knows what time it took. Vite even reports that time in the output. That's where the heart of this issue is.

[Yokozuna59]

This is just how JavaScript engine works, I don't what else to update here. while loop doesn't empty the event queue so setTimeout cannot be run to stop the test.

Then what about the assertion itself? I used except.objectContaing to validate that a given object containing some variables and arrays , but it went into an infinite loop, although there is a valid returned value from except.objectContaing.

The code was some thing like this:

// This is just a sample script. Paste your real code (javascript or HTML) here.
it('title', () => {
    const obj = someFunction();
    except(object).toEqual(
        except.objectContaining({
            sample1: [],
            sample2: [],
            sample3: {
                foo: undefined,
                bar: undefined,
                title: undefined,
            },
        })
    });
});

Note: I used toBe and toEqual but same issue.

[bensaufley]

That doesn't seem related to this discussion.