First 'promise_test' should synchronously invoke it's 'func'.

[wpt-issue-mover]

Originally posted as w3c/testharness.js#159 by @DLehenbauer on 05 Nov 2015, 20:43 UTC:

The common solution to Issue #96 is to use promise_test to execute dependent asynchronous test cases sequentially.

As the Edge team converts tests to TestHarness.js, one pitfall we've discovered is that promise_test() defers invoking its given test func, even if the promise_test is the first in the chain.

For example, this test fails:

var func_is_prompt = true;
promise_test((test) => {
    return new Promise((accept, reject) => {
        assert_true(func_is_prompt, "Promise was created at the time 'promise_test()' was invoked.");
        accept();
    });
}, "First 'promise_test()' is not deferred");
func_is_prompt = false;

Because invoking the func given to the first promise_test() is deferred, you cannot reliably have your first promise_test() wait on a global DOM event like window load.

It's relatively simply to change promise_test() to avoid artificially deferring the first func, which I think makes it more useful as a stand-in for sequential_async_test.

(I'll put up a proposed fix for discussion shortly -- thanks to @zhanbox for noticing the race and helping to vet the fix.)

[wpt-issue-mover]

Originally posted as w3c/testharness.js#159 (comment) by @jyasskin on 06 Nov 2015, 21:50 UTC:

Instead, would it make sense to define a global 'loaded' promise as roughly:

const loaded = new Promise(resolve => { window.addEventListener('load', () => resolve()); });

so you could unconditionally wait on it, instead of depending on which promise_test is listed first?

[wpt-issue-mover]

Originally posted as w3c/testharness.js#159 (comment) by @DLehenbauer on 07 Nov 2015, 19:50 UTC:

It's a reasonable "plan B" if, for some reason, we feel the first promise_test must be deferred. Given issue #115, though, I think it's too late to prevent developers from depending on the order of promise_tests in their code.

If we're open to breaking changes, I would suggest that a better way to address issue #115 is to have promise_test return a Promise that is fulfilled when the test completes (pass or fail), and then have developers express their inter-test dependencies explicitly:

promise_test(() => { ... }, "First test.")
    .then(promise_test(() => { ... }, "2nd test"))
    .then(promise_test(() => { ... }, "3rd test"));

promise_test(() => { ... }, "An independent test.")

(Just a thought.)