Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support async iterable declarations in idlharness #24266

Conversation

MattiasBuelens
Copy link
Contributor

@MattiasBuelens MattiasBuelens commented Jun 21, 2020

Currently, the idlharness incorrectly interprets async iterable declarations (as used in the Streams spec) as if they were regular iterables. This PR fixes that by adding separate testing logic for such async iterable declarations.

I also added tests for the entries, keys, values and forEach methods of pair iterable declarations, to match the new tests for the methods on asynchronous pair iterables.

Fixes #24174

@MattiasBuelens
Copy link
Contributor Author

Looks like Chrome is failing some of the new tests for methods of a pair iterable. Specifically, the keys, values and forEach methods of MediaKeyStatusMap have an empty string as their function object name.
image

Firefox and Safari pass these new tests successfully.

Should I file a bug with Chromium for this? (I'm new to this. 😅)

@stephenmcgruer
Copy link
Contributor

Should I file a bug with Chromium for this? (I'm new to this. 😅)

Yes, please do! https://crbug.com/new , much appreciated :)

Copy link
Member

@domenic domenic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a potential addition.

Note that this closes #24174, and that you may want to remove some of the tests in https://github.com/web-platform-tests/wpt/blob/master/streams/readable-streams/async-iterator.any.js that become redundant with this, either as part of this PR or as a followup.

resources/idlharness.js Show resolved Hide resolved
@MattiasBuelens
Copy link
Contributor Author

MattiasBuelens commented Jun 22, 2020

Yes, please do! https://crbug.com/new , much appreciated :)

Done: Chromium bug #1097814. 🙂

Note that this closes #24174

Oh woops, I should probably have searched through the issues first. 😅 Thanks, I've updated the PR description so it will auto-close that issue.

and that you may want to remove some of the tests in https://github.com/web-platform-tests/wpt/blob/master/streams/readable-streams/async-iterator.any.js that become redundant with this

I think only this test would be duplicated? The other tests are related to the prototype of the async iterator, whereas the tests in this PR are about the prototype of the async iterable interface.

Most notably, the tests in this PR never actually create an async iterator. Perhaps we could add that, so we can also cover this test with the WebIDL harness tests.

@domenic
Copy link
Member

domenic commented Jun 22, 2020

I think only this test would be duplicated?

Agreed.

Most notably, the tests in this PR never actually create an async iterator. Perhaps we could add that, so we can also cover this test with the WebIDL harness tests.

Yeah, although it's a bit tricky, since there's no guarantee that an async iterator will always be creatable. You'd probably want to add a factory mechanism similar to what exists for interfaces. Best left as a followup, I think.

@domenic
Copy link
Member

domenic commented Jun 22, 2020

I guess this will need admin-merging due to macOS infrastructure problems? @stephenmcgruer and @jgraham are already in this thread, so hopefully they can do that.

@stephenmcgruer
Copy link
Contributor

stephenmcgruer commented Jun 22, 2020

MacOS failures are non-blocking, this needs Taskcluster to finish. Will keep an eye on that :)

@stephenmcgruer
Copy link
Contributor

Both chrome-dev stability and firefox-nightly stability failed.

Chrome-dev was layout-instability:

67:55.98 INFO ## Unstable results ##
67:55.98 INFO |                  Test                 |                                                       Subtest                                                        |          Results           |                                                                      Messages                                                                     |
67:55.98 INFO |---------------------------------------|----------------------------------------------------------------------------------------------------------------------|----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|
67:55.98 INFO | `/layout-instability/idlharness.html` | `idl_test setup`                                                                                                     | **FAIL: 1/10, PASS: 9/10** | `promise_test: Unhandled rejection with value: "Timed out waiting for LayoutShift entry"`                                                         |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShift must be primary interface of layoutShift`                                                               | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShift is not defined"`            |
67:55.98 INFO | `/layout-instability/idlharness.html` | `Stringification of layoutShift`                                                                                     | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShift is not defined"`            |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShift interface: layoutShift must inherit property "value" with the proper type`                              | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShift is not defined"`            |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShift interface: layoutShift must inherit property "hadRecentInput" with the proper type`                     | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShift is not defined"`            |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShift interface: layoutShift must inherit property "lastInputTime" with the proper type`                      | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShift is not defined"`            |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShift interface: layoutShift must inherit property "sources" with the proper type`                            | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShift is not defined"`            |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShift interface: layoutShift must inherit property "toJSON()" with the proper type`                           | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShift is not defined"`            |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShift interface: default toJSON operation on layoutShift`                                                     | **FAIL: 1/10, PASS: 9/10** | `Cannot read property 'toJSON' of undefined`                                                                                                      |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShiftAttribution must be primary interface of layoutShiftAttribution`                                         | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShiftAttribution is not defined"` |
67:55.98 INFO | `/layout-instability/idlharness.html` | `Stringification of layoutShiftAttribution`                                                                          | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShiftAttribution is not defined"` |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShiftAttribution interface: layoutShiftAttribution must inherit property "node" with the proper type`         | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShiftAttribution is not defined"` |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShiftAttribution interface: layoutShiftAttribution must inherit property "previousRect" with the proper type` | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShiftAttribution is not defined"` |
67:55.98 INFO | `/layout-instability/idlharness.html` | `LayoutShiftAttribution interface: layoutShiftAttribution must inherit property "currentRect" with the proper type`  | **FAIL: 1/10, PASS: 9/10** | `assert_equals: Unexpected exception when evaluating object expected null but got object "ReferenceError: layoutShiftAttribution is not defined"` |
67:55.98 INFO 

Firefox stability check timed out (common if there are lots of tests affected).

The Chrome change looks like pre-existing flake, I would guess, so admin-merging this.

@stephenmcgruer stephenmcgruer merged commit 29b173a into web-platform-tests:master Jun 22, 2020
@MattiasBuelens MattiasBuelens deleted the idlharness-async-iterable branch June 22, 2020 22:18
ricea pushed a commit that referenced this pull request Jul 3, 2020
As noted in #24266, this Streams test is now covered by the new async
iterable tests from the WebIDL harness. Therefore, we can remove the
Streams-specific test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

idlharness does not support async iterators
5 participants