Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add test-262 script #13

Merged
merged 11 commits into from
Aug 27, 2021
Merged

test: add test-262 script #13

merged 11 commits into from
Aug 27, 2021

Conversation

dnalborczyk
Copy link
Contributor

@dnalborczyk dnalborczyk commented Aug 26, 2021

adds test-262 runner. based on: https://github.com/acornjs/acorn-class-fields/blob/89a1c7a6795e0fdac3e8fc8587ee6e1b9ea67dd0/run_test262.js also bumped deps and added LTS versions to ci.

test-262 import assertions:
tc39/test262#2980

test-262 json modules:
tc39/test262#2999

added the failing tests to the whitelist: 3ead374#diff-0760fe08ac8e4dce6d416f554fcb99850cb731a9df9d37cd3743b9bf7bf80a52R14

not sure if all test-262 issues can be fixed and subsequently removed from the whitelist?

@xtuc xtuc closed this Aug 26, 2021
@xtuc xtuc reopened this Aug 26, 2021
@xtuc
Copy link
Owner

xtuc commented Aug 26, 2021

Not related to your change but GitHub actions complains with:


The workflow is not valid. .github/workflows/main.yml (Line: 1, Col: 1): Unexpected value 'ame'

@xtuc
Copy link
Owner

xtuc commented Aug 26, 2021

CI fails with:

2021-08-26T08:21:47.0777328Z npm ERR! Error while executing:
2021-08-26T08:21:47.0779563Z npm ERR! /usr/bin/git ls-remote -h -t ssh://git@github.com/tc39/test262.git
2021-08-26T08:21:47.0781414Z npm ERR! 
2021-08-26T08:21:47.0783324Z npm ERR! Warning: Permanently added the RSA host key for IP address '140.82.114.3' to the list of known hosts.
2021-08-26T08:21:47.0784542Z npm ERR! git@github.com: Permission denied (publickey).
2021-08-26T08:21:47.0785671Z npm ERR! fatal: Could not read from remote repository.
2021-08-26T08:21:47.0786667Z npm ERR! 
2021-08-26T08:21:47.0787504Z npm ERR! Please make sure you have the correct access rights
2021-08-26T08:21:47.0788423Z npm ERR! and the repository exists.
2021-08-26T08:21:47.0789094Z npm ERR! 
2021-08-26T08:21:47.0789787Z npm ERR! exited with error code: 128
2021-08-26T08:21:49.9580977Z 
2021-08-26T08:21:49.9599609Z npm ERR! A complete log of this run can be found in:
2021-08-26T08:21:49.9609713Z npm ERR!     /home/runner/.npm/_logs/2021-08-26T08_21_47_079Z-debug.log
2021-08-26T08:21:49.9743112Z ##[error]Process completed with exit code 1.

do you have any idea where that's coming from?

@dnalborczyk
Copy link
Contributor Author

dnalborczyk commented Aug 26, 2021

I forgot to add a reference to the test262 repository: https://github.com/acornjs/acorn-class-fields/blob/89a1c7a6795e0fdac3e8fc8587ee6e1b9ea67dd0/package.json#L30 . I pinned it to the current latest commit.

not sure if that's the issue tho, the ci is not running (needs approval). I got the ci running in the forked repo. I [temporary] removed package-lock and yarn-lock, and changed the ci to use install instead of ci. there seems to be a different issue now. still looking ...

@xtuc
Copy link
Owner

xtuc commented Aug 26, 2021

it failed with now

Run npm run test:test262

> acorn-import-assertions@1.7.6 test:test262 /home/runner/work/acorn-import-assertions/acorn-import-assertions
> node run_test262.js

internal/modules/cjs/loader.js:303
      throw err;
      ^

Error: Cannot find module '/home/runner/work/acorn-import-assertions/acorn-import-assertions/lib/index.js'. Please verify that the package.json has a valid "main" entry
    at tryPackage (internal/modules/cjs/loader.js:295:19)
    at Function.Module._findPath (internal/modules/cjs/loader.js:508:18)
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:802:27)
    at Function.Module._load (internal/modules/cjs/loader.js:667:27)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (/home/runner/work/acorn-import-assertions/acorn-import-assertions/run_test262.js:8:36)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32) {
  code: 'MODULE_NOT_FOUND',
  path: '/home/runner/work/acorn-import-assertions/acorn-import-assertions/package.json',
  requestPath: '.'
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! acorn-import-assertions@1.7.6 test:test262: `node run_test262.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the acorn-import-assertions@1.7.6 test:test262 script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

I can help debug later today.

@dnalborczyk
Copy link
Contributor Author

dnalborczyk commented Aug 26, 2021

I'm guessing the build step is missing, otherwise the lib folder does not exist and therefore the main entry point for cjs is invalid. I'll give it a try.

@dnalborczyk
Copy link
Contributor Author

yap, that was the issue. 🙌

@dnalborczyk
Copy link
Contributor Author

the current test-262 returns the following:

> acorn-import-assertions@1.7.6 test:test262 /home/runner/work/acorn-import-assertions/acorn-import-assertions
> node run_test262.js

Testing complete.
Summary:
 ✔ 31 valid programs parsed without error
 ✔ 7 invalid programs produced a parsing error
 ✔ 4 invalid programs did not produce a parsing error (and allowed by the whitelist file)
 ✔ 127 valid programs produced a parsing error (and allowed by the whitelist file)
 ✔ 80495 programs were skipped

Copy link
Owner

@xtuc xtuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@xtuc xtuc merged commit 7519566 into xtuc:main Aug 27, 2021
@dnalborczyk dnalborczyk deleted the dn/test-262 branch August 27, 2021 15:59
alexandrenicol added a commit to alexandrenicol/acorn-import-attributes that referenced this pull request Mar 27, 2024
Fixes the `test:test262` step of the Github Action flow.

Should fix: xtuc#27

- updating the `test262` dependency
- updating the files whitelist in `run_test262.js`

- This was preventing the Github Actions to succeed
- Therefore preventing Yarn to public the new tag release
- Therefore preventing Webpack to update to any version above 1.9.0

```
@alexandrenicol ➜ /workspaces/acorn-import-attributes (fix-test-262) $ npm run test:test262

> acorn-import-attributes@1.9.3 test:test262
> node run_test262.js

Testing complete.
Summary:
 ✔ 71 valid programs parsed without error
 ✔ 5 invalid programs produced a parsing error
 ✔ 10 invalid programs did not produce a parsing error (and allowed by the whitelist file)
 ✔ 111 valid programs produced a parsing error (and allowed by the whitelist file)
 ✔ 97695 programs were skipped
```

Which is similar to what it was when first introduced in xtuc#13 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants