Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: route runfiles of js_library/JsInfo 1p linked dep through npm_package_store and npm_link_package_store #2027

Merged
merged 3 commits into from
Dec 4, 2024

Conversation

gregmagolan
Copy link
Member

@gregmagolan gregmagolan commented Dec 3, 2024

Solves the following case: jfirebaugh/rules_ts@d0e8522.

Failure in that minimal repro is:

$ bazel test //examples/js_lib_pkg/b:test                                     
INFO: Analyzed target //examples/js_lib_pkg/b:test (0 packages loaded, 0 targets configured).
FAIL: //examples/js_lib_pkg/b:test (see /private/var/tmp/_bazel_john/21a27cee0c79a1509a094219b285fcfa/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/examples/js_lib_pkg/b/test/test.log)
INFO: From Testing //examples/js_lib_pkg/b:test:
==================== Test output for //examples/js_lib_pkg/b:test:
node:internal/fs/utils:356
    throw err;
    ^

Error: ENOENT: no such file or directory, open '/private/var/tmp/_bazel_john/21a27cee0c79a1509a094219b285fcfa/sandbox/darwin-sandbox/26/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/examples/js_lib_pkg/b/test_/test.runfiles/_main/examples/js_lib_pkg/a/test.txt'
    at Object.openSync (node:fs:596:3)
    at readFileSync (node:fs:464:35)
    at Object.<anonymous> (/private/var/tmp/_bazel_john/21a27cee0c79a1509a094219b285fcfa/sandbox/darwin-sandbox/26/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/examples/js_lib_pkg/b/test_/test.runfiles/_main/examples/js_lib_pkg/a/index.js:5:43)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/private/var/tmp/_bazel_john/21a27cee0c79a1509a094219b285fcfa/sandbox/darwin-sandbox/26/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/examples/js_lib_pkg/b/test_/test.runfiles/_main/examples/js_lib_pkg/b/test.js:2:18) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/private/var/tmp/_bazel_john/21a27cee0c79a1509a094219b285fcfa/sandbox/darwin-sandbox/26/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/examples/js_lib_pkg/b/test_/test.runfiles/_main/examples/js_lib_pkg/a/test.txt'
}

The fix is to pipe runfiles through npm_package_store and npm_link_package_store so they make it down the chain of js_library -> npm_package_store -> npm_link_package_store to binaries depending on the npm_link_package_store (the :node_modules/foo target).

For reference, the rules in the minimal repro look like this:

//examples/js_lib_pkg/b:node_modules/js_lib_pkg_a is a npm_link_package_store who's src is //examples:.aspect_rules_js/node_modules/js_lib_pkg_a@0.0.0:

$ bazel query //examples/js_lib_pkg/b:node_modules/js_lib_pkg_a --output=build
# /Users/greg/aspect/rules/js/rules_ts/examples/js_lib_pkg/b/BUILD.bazel:4:22
npm_link_package_store(
  name = "node_modules/js_lib_pkg_a",
  visibility = ["//visibility:public"],
  tags = ["manual"],
  generator_name = "node_modules",
  generator_function = "npm_link_all_packages",
  generator_location = "examples/js_lib_pkg/b/BUILD.bazel:4:22",
  src = "//examples:.aspect_rules_js/node_modules/js_lib_pkg_a@0.0.0",
)

//examples:.aspect_rules_js/node_modules/js_lib_pkg_a@0.0.0 is an npm_link_package_store who's src is //examples/js_lib_pkg/a:pkg:

$ bazel query //examples:.aspect_rules_js/node_modules/js_lib_pkg_a@0.0.0 --output=build
# /Users/greg/aspect/rules/js/rules_ts/examples/BUILD.bazel:52:22
npm_package_store(
  name = ".aspect_rules_js/node_modules/js_lib_pkg_a@0.0.0",
  visibility = ["//visibility:public"],
  tags = ["manual"],
  generator_name = "node_modules",
  generator_function = "npm_link_all_packages",
  generator_location = "examples/BUILD.bazel:52:22",
  src = "//examples/js_lib_pkg/a:pkg",
  deps = {"//examples:.aspect_rules_js/node_modules/@types+node@22.9.0": "@types/node"},
  package = "js_lib_pkg_a",
  version = "0.0.0",
)

and finally //examples/js_lib_pkg/a:pkg is the js_library being linked to node_modules/js_lib_pkg_a:

$ bazel query //examples/js_lib_pkg/a:pkg --output=build
# /Users/greg/aspect/rules/js/rules_ts/examples/js_lib_pkg/a/BUILD.bazel:19:11
js_library(
  name = "pkg",
  visibility = ["//visibility:public"],
  generator_name = "pkg",
  generator_function = "js_library",
  generator_location = "examples/js_lib_pkg/a/BUILD.bazel:19:11",
  srcs = ["//examples/js_lib_pkg/a:ts"],
  types = [],
)

@gregmagolan gregmagolan requested a review from jbedard December 3, 2024 07:51
Copy link

aspect-workflows bot commented Dec 3, 2024

Test

All tests were cache hits

212 tests (100.0%) were fully cached saving 34s.


Test

e2e/bzlmod

All tests were cache hits

5 tests (100.0%) were fully cached saving 857ms.


Test

e2e/gyp_no_install_script

All tests were cache hits

2 tests (100.0%) were fully cached saving 503ms.


Test

e2e/js_image_oci

All tests were cache hits

1 test (100.0%) was fully cached saving 7s.


Test

e2e/npm_link_package

All tests were cache hits

3 tests (100.0%) were fully cached saving 820ms.


Test

e2e/npm_link_package-esm

All tests were cache hits

3 tests (100.0%) were fully cached saving 1s.


Test

e2e/npm_translate_lock

All tests were cache hits

1 test (100.0%) was fully cached saving 34ms.


Test

e2e/npm_translate_lock_empty

All tests were cache hits

1 test (100.0%) was fully cached saving 34ms.


Test

e2e/npm_translate_lock_multi

All tests were cache hits

2 tests (100.0%) were fully cached saving 171ms.


Test

e2e/npm_translate_lock_partial_clone

All tests were cache hits

1 test (100.0%) was fully cached saving 129ms.


Test

e2e/npm_translate_lock_replace_packages

All tests were cache hits

3 tests (100.0%) were fully cached saving 503ms.


Test

e2e/npm_translate_lock_subdir_patch

All tests were cache hits

1 test (100.0%) was fully cached saving 222ms.


Test

e2e/npm_translate_package_lock

All tests were cache hits

1 test (100.0%) was fully cached saving 131ms.


Test

e2e/npm_translate_yarn_lock

All tests were cache hits

1 test (100.0%) was fully cached saving 131ms.


Test

e2e/package_json_module

All tests were cache hits

1 test (100.0%) was fully cached saving 590ms.


Test

e2e/pnpm_lockfiles

All tests were cache hits

40 tests (100.0%) were fully cached saving 3s.


Test

e2e/pnpm_workspace

All tests were cache hits

10 tests (100.0%) were fully cached saving 3s.


Test

e2e/pnpm_workspace_rerooted

All tests were cache hits

12 tests (100.0%) were fully cached saving 2s.


Test

e2e/repo_mapping

All tests were cache hits

2 tests (100.0%) were fully cached saving 474ms.


Test

e2e/rules_foo

All tests were cache hits

2 tests (100.0%) were fully cached saving 470ms.


Test

e2e/runfiles

All tests were cache hits

1 test (100.0%) was fully cached saving 443ms.


Test

e2e/vendored_node

All tests were cache hits

1 test (100.0%) was fully cached saving 199ms.


Buildifier      Format

@jbedard
Copy link
Member

jbedard commented Dec 3, 2024

Can we also add that test?

@jbedard jbedard force-pushed the pipe_runfiles_of_1p_linked_dep_through_npm_link branch from efc9162 to c3188f9 Compare December 3, 2024 21:00
@jbedard jbedard enabled auto-merge (squash) December 3, 2024 21:01
@gregmagolan gregmagolan force-pushed the pipe_runfiles_of_1p_linked_dep_through_npm_link branch from 7b8d76d to 4c1a2ad Compare December 4, 2024 06:21
@gregmagolan gregmagolan disabled auto-merge December 4, 2024 06:25
@gregmagolan gregmagolan merged commit 20b5216 into main Dec 4, 2024
93 of 102 checks passed
@gregmagolan gregmagolan deleted the pipe_runfiles_of_1p_linked_dep_through_npm_link branch December 4, 2024 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants