Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rollup_bundle should have a stamp attr to control if that target should ever use stamping #1693

Closed
joeljeske opened this issue Mar 6, 2020 · 10 comments · Fixed by #2160
Closed
Labels
Can Close? We will close this in 30 days if there is no further activity

Comments

@joeljeske
Copy link
Contributor

joeljeske commented Mar 6, 2020

🚀 feature request

Relevant Rules

rollup_bundle(
  stamp = 0 # or 1
)

Description

rollup_bundle (and I presume other packaging steps like pkg_npm) include the version file as a dep whenever --stamp is passed. I believe that bazel typically pairs this stamping config with a stamp attr on each rule that might use stamping info. This allows targets to declare that they never use stamping information, even if --stamp is passed (for other targets).

Describe the solution you'd like

If you have a solution in mind, please describe it.
rollup_bundle(
  stamp = 0 # or 1
)

If stamp is 0, never include the version file. If stamp is 1, include the version file if --stamp is passed.

Describe alternatives you've considered

Have you considered any alternative solutions or workarounds?

This should be supported inside rollup_bundle, not attempted to be patched from the outside.

@alexeagle
Copy link
Collaborator

I'm surprised that you'd want a build with --stamp in which some targets don't get stamped. What's the use case for that?
Also you say " I believe that bazel typically pairs this stamping config" - could you point to any examples of that?

@joeljeske
Copy link
Contributor Author

I'm referring to the behavior that I understand from java_binary where stamp = 1|0|-1

Integer; optional; default is -1

Enable link stamping. Whether to encode build information into the binary. Possible values:

stamp = 1: Stamp the build information into the binary. Stamped binaries are only rebuilt when their dependencies change. Use this if there are tests that depend on the build information.
stamp = 0: Always replace build information by constant values. This gives good build result caching.
stamp = -1: Embedding of build information is controlled by the --[no]stamp flag.

My use case is that I have lots of rollup_bundles for each of my intermediate packages. These packages are linked together at the end in a web archive using importmaps and systemjs without re-processing or re-bundling. Most of these rollup_bundles don't leverage stamping since they are not the final product, however, I have some downstream tasks (like pkg_npm) that I want stamping enabled. So I guess the issue is that the rollup_bundle is presuming that the presence of --stamp is always relevant itself and thus it uses the volatile version file as a dep even though I will never use it in some rules. This makes it always misses the remote cache (bazelbuild/bazel#10075) which makes it less performant than it could be

@joeljeske
Copy link
Contributor Author

Any more thoughts on this?

@alexeagle
Copy link
Collaborator

Oops, finally cleaning up stamping now. #1441 gave us the same behavior as the java_binary default of -1 - if you pass --stamp to the build, we sense that (via a config_setting like rules_go) and add the stamp vars for rollup to read. This is like rules_go. IMO it's not a good idea that every stamping-capable rule has to repeat a stamp attribute, do you know why that's useful on java_binary?

@alexeagle alexeagle added the Can Close? We will close this in 30 days if there is no further activity label Aug 31, 2020
@joeljeske
Copy link
Contributor Author

It's valuable for rules to be able to always opt out of stamping for a given target (0 behavior). It's also valuable to have the 1 behavior, since then one could control stamping or not based on anything configurable and not just the stamp flag.

@alexeagle
Copy link
Collaborator

It makes the docs more complex for users, so I'd still like more data on why those are valuable features vs. only having --stamp to control it.

@joeljeske
Copy link
Contributor Author

Often, we may want to stamp some stamp some things in a build, but not other things. The --stamp is a global flag that makes all things be stamped. Just because some rule supports stamping, does not mean I want to stamp every usage of the rule and invalidate downstream caching.

@alexeagle
Copy link
Collaborator

Oh, actually there's already a way to do this, it's just less obvious than what you propose.

https://github.com/bazelbuild/rules_nodejs/blob/ba4fdb83a05b1e7cb0af7f92dbd2229f5eda2586/internal/pkg_npm/test/BUILD.bazel#L49 shows how I stamp an action even without --stamp - you just need to pass a NodeContextInfo to any stamp-aware rule

@alexeagle
Copy link
Collaborator

alexeagle commented Aug 31, 2020

WDYT about resolving this issue by updating the docs for node_context_data attributes, instead of just "Internal use only" it could say

Provides info about the build context, such as stamping. By default it reads from the bazel command line, such as the --stamp argument. Use this to override values for this target, such as enabling or disabling stamping.

but not document this in the stamping.md user doc because I think it's probably just confusing there

@joeljeske
Copy link
Contributor Author

Yea that could work fine then, I would just like to be able to control the stamping beyond the global flag. Sounds good!

alexeagle pushed a commit to alexeagle/rules_nodejs that referenced this issue Aug 31, 2020
It can be useful if you need to turn on/off stamping per-target.
`java_binary` gives a dedicated stamp attribute for this purpose

Fixes bazel-contrib#1693
alexeagle pushed a commit to alexeagle/rules_nodejs that referenced this issue Aug 31, 2020
It can be useful if you need to turn on/off stamping per-target.
`java_binary` gives a dedicated stamp attribute for this purpose

Fixes bazel-contrib#1693
alexeagle pushed a commit that referenced this issue Aug 31, 2020
It can be useful if you need to turn on/off stamping per-target.
`java_binary` gives a dedicated stamp attribute for this purpose

Fixes #1693
lherman-cs pushed a commit to lherman-cs/rules_nodejs that referenced this issue Jan 15, 2021
fix: remove duplicate Importing

chore: upgrade rollup to ^2.3.0

fix: coverage (bazel-contrib#2100)

* test: add coverage e2e test

* fix(builtin): fix coverage lcov merger script jumping out of sandbox and not being able to find c8

* test: don't test coverage on Windows

docs: use @npm//@bazel for example load sites

fix(cypress): allow for async cypress plugins

cypress_repository now fails if cypress verify fails
set the HOME env variable since cypress writes files to it
remove unused includeScreenshots / includeVideos variables
browserify files are no longer included in test output files
screenshots/videos are stored directory in TEST_UNDECLARED_OUTPUTS_DIR

docs: update docs for release

chore(release): 2.0.2

chore: update lock files for release

feat(example): add targets in angular_bazel_architect for production serve and build

fix(examples): use ./ prefix on babel config file

Otherwise if it's in a subfolder, babel will think it's an npm package

chore: update dependency com_google_protobuf to v3.13.0 (bazel-contrib#2103)

refactor(rollup): use rollup loadConfigFile API

chore: update dependency bazel_toolchains to v3.4.1

build(deps): bump elliptic in /e2e/fine_grained_symlinks

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.4.1 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](indutny/elliptic@v6.4.1...v6.5.3)

Signed-off-by: dependabot[bot] <support@github.com>

fix(typescript): only expect .js outs for .tsx? srcs (bazel-contrib#2118)

Fixes bazel-contrib#2115

feat(builtin): new js_library rule (bazel-contrib#2109)

Removes `node_module_library` private API and renames it to `js_library`

the latter will be promoted to a public API in a followup change

fix(typescript): produce .d.ts as default output rather than empty (bazel-contrib#2117)

This lets you type-check a ts_project(emit_declaration_only=True) just by building it (its default outputs)

Fixes bazel-contrib#2116

added script to generate NodeJS versions for node_repositories and added most up-to-date versions (bazel-contrib#2126)

Co-authored-by: Matt Insler <matt.insler@airbnb.com>

refactor: move node version list into its own file (bazel-contrib#2128)

Add a package.json script for updating it

docs: update docs for release

chore(release): 2.0.3

chore: update lock files for release

fix(typescript): add the tsBuildInfoFile option to ts_project (bazel-contrib#2138)

Like other options that affect tsc outputs, Bazel needs to know the value from the tsconfig if its set.
Add it to our validator so there's a buildozer command to sync the option value into the BUILD file.

Fixes bazel-contrib#2137

refactor(typescript): remove private _TsConfigInfo in ts_project (bazel-contrib#2139)

Instead use the same TsConfigInfo that ts_library uses.

Fixes bazel-contrib#1931

docs: refresh docsite

docs: minor cleanup of builtins page

don't include the defaults which are giant dictionaries as they render poorly

docs: fix spacing and link colours on rhs sidenav

Fix circleci status badges to pin to stable branch

CircleCI docs claim that they should use the default branch by default, but this icon is green while stable is red, so I think it's still reporting on master.

ci: all angular examples timeout=long

chore: update dependency bazel_toolchains to v3.4.2

chore: update circleci image to latest to pick up newer chrome

docs: add search to docsite using gcse

feat(typescript): generate tsconfig.json for ts_project (bazel-contrib#2130)

This is an experimental, opt-in feature where ts_project will generate a tsconfig.json file in place of
the user specifying one in the source directory.

I expect a long tail of compatibility bugs since any path appearing in this generated file needs to point from
bazel-out back to the source directory.

Fixes bazel-contrib#2058

docs: replace references to //packages/ with the @npm//@bazel/ equivalent (bazel-contrib#2154)

feat(builtin): accept any stamp vars in pkg_npm

NOTE: instead of replacing the replace_with_version with 0.0.0 in unstamped builds,
we now omit the replacement.

See bazel-contrib#1694

docs(rollup): add stamping and debug to rollup doc

Also move most of the content above the list of rules. It shows up better in the TOC since rollup_bundle
can be its own entry later in the page, and avoids the API doc table of attributes just appearing with no header.

docs: document the node_context_data attribute

It can be useful if you need to turn on/off stamping per-target.
`java_binary` gives a dedicated stamp attribute for this purpose

Fixes bazel-contrib#1693

chore: hide generated .html files in code review

feat(builtin): support for substitutions

fix: use golden_file_test instead

refactor: cleanup pkg_web stamping feature

chore: update dependency bazel_toolchains to v3.5.0 (bazel-contrib#2168)

Updated nodejs versions (14.9.0) (bazel-contrib#2169)

docs: update docs for release

chore(release): 2.1.0

chore: update lock files for release

chore: try github actions version of stale bot

The probot one never triggered despite being setup correctly AFAICT.
This bot seems better supported and I sow it working on bazel-watcher repo.

build(deps): bump http-proxy from 1.18.0 to 1.18.1

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](http-party/node-http-proxy@1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump yargs-parser in /e2e/ts_devserver

Bumps [yargs-parser](https://github.com/yargs/yargs-parser) from 13.1.1 to 13.1.2.
- [Release notes](https://github.com/yargs/yargs-parser/releases)
- [Changelog](https://github.com/yargs/yargs-parser/blob/master/docs/CHANGELOG-full.md)
- [Commits](https://github.com/yargs/yargs-parser/commits)

Signed-off-by: dependabot[bot] <support@github.com>

ci: stamp our releases with a stable key

fix(builtin): don't set --preserve-symlinks-main by default (bazel-contrib#2176)

feat: add link_workspace_root to nodejs_binary, npm_package_bin, rollup_bundle, terser_minified, ts_project

Link the workspace root to the bin_dir to support absolute requires like 'my_wksp/path/to/file'.
If source files need to be required then they can be copied to the bin_dir with copy_to_bin.

fix(builtin): fix bazel coverage masking test failures

test(builtin): test failing test with bazel coverage

build(deps): bump http-proxy in /examples/protocol_buffers

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](http-party/node-http-proxy@1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump http-proxy in /examples/web_testing

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](http-party/node-http-proxy@1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <support@github.com>

test(rollup): fix test failing due to missing dep

fix(rollup): allow config files to override default onwarn method

Fixes bazel-contrib#2084

feat: link_workspace_root not needed in terser_minified

docs: add spica to adopter organization list (bazel-contrib#2184)

* Add spica to adopter organization list (bazel-contrib#2183)
* docs: update the 'send a PR' link to point to the stable branch

Co-authored-by: Sahin Yort <thesayyn@gmail.com>

feat: promote js_library to public API

Fixes bazel-contrib#149
Fixes bazel-contrib#1771

docs: update docs for release

chore(release): 2.2.0

chore: update lock files for release

Update readme (bazel-contrib#2190)

Fix missing `}` and indentation

feat: add strict_visibility to npm_install / yarn_install rules (bazel-contrib#2193)

With strict_visibility enabled (currently defaults false), only dependencies within the given `package.json` file are given public visibility.

All transitive dependencies are given limited visibility, enforcing that all direct dependencies are listed in the `package.json` file.

closes bazel-contrib#2110

fix: don't glob yarn or node files when using vendored_node or vendored_yarn

- Fixes a visibility issue with rollup-worker

fix(examples): prevent ibazel EOF

Fixes bazel-contrib#2143

docs(karma): document how to use non-headless Chrome (bazel-contrib#2202)

Fixes bazel-contrib#1881

docs(labs): produce stardoc API for labs

Fixes bazel-contrib#2195

fix(karma): allow custom browsers to specify args (fixes bazel-contrib#595)

Today, the args as specified in the various manifests of the rules_webtesting
browsers never make it into the generated karma.conf.js. This results in these
arguments never being used when launching Chrome, preventing customization of
browsers such as window size, enabling remote debugging, or other flag-based
options. This PR fixes this by reading those arguments from the manifest, and
adding them to the browsers list in the generated karma.conf.js.

Also, this PR includes a change to generated_file_test to allow a golden file
to represent a substring of the generated output.

Also Also: This PR includes a golden file test that verified that the generated
karma.conf.js does read in the specified value.

Furthermore, the effect of this can be verified manually via:
```
VERBOSE_LOGS=1 bazel run packages/karma/test/karma:testing_custom_chrome
```
Note the appearance of the additional flags in the new debug output.

fix(builtin): js_library: correctly propagate DeclarationInfos

Transitive typings are added to typings_depsets, but we were checking
the typings array instead.

Signed-off-by: Duarte Nunes <duarte@hey.com>

feat: update nodejs versions (bazel-contrib#2207)

include 10.22.1, 12.18.4, 14.10.0, 14.10.1, 14.11.0 and 14.12.0 in node_versions

Added s390x support

docs: update docs for release

chore(release): 2.2.1

chore: update lock files for release

ci: tell Angular CLI not to update webdriver

It can get ahead of the Chrome version on our CI machine

ci: replace --webdriver-update=false with --no-webdriverUpdate

Architect CLI uses minimist under the hood to parse arguments. Minimist will not convert boolean like values unless they are known options or the `{boolean: true}` option is configured (Which is not in Architect CLI).

feat(karma): use Trusted Types policy when loading scripts for Karma

When the Karma plugin is used in a testing environment that enforces
Trusted Types, its loadFile functionality currently fails due to a
Trusted Types violation when assigning to script.textContent. This makes
it impossible to use the plugin with integration tests that ensure an
application is compatible with Trusted Types.

This is fixed by creating a Trusted Types policy specifically for the
Karma plugin, and use it to promote any loaded scripts to a TrustedScript
before assigning to script.textContent. This is done in a way that is
backwards compatible:
- The policy is `null` in browsers that don't yet support Trusted Types,
  in which case the original script string is used as before.
- When Trusted Types are supported in the browser but not enforced by
  the application, the browser treats the TrustedScript as if it were a
  string when it is assigned to script.textContent.

chore: update dependency bazel_toolchains to v3.6.0

docs: remove HTML tables so docs can be authored in markdown without need for html escaping

docs(rollup): improve clarity

feat(example): add full pwa support

feat(example): service worker update handling

fix(exmaple): add docstring to ngsw_config rule

fix(example): remove server side compression

fix(example): remove index.html from prodapp srcs

fix(example): remove compression dependencies

fix(builtin): js_library supports --output_groups=types

If providing DeclarationInfo we should also give this way for filegroup/bazel CLI to select them

chore: docgen

docs: update docs for release

chore(release): 2.2.2

chore: update lock files for release

fix(builtin): give a longer timeout for _create_build_files

A user reports that their build file generation takes longer than 600 seconds.
It's probably a bug that we are so slow, but let's address that separately (maybe in our work for npm v7)

Also note we could have made this timeout user-configurable, but I think the extra API surface isn't worth the benefit.

Fixes bazel-contrib#2231

fix(typescript): don't include _valid_options marker file in outs

Tested by building //packages/angular:npm_package and don't get an extra .d.ts file in the package.
Also tested the negative case by making an invalid option and check that it's still reported.

Fixes bazel-contrib#2078

chore: allow node versions <= 14.14.0

chore: update rules_docker to 0.14.4

docs: minor update for readability and formatting (bazel-contrib#2243)

* docs: minor update for readibility and formating

Fix small typo in changelog

chore(docs): move ts_library docs into ts_library

Also recommend ts_project for new uses

chore(karma): remove dependency on tmp

feat(typescript): add allow_js support to ts_project

fix(exmaples/nestjs): add module_name field in ts_library

docs: fix a typo of angular-architect example (bazel-contrib#2272)

chore: bump days before closing stale issues to 90

Fix select with conditons on darwin.

Bazel is changing how @bazel_tools/conditions:darwin is implemented.
With old implementation based on flags, it was ok to have darwin and
darwin_x86_64 in a select. This was based on flag --cpu=darwin and
--cpu=dawin_x86_64.

New implementation is based on constraints, where "darwin" means OS (and
any CPU), while "darwin_x86_64" mean only specific CPU. As such they
cannot be used in the same select, because the selection would be
ambiguous.

build: add 'fix-windows' tag due to windows flaky tests

Will revisit the linker and test, but this is blocking PRs from being merged

feat(typescript): worker mode for ts_project (bazel-contrib#2136)

* feat(typescript): worker mode for ts_project

* chore: cleanup and declare protobufjs dependency

* fix: do not pass --watch for non-worker mode

* fix: do not test worker on windows

* fix: log on standalone failures

* chore: docs improvements

Co-authored-by: Alex Eagle <alex.eagle@robinhood.com>
Co-authored-by: Dan Muller <mrmeku@stairwell.com>

feat(node_repositories): Added auth option for downloading nodejs and yarn

fix: npm_package.pack should work in windows os

feat(examples): adds example for running jest with typescript (bazel-contrib#2245)

fix(typescript): specify rootDir as absolute path

Also allow non-ts files across src/bin dir, since they are not emitted we don't need the rootdir
calculation to take them into account. This lets the Angular compiler accept .ts sources from
source dir even if the css inputs are generated.

refactor: ts_project cleanup

feat(cypress): remove browiserify preprocessor

docs: update docs for release

chore(release): 2.3.0

chore: update lock files for release

fix: npm_package.pack on Windows should not generate undefined.tgz

chore: update nodejs versions (bazel-contrib#2284)

perf(cypress): pack cypress runfiles into a single tar

chore: add GitHub Workflow to update NodeJS versions every night

chore: fix a link

Current link returns 404, it seems that the url was somehow changed to .html from .md.

chore: fix the link again

haha

fix(builtin): make linker deterministic when resolving from manifest & fix link_workspace_root with no runfiles

chore: re-lock jest dependencies

It was using a non-standard repository, which had downtime this morning

chore(release): 2.3.1

chore: update lock files for release

chore: update dependency com_google_protobuf to v3.14.0

chore: update dependency bazel_toolchains to v3.7.0

fix launcher script terminating before child terminates on SIGTERM

chore: update dependency bazel_toolchains to v3.7.1

fix(examples): fix jest example on windows

Fixes bazel-contrib#1454

fix(builtin): give better error when linker runs on Node <10

Example how this looks now:

```
[20 / 21] Bundling JavaScript bundle.js [parcel]; 4s remote-cache, darwin-sandbox
ERROR: /private/var/folders/r7/2kp53v7n091gcz93xlt7wtd80000gn/T/tmp-48948DHy4HrXTwhRy/BUILD.bazel:4:1: Bundling JavaScript bundle.js [parcel] failed (Exit 1) parcel.sh failed: error executing command bazel-out/host/bin/external/npm/parcel-bundler/bin/parcel.sh build foo.js --out-dir bazel-out/darwin-fastbuild/bin --out-file bundle.js ... (remaining 2 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
ERROR: rules_nodejs linker requires Node v10 or greater, but is running on 8.11.2
Note that earlier Node versions are no longer in long-term-support, see
https://nodejs.org/en/about/releases/
INFO: Elapsed time: 40.360s, Critical Path: 12.97s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
//:test                                                               NO STATUS
```

Fixes bazel-contrib#2304

docs(typescript): link some user-contributed doc (bazel-contrib#2322)

Fixes bazel-contrib#2298

chore: allow node versions >=12 <=14 (bazel-contrib#2332)

chore: allow node versions >=12 <=14

feat: add esbuild package

chore: add module_mapping support via the aspect instead of manual implementing it

chore: sqash

feat: update to latest esbuild, add additional flags when not minifying

fix: consume both JSEcmaScriptModuleInfo and JSModuleInfo to support js_library

fix: add missing bzl libraries and publish helpers

test: use runfiles helper in test

fix: use exe for Windows
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Can Close? We will close this in 30 days if there is no further activity
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants