Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: rebuild test addons conditionally #335

Closed
wants to merge 1 commit into from

Conversation

bnoordhuis
Copy link
Member

Before this commit, make test-addons and the targets that depend on
it (test-ci, test-all) would always do a full rebuild of the files in
test/addons and the addons scraped from doc/api/addons.markdown.

This commit introduces a proper dependency chain so that files are only
rebuilt when changed, shaving off about 10-20 seconds from each run.

R=@chrisdickinson

Before this commit, `make test-addons` and the targets that depend on
it (test-ci, test-all) would always do a full rebuild of the files in
test/addons and the addons scraped from doc/api/addons.markdown.

This commit introduces a proper dependency chain so that files are only
rebuilt when changed, shaving off about 10-20 seconds from each run.
$(NODE_EXE) $(ADDONS_TESTS:%=test/addons/%/build/Release/binding.node)

# Note: don't depend on $(NODE_EXE) here, that unconditionally forces a rebuild.
test/addons/.stamp: tools/doc/addon-verify.js doc/api/addons.markdown
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't look like addon-verify is run, so those doc tests won't be built?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Which I'm fine with for now, by the by, I'm not a huge fan of doctests.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, perhaps I made it too obtuse for the sake of DRY. In the line below, $< is replaced with tools/doc/addon-verify.js.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still breaks on OSX for me on first run -- running ./$(NODE_EXE) addon-verify.js in a separate target to create the test/addon/doc-* dirs that the test/addons/.stamp target depends on should fix this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! That's the problem -- it evaluates and expands $(foreach dir... before executing addon-verify.js -- and thus it can't see those new directories, and the test fails because node-gyp hasn't run in those dirs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this is a make issue... I see what you are saying but I can only reproduce it on OS X with make 3.81. On Linux with make 4.0, it always builds things in the right order, no matter how many jobs I throw at it. Will investigate.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't reproduce it on OS X anymore either, I wonder what changed... @chrisdickinson Can you give it one more try? If it still fails for you, can you tell me how to reproduce?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To reproduce you have to rm -rf test/addons/doc-* between test runs.

On Jan 16, 2015, at 4:04 AM, Ben Noordhuis notifications@github.com wrote:

In Makefile:

  • at-exit \
  • hello-world-function-export \
  • hello-world \
  • repl-domain-abort \

+# Note: don't depend on $(NODE_EXE) here, that unconditionally forces a rebuild.
+test/addons/%/build/Release/binding.node: \

  •   test/addons/%/binding.cc test/addons/%/binding.gyp
    
  • ./$(NODE_EXE) deps/npm/node_modules/node-gyp/bin/node-gyp rebuild \
  •   --nodedir="$(PWD)" --directory="$(dir $<)"
    
    +build-static-addons: \
  • $(NODE_EXE) $(ADDONS_TESTS:%=test/addons/%/build/Release/binding.node)

+# Note: don't depend on $(NODE_EXE) here, that unconditionally forces a rebuild.
+test/addons/.stamp: tools/doc/addon-verify.js doc/api/addons.markdown
I can't reproduce it on OS X anymore either, I wonder what changed... @chrisdickinson Can you give it one more try? If it still fails for you, can you tell me how to reproduce?


Reply to this email directly or view it on GitHub.

@chrisdickinson
Copy link
Contributor

To recap: the issue is that this expands before any targets are run. The directories that expansion looks for are created by the previous step. On clean builds, this fails because the generated directories aren't present. On subsequent re-runs, the directories are there and the tests pass.

@jbergstroem
Copy link
Member

I just tried this on my mac as well (10.10 should that matter, gnu make 3.81) - suspecting that is was an parallelism issue (gmake has .NOTPARALLEL if needed). Can't reproduce your error @chrisdickinson. I'd really like to sort this out to be able to get jenkins running test-ci. How can we proceed? @chrisdickinson could you possibly guide from an assumed empty repo?

@mscdex mscdex added the build Issues and PRs related to build files or the CI. label Mar 22, 2015
@bnoordhuis
Copy link
Member Author

Closing, stale.

@bnoordhuis bnoordhuis closed this Jun 26, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants