Check that tarballs can build

[mhart]

As per nodejs/node#29712

[MylesBorins]

Adding a note here that I think a slightly better solution would be to create the tarball and then build all release assets from the tarballs. That way what we are building and releasing can better match what downstream folks are building

[mistydemeo]

Cross-linking nodejs/node#31858 as another example where this happened.

[rvagg]

someone needs to script this up, it might be a good one to put into GitHub Actions even, but if not, we have our containerised builds where we can put it, but it needs someone with the time work up a script to do a full "build tarball, unpack tarball, build from unpacked tarball, test " cycle.

[jkleinsc]

I can take a look at this. Just to verify - this is something that would run on release CI only?

[rvagg]

@jkleinsc nope, we should just integrate it into ci.nodejs.org or GitHub Actions for each commit. Make sure that nothing introduced during normal dev breaks tarballs—this has happened numerous times in the past and never gets noticed until a release breaks.

We have some miscellaneous things going through GitHub actions already: https://github.com/nodejs/node/tree/master/.github/workflows this might fit, but also might incur built time timeouts? Traditionally we've put build permutations into https://ci.nodejs.org/job/node-test-commit-linux-containered/ and this would certainly fit there, but we're embedding the scripts directly into Jenkins which isn't great. So for now, the easy path is coming up with a script that takes a clean cloned nodejs/node and steps through a full make-tarball (make tar ...), unpack tarball, build from unpacked tarball, test from binary built from tarball. We can then either drop that script into Jenkins, or put it in this repo and run it from a curl|bash which is a pattern we've been adopting more and more with Jenkins.

[jkleinsc]

@rvagg ok makes sense. I think I’ll see if I can get it working as a GitHub action.

[richardlau]

We have some miscellaneous things going through GitHub actions already: https://github.com/nodejs/node/tree/master/.github/workflows this might fit, but also might incur built time timeouts?

Unlikely. GitHub is much more generous over how long a workflow/job can run compared to Travis CI (currently 6 hours per job or 72 hours for a workflow, https://help.github.com/en/actions/getting-started-with-github-actions/about-github-actions#usage-limits, compared to 50 mins for a Travis job).

There was a recently added job to nodejs/node that builds debug and runs ASAN tests that currently takes 3 hours to run.

[jkleinsc]

nodejs/node#32129 should resolve this issue.

[MylesBorins]

awesome work @jkleinsc left a comment on the issue. A thought I had, we should likely still do this directly in CI with the tarball we plan to ship... not that we should be breaking anything with a release commit, but I'd personally be a bit more comfortable if we had the extra gate.

[mhart]

🙌