download from node-versions and fallback to node dist #147

bryanmacfarlane · 2020-04-24T13:00:44Z

Node distributions are now available from https://github.com/actions/node-versions for reliability since the releases are backed by a CDN. That repo offers a versions-manifest.json which abstracts away the location of each version. Therefore in the future it could move from releases to some other CDN or endpoint. This set of changes queries the versions-manifest against the node version semver spec provided by the workflow to resolved to latest for that spec. If not found in the versions-manifest, it falls back to the node dist ( https://nodejs.org/dist/index.json )

updated tests

bryanmacfarlane · 2020-05-04T13:41:22Z

src/installer.ts

+  if (toolPath) {
+    console.log(`Found in cache @ ${toolPath}`);
+  } else {
+    console.log(`Attempting to download ${versionSpec}...`);


I added much more output to the logs. The previous impl was pretty much sllent

bryanmacfarlane · 2020-05-04T13:42:26Z

src/installer.ts

+    let info = await getInfoFromManifest(versionSpec, stable, token);
+    if (!info) {
+      console.log(
+        'Not found in manifest.  Falling back to download directly from Node'


LTS lineages (8,10,12,14) will come from manifest while non-LTS/old versions will still fall through to node dist. We can choose to pull more into manifest if needed.

compute is also going to do separate changes so the latest of each LTS is pre-cached on the image as part of image gen

bryanmacfarlane · 2020-05-04T13:44:01Z

src/installer.ts

+      downloadPath = await tc.downloadTool(info.downloadUrl, undefined, token);
+    } catch (err) {
+      if (err instanceof tc.HTTPError && err.httpStatusCode == 404) {
+        await acquireNodeFromFallbackLocation(info.resolvedVersion);


a few really old versions like 0.12.8 will have a different layout without a targz or 7z. it will have just a node binary

bryanmacfarlane · 2020-05-04T13:45:44Z

src/installer.ts

+      let _7zPath = path.join(__dirname, '..', 'externals', '7zr.exe');
+      extPath = await tc.extract7z(downloadPath, undefined, _7zPath);
+      // 7z extracts to folder matching file name
+      let nestedPath = path.join(extPath, path.basename(info.fileName, '.7z'));


node dist creates a 7z with a nested folder matching the filename in the unzipped structure. The manifest extracts without that. That's a good thing because later I'm considering extracting directly into the cache instead of unzip and copy. Copy on windows consumes the vast majority of the time.

It looks like unzip + copy may be related to the extra folder after unzipping.

Move is unsafe on Windows because defender commonly still has a lock reading the newly created files on disk. Copy is safe.

bryanmacfarlane · 2020-05-04T13:47:33Z

src/installer.ts

+      extPath = await tc.extractTar(downloadPath, undefined, [
+        'xz',
+        '--strip',
+        '1'


same here. strip 1 will strip the root folder if it exists. I validated it won't if it doesn't. Note that I extended the interface to accept string | string[] which should be fine for compat.

bryanmacfarlane · 2020-05-04T13:48:12Z

src/installer.ts

+    'actions',
+    'node-versions',
+    token,
+    'update-versions-manifest-file' // TODO: remove after testing


I will remove after Maxim merges his last set of changes today. That's the branch to pull it from and after removing it will default to master.

bryanmacfarlane · 2020-05-04T13:49:35Z

src/installer.ts

+    return null;
+  }
+
+  //


This is all mostly the same code - just moved into another function for fallback.

bryanmacfarlane · 2020-05-04T13:51:06Z

src/main.ts

+import * as installer from './installer';
+import * as auth from './authutil';
+import * as path from 'path';
+


moved logic into a main for testing. Mostly the same logic.

bryanmacfarlane · 2020-05-04T13:51:46Z

src/main.ts

+    //
+    let version = core.getInput('node-version');
+    if (!version) {
+      version = core.getInput('version');


even though this is officially deprecated, I left it in because I wasn't in the mood to break folks as part of these larger changes.

ericsciple · 2020-05-04T16:47:16Z

__tests__/authutil.test.ts

-    }
+  beforeEach(async () => {
+    await io.rmRF(rcFile);
+    // if (fs.existsSync(rcFile)) {


joshmgross · 2020-05-04T17:06:50Z

src/installer.ts

+  version = semver.clean(version) || '';
+  let fileName: string =
+    osPlat == 'win32'
+      ? `node-v${version}-win-${osArch}`


If version is an empty string, won't this file name be invalid?

joshmgross

LGTM. Thanks for all the review comments, makes it a lot easier to get context on the changes

ericsciple · 2020-05-05T18:41:57Z

__tests__/__snapshots__/authutil.test.ts.snap

@@ -1,31 +0,0 @@
-// Jest Snapshot v1, https://goo.gl/fbAQLP


i never understood the snapshot thing. Nice to see it deleted :) one less thing to learn

ericsciple · 2020-05-05T19:03:36Z

src/main.ts

+
+    console.log(`version: ${version}`);
+    if (version) {
+      let token = core.getInput('token');


Consider validating token is supplied

In checkout, I have seen customers with token: '' i think it's related to the marketplace copy/paste

ericsciple · 2020-05-05T19:06:30Z

src/main.ts

+    const registryUrl: string = core.getInput('registry-url');
+    const alwaysAuth: string = core.getInput('always-auth');
+    if (registryUrl) {
+      auth.configAuthentication(registryUrl, alwaysAuth);


should this be await auth.configAuthentication? The implementation looks "sync" but unit tests await

ericsciple · 2020-05-05T19:15:15Z

src/installer.ts

+
+    let downloadPath = '';
+    try {
+      downloadPath = await tc.downloadTool(info.downloadUrl, undefined, token);


shouldnt this use info.token?

when downloading from nodejs.org, shouldnt use a token

or would it better to check the hostname of the download url to determine whether to pass token?

Anglemmj7 · 2023-06-10T09:40:50Z

Node distributions are now available from https://github.com/actions/node-versions for reliability since the releases are backed by a CDN. That repo offers a versions-manifest.json which abstracts away the location of each version. Therefore in the future it could move from releases to some other CDN or endpoint. This set of changes queries the versions-manifest against the node version semver spec provided by the workflow to resolved to latest for that spec. If not found in the versions-manifest, it falls back to the node dist ( https://nodejs.org/dist/index.json )

Bumps [husky](https://github.com/typicode/husky) from 5.0.9 to 5.1.0. - [Release notes](https://github.com/typicode/husky/releases) - [Commits](typicode/husky@v5.0.9...v5.1.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

bryanmacfarlane added 10 commits April 24, 2020 08:58

wip

ecf0ce6

updated tests

8885b71

update ts-jest

889def3

dbg

ec6092f

format

1c2f59f

cya snapshot

22c0aea

format

8a1d983

dist

efab84a

Merge pull request #153 from actions/start-v2

ed3a918

updated tests

strip 1 on fallback extract

2bdb2ab

bryanmacfarlane marked this pull request as ready for review May 2, 2020 20:31

bryanmacfarlane added 19 commits May 2, 2020 16:34

Merge branch 'master' into node-versions

501fd29

dist

e3ad114

update workflows

2fb08c4

update

5e5ef8f

win change

1b2d431

win change

c8617ac

dbg

42746a4

dbg

a414557

dbg

f7c5caf

only node binary issue

dd6f5ab

no proxy issue

11f9205

testing 7z alt

beb2155

testing 7z alt

9fcef3f

dbg

6a1b66a

dbg

8e22109

dbg

5e7b076

no proxy fix

a80a170

output info

a875da2

output info

0a9e8b1

bryanmacfarlane commented May 4, 2020

View reviewed changes

src/installer.ts

return null;

}

//

Copy link

Member Author

bryanmacfarlane May 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all mostly the same code - just moved into another function for fallback.

bryanmacfarlane commented May 4, 2020

View reviewed changes

bryanmacfarlane requested review from ericsciple, joshmgross and maxim-lobanov May 4, 2020 13:56

back to master

2ae9264

ericsciple reviewed May 4, 2020

View reviewed changes

__tests__/authutil.test.ts

}

beforeEach(async () => {

await io.rmRF(rcFile);

// if (fs.existsSync(rcFile)) {

Copy link

Collaborator

ericsciple May 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dead code

joshmgross reviewed May 4, 2020

View reviewed changes

joshmgross approved these changes May 4, 2020

View reviewed changes

ericsciple reviewed May 5, 2020

View reviewed changes

ericsciple approved these changes May 5, 2020

View reviewed changes

support ghes (#157)

e73ffbc

bryanmacfarlane merged commit e434342 into master May 19, 2020

kenhuang mentioned this pull request Jun 4, 2020

Reduce latency when new node versions are released #160

Closed

joshmgross mentioned this pull request Nov 2, 2021

Log Node.js version using #90

Closed

havenchyk mentioned this pull request Nov 29, 2021

Print node, npm and yarn versions after installation #368

Merged

2 tasks

ndxbn mentioned this pull request Jun 25, 2022

Update actions/setup-node action to v3 sanabutton/sanabutton.github.io#917

Merged

1 task

Anglemmj7 mentioned this pull request Jun 10, 2023

Node distribution #778

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

download from node-versions and fallback to node dist #147

download from node-versions and fallback to node dist #147

bryanmacfarlane commented Apr 24, 2020 •

edited

Loading

bryanmacfarlane May 4, 2020

bryanmacfarlane May 4, 2020

bryanmacfarlane May 4, 2020

bryanmacfarlane May 4, 2020

bryanmacfarlane May 4, 2020

ericsciple May 5, 2020

bryanmacfarlane May 4, 2020

bryanmacfarlane May 4, 2020 •

edited

Loading

bryanmacfarlane May 4, 2020

bryanmacfarlane May 4, 2020

bryanmacfarlane May 4, 2020

ericsciple May 4, 2020

joshmgross May 4, 2020

joshmgross left a comment

ericsciple May 5, 2020

ericsciple May 5, 2020 •

edited

Loading

ericsciple May 5, 2020

ericsciple May 5, 2020

ericsciple May 5, 2020

ericsciple May 5, 2020

Anglemmj7 commented Jun 10, 2023

download from node-versions and fallback to node dist #147

download from node-versions and fallback to node dist #147

Conversation

bryanmacfarlane commented Apr 24, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bryanmacfarlane May 4, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joshmgross left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericsciple May 5, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Anglemmj7 commented Jun 10, 2023

bryanmacfarlane commented Apr 24, 2020 •

edited

Loading

bryanmacfarlane May 4, 2020 •

edited

Loading

ericsciple May 5, 2020 •

edited

Loading