build: introduce a 'prepare' script in embark's package.json

** TL;DR **

These changes affect workflow with yarn. To prevent embark's 'prepare' script
from running undesirably:

- If node_modules is in place and you're reinstalling after switching branches:

  yarn run install_all

- If node_modules is missing (fresh clone or deleted):

  EMBARK_NO_PREPARE=t yarn install && cd embark-ui && yarn install && cd ..

It's not recommended to set EMBARK_NO_PREPARE in your environment
(e.g. in '.bashrc') since that would interfere with embark's 'release' script
if/when you run it.

-----------------

1. Specify embark's build-related steps in the 'prepare' script of
package.json.

When embark is installed directly from GitHub the 'prepare' script results in a
"pre install" phase (handled automatically by npm/yarn) that fetches
devDependencies, builds embark (including embark-ui), packs a tarball with the
same steps (minus testing and tree-checking) as would happen during an embark
release, and finally does a production install from that tarball.

Important point: installs from GitHub must be performed with yarn; they're no
longer possible with npm since during the "pre install" phase npm will honor
embark's .npmrc and "engines" settings.

The following will work correctly after this commit is merged:

  yarn add git+https://github.com/embark-framework/embark.git

Use of "hosted git" shortcuts (e.g. 'embark-framework/embark#bracnh') won't
work correctly because yarn doesn't fully support them:

  yarnpkg/yarn#5235

It's important to use 'git+https' urls. Following a succesful install with
'git+https' it is possible to use a "hosted git" shortcut or 'https' url, but
that's owing to a subtle and unreliable interaction between yarn's cache and
yarn's logic for installing from a url/shortcut.

2. Adjust the npm configs (.npmrc) for embark/-ui so that 'yarn run [cmd]
[--opt]' can be used in place of 'npm run [cmd] -- [--opt]'.

Either way is okay for running scripts, they're equivalent, but note the
requirement to use '--' before specifying command options with 'npm run'.

3. Introduce yarn configs (.yarnrc) for embark/-ui and include the
'check-files' directive.

H/t to @alaibe for the recommendation.

4. Ignore embark's 'dist/typings' and 'scripts' directories when packing a
tarball.

5. Refactor embark/-ui's npm-scripts in relation to the 'prepare' script, and
make other small improvements.

Notably, if the environment variable EMBARK_NO_PREPARE is truthy (from JS
perspective) then embark's 'prepare' script will exit early. This prevents
'install_all' and 'prepare' from getting stuck in a loop ('install:core' uses
cross-env to set EMBARK_NO_PREPARE) and provides a mechanism for users to skip
the 'prepare' script when doing a fresh install:

  EMBARK_NO_PREPARE=t yarn install

6. Give .js extensions to node scripts in embark's 'scripts/', remove the
shebang lines, and have npm-scripts explicitly invoke them with node.

This arrangement works for all platforms: Linux, macOS, and Windows.

7. Adjust travis and appveyor configs.

Since at present there aren't any tests or other CI steps that make use of
embark-ui's production build, set EMBARK_NO_PREPARE in the CI environments and
invoke 'build:node' directly.

Check the working tree after 'yarn install' for embark/-ui. This detects
situations where changes should have been committed to yarn.lock but were
not. Check the working tree again at the end to detect situations where ignore
files should have been adjusted but were not. Both checks could also detect
other surprising behavior that needs to be investigated. Any time the working
tree is not clean (there are untracked files or changes) CI will fail.

Drop CI runs for node 8.11.3 because that version uses an older npm that
results in unstaged changes to the test apps' package-lock.json files, causing
the working tree check to fail at the end of the CI run. A simple workaround
isn't apparent, but the matter can be revisited.

8. Refactor embark's 'release' script in light of the 'prepare' script.

Notably, do the push step only after 'npm publish' completes successfully. This
allows embark's 'prepare' and 'prepublishOnly' scripts to detect problems
before a commit and tag are pushed to GitHub, avoiding a need to rebase/revert
the remote release branch; the local branch will still need to have a commit
dropped and tag deleted before rerunning the 'release' script.

Prompt the user if the 'release' script is not being run in '--dry-run' mode.

Provide additional visual indicators of '--dry-run' mode.

Force the user to supply '--repo-branch [branch]' if the intention is to
release from a branch other than master.

Author: michaelsbradleyjr

.npmignore

@@ -9,10 +9,12 @@ CONTRIBUTING.md
 appveyor.yml
 babel.config.js
 dist/test
+dist/typings
 header.png
 npm-debug.log*
 npm-shrinkwrap.json
 package-lock.json
+scripts
 src
 test_apps
 tsconfig.json

.npmrc

@@ -1,3 +1,4 @@
 engine-strict = true
 package-lock = false
 save-exact = true
+scripts-prepend-node-path = true

.travis.yml

@@ -3,18 +3,20 @@ os:
   - linux
   - osx
 node_js:
-  - "8.11.3"
   - "8"
   - "10"
 before_install:
   - curl -o- -L https://yarnpkg.com/install.sh | bash -s -- --version 1.12.3
   - export PATH="$HOME/.yarn/bin:$HOME/.config/yarn/global/node_modules/.bin:$PATH"
 cache:
   yarn: true
+env:
+  - EMBARK_NO_PREPARE=true
 install:
   - yarn install
   - cd embark-ui && yarn install && cd ..
-  - git status && test -z "$(git status --porcelain)"
+  - npm run check-working-tree
 script:
   - npm run build:node
   - npm test
+  - npm run check-working-tree

.yarnrc

@@ -0,0 +1 @@
+--install.check-files true

appveyor.yml

@@ -1,8 +1,8 @@
 environment:
   matrix:
-    - nodejs_version: "8.11.3"
     - nodejs_version: "8"
     - nodejs_version: "10"
+  EMBARK_NO_PREPARE: true
 cache:
  - "%LOCALAPPDATA%\\Yarn"
 install:
@@ -11,11 +11,13 @@ install:
   - cmd /c start /wait msiexec.exe /i yarn-1.12.3.msi /quiet /qn /norestart
   - rm yarn-1.12.3.msi
   - node --version
+  - npm --version
   - yarn --version
   - yarn install
   - cd embark-ui && yarn install && cd ..
-  - git status && node -e "process.exit(require('child_process').execSync('git status --porcelain').toString().trim()===''?0:1)"
+  - npm run check-working-tree
 test_script:
   - npm run build:node
   - npm test
+  - npm run check-working-tree
 build: off

embark-ui/.npmrc

@@ -1,3 +1,4 @@
 engine-strict = true
 package-lock = false
 save-exact = true
+scripts-prepend-node-path = true

embark-ui/.yarnrc

@@ -0,0 +1 @@
+--install.check-files true

embark-ui/package.json

@@ -72,6 +72,7 @@
     "redux": "4.0.1",
     "redux-saga": "0.16.2",
     "resolve": "1.8.1",
+    "rimraf": "2.6.2",
     "sass-loader": "7.1.0",
     "simple-line-icons": "2.4.1",
     "style-loader": "0.23.0",
@@ -85,6 +86,7 @@
   },
   "scripts": {
     "build": "node scripts/build.js",
+    "clean": "rimraf build",
     "lint": "eslint src/",
     "start": "node scripts/start.js",
     "test": "node scripts/test.js"

embark-ui/yarn.lock

@@ -9364,7 +9364,7 @@ rgba-regex@^1.0.0:
   resolved "https://registry.yarnpkg.com/rgba-regex/-/rgba-regex-1.0.0.tgz#43374e2e2ca0968b0ef1523460b7d730ff22eeb3"
   integrity sha1-QzdOLiyglosO8VI0YLfXMP8i7rM=
 
-rimraf@2, rimraf@^2.2.8, rimraf@^2.5.4, rimraf@^2.6.1, rimraf@^2.6.2, rimraf@~2.6.2:
+rimraf@2, rimraf@2.6.2, rimraf@^2.2.8, rimraf@^2.5.4, rimraf@^2.6.1, rimraf@^2.6.2, rimraf@~2.6.2:
   version "2.6.2"
   resolved "https://registry.yarnpkg.com/rimraf/-/rimraf-2.6.2.tgz#2ed8150d24a16ea8651e6d6ef0f47c4158ce7a36"
   integrity sha512-lreewLK/BlghmxtfH36YYVg1i8IAce4TI7oao75I1g245+6BctqTVQiBP3YUJ9C6DQOXJmkYR9X9fCLtCOJc5w==

package.json

@@ -32,28 +32,33 @@
     "build": "npm-run-all build:*",
     "build:node": "npm run babel:node",
     "build:ui": "cd embark-ui && npm run build",
-    "clean": "rimraf dist embark-*.tgz package embark-ui/build",
+    "check-no-prepare": "node scripts/check-no-prepare.js",
+    "check-working-tree": "node scripts/check-working-tree.js",
+    "clean": "npm-run-all clean:*",
+    "clean:core": "rimraf dist embark-*.tgz package",
+    "clean:ui": "cd embark-ui && npm run clean",
     "eslint": "eslint",
-    "install:core": "yarn install",
+    "install:core": "cross-env EMBARK_NO_PREPARE=t yarn install",
     "install:ui": "cd embark-ui && yarn install",
     "install_all": "npm-run-all install:*",
     "lint": "npm-run-all lint:*",
     "lint:js": "npm-run-all lint:js:*",
-    "lint:js:core": "eslint babel.config.js bin/embark src/bin/ src/lib/",
+    "lint:js:core": "eslint babel.config.js bin/embark scripts/ src/bin/ src/lib/",
     "lint:js:ui": "cd embark-ui && npm run lint",
     "lint:ts": "tslint -c tslint.json 'src/**/*.ts'",
-    "prepublishOnly": "npm-run-all clean build test",
-    "test": "npm-run-all lint test:*",
-    "test:core": "mocha dist/test/ --exit --no-timeouts --require source-map-support/register",
-    "test:test_app": "cross-env DAPP=\"test_app\" npm run test_dapp",
-    "test:contracts_app": "cross-env DAPP=\"contracts_app\" npm run test_dapp",
-    "test_dapp": "cross-env-shell \"cd test_apps/$DAPP && npm install && npm test\"",
-    "release": "./scripts/release",
+    "prepare": "npm run --silent check-no-prepare && npm-run-all install_all clean build || exit 0",
+    "prepublishOnly": "npm-run-all test check-working-tree",
+    "release": "node scripts/release.js",
     "start": "run-p start:*",
     "start:embark": "run-p start:embark:*",
     "start:embark:babel": "npm run babel:watch",
     "start:embark:type-check": "npm run type-check:watch",
     "start:ui": "cd embark-ui && npm run start",
+    "test": "npm-run-all lint test:*",
+    "test:core": "mocha dist/test/ --exit --no-timeouts --require source-map-support/register",
+    "test:test_app": "cross-env DAPP=test_app npm run test_dapp",
+    "test:contracts_app": "cross-env DAPP=contracts_app npm run test_dapp",
+    "test_dapp": "cross-env-shell \"cd test_apps/$DAPP && npm install && npm test\"",
     "tsc": "tsc",
     "tslint": "tslint",
     "type-check": "tsc",

scripts/check-no-prepare.js

@@ -0,0 +1,3 @@
+/* global process */
+
+process.exit(process.env.EMBARK_NO_PREPARE ? 1 : 0);

scripts/check-working-tree.js

@@ -0,0 +1,12 @@
+/* global process require */
+
+const {execSync} = require('child_process');
+
+try {
+  execSync('git status', {stdio: 'inherit'});
+  process.exit(
+    execSync('git status --porcelain').toString().trim() === '' ? 0 : 1
+  );
+} catch (e) {
+  process.exit(1);
+}