chore(build): Use chunked compilation (#5721)

* chore(build): Add "all" modules for blocks & generators

These modules (Blockly.blocks.all and Blockly.<Generator>.all) will
be the entry points for the corresponding chunks.

They also make it easier to pull in all the modules in each package
(e.g. for playground and tests).

It is necessary to set the Closure Compiler dependency_mode to
SORT_ONLY as otherwise it tries to compile the "all" modules before
their dependencies, which fails.

The only impact on the _compressed.js files is the addition of a short
string to the very end of each file, e.g.:

    var module$exports$Blockly$JavaScript$all={};

* chore(deps): Add devDependency on closure-calculate-chunks

* feat(build): First pass at chunked complation

Add a new buildCompiled gulp target (npm run build:compiled) that
uses closure-calculate-chunks to do chunked compliation of core/,
blocks/ and generators/ all in a single pass.

This work is incomplete: the resulting *_compressed.js files don't
(yet) have UMD wrappers.

* chore(build): Generate chunk wrappers

A first pass; this does not have support for a namespace object yet.

* refactor(build): Use chunked compilation by default

Remove old "compressed" gulp tasks in favour of new "compiled" task.

* chore(build): Remove cruft from buildCompiled

Remove unneeded `done` parameter and commented-out options that had
been cargo-culted from the old build pipeline.

* fix(build): Fix test failures caused by new build pipeline

- Exclude closure/goog/base.js from compiler input; use
  externs/goog-externs.js instead.

- Have the build:debug and build:strict targets only build the first
  chunk (blockly_compressed.js).

- Fix namespace entries for blocks and generators.

* fix(build): Fix build failures on node v12

closure-calculate-chunks requires node.js v14 or later.

When running on node.js v14 or later have getChunkOptions save
the output of closure-calculate-chunks to
scripts/gulpfiles/chunks.json.  When running on older versions of
node.js have it use this checked-in, cached output instead of
attempting to run closure-calculate-chunks.

* chore(build): enable --rename_prefix_namespace

This will allow modules in blocks/ and generators/ to use
goog.require to obtain the exports object of goog.modules from
core/.

* fix(build): Always build all chunks

The previous commit enabled --rename_prefix_namespace option to
Closure Compiler, and this causes the buildCompressed target to
work fine when run without --debug or --strict, but adding either
of those flags (as for example when `npm test` runs
`npm run build:debug`) causes an issue:

- Because of many compiler errors in blocks/ and generators/,
  a previous commit added a hack to only build the first chunk
  when doing debug/strict builds.

- When asked to build only one chunk, Closure Compiler ignores the
  --rename_prefix_namespace flag, because it 'correctly' infers
  that there are no later chunks that will need to access global
  variables from the first chunk.

- This causes a test failure, because `npm test` first runs
  `npm run build`, which generates a valid blockly_compressed.js,
  but this is then overrwritten by an invalid one when it next runs
  `npm run build:debug`.

  (The invalid one is missing all `$.` prefixes on 'global' variables,
  including on Blockly, so the wrapper's last two lines -
  "$.Blockly.internal_ = $;" and "return $.Blockly" - fail.)

The fix is to add appropriate @Suppress annotations to blocks/*.js and
generators/**/*.js and then remove the first-chunk-only hack.

* refactor(build): Just build once

Since the previous commit caused `npm run build:debug` to do
everything that `... build:compressed` does - and to produce
byte-for-byte identical output - it doesn't make sense to run
both when testing.  To that end:

- Replace the build:debug and build:strict package scripts that
  did `gulp buildCompressed --...` with new scripts build-debug
  and build-strict that do `gulp build --...` instead.

  (The target names are changed so as to extend our existing naming
  convention as follows: a target named "foo:bar" does some sub-part
  of the job done by target "foo", but a target named "foo-bar" does
  all the work of the target "foo" with some extra options.)

- build:debug:log and build:strict:log are similarly replaced with
  build-debug-log and build-strict-log.

- Modify run_all_tests.js to just do `npm run build-debug` instead of
  doing both `npm run build` and `npm run build:debug`.

- Also remove the 'build:blocks' script that should have been removed
  when the buildBlocks gulp task was deleted previously.

* refactor(build): Compile with base_minimal.js instead of base.js

Introduce a (very!) cut-down version of closure/goog/base.js named
base_minimal.js that is used as input to the compiler as an
alternative to using externs/goog-externs.js (which will be deleted
once the buildAdvancedCompilationTest target has been updated).

This will allow use of goog.setTestOnly since it will now exist in
compiled mode, and allows the changes made in 5b112db to filter
base.js out of the files for the first chunk to be reverted.
(It also obliges a change to the compiled-mode check in blockly.js.)

* fix(build): Fix buildAdvanceCompilationTest

- In build_tasks.js:
  - Replace the old compile() function with a new one factored out of
    buildCompiled().
  - Update buildAdvancedCompilationTest to use the new compile()
    and other helpers created in the meantime.
  - Remove no-longer-used maybeAddClosureLibrary().

- Remove externs/{block,generator,goog}-externs.js, which are no longer
  used by any compile pipeline.

- Update core/blockly.js to fix issue with detection of compiled mode
  when using ADVANCED_OPTIMISATIONS.

- Update only other use of globalThis, in core/utils/xml.js, to
  consistently treat it as a dictionary object.

- Update instructions in tests/compile/index.html.

This commit is sort-of-a-prerequisite to #5602; test:compile:advanced
was previously working but the generated `main_compresed.js` would
throw errors upon loading.

blocks/all.js

@@ -0,0 +1,23 @@
+/**
+ * @license
+ * Copyright 2021 Google LLC
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+/**
+ * @fileoverview All the blocks.  (Entry point for blocks_compressed.js.)
+ * @suppress {extraRequire}
+ */
+'use strict';
+
+goog.module('Blockly.blocks.all');
+
+goog.require('Blockly.blocks.colour');
+goog.require('Blockly.blocks.lists');
+goog.require('Blockly.blocks.logic');
+goog.require('Blockly.blocks.loops');
+goog.require('Blockly.blocks.math');
+goog.require('Blockly.blocks.procedures');
+goog.require('Blockly.blocks.texts');
+goog.require('Blockly.blocks.variables');
+goog.require('Blockly.blocks.variablesDynamic');

blocks/colour.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Colour blocks for Blockly.
+ * @suppress {extraRequire|missingRequire}
  */
 'use strict';
 

blocks/lists.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview List blocks for Blockly.
+ * @suppress {extraRequire|missingRequire|checkTypes}
  */
 'use strict';
 

blocks/logic.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Logic blocks for Blockly.
+ * @suppress {extraRequire|missingRequire|checkTypes}
  */
 'use strict';
 

blocks/loops.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Loop blocks for Blockly.
+ * @suppress {extraRequire|missingRequire|checkTypes}
  */
 'use strict';
 

blocks/math.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Math blocks for Blockly.
+ * @suppress {extraRequire|missingRequire|checkTypes}
  */
 'use strict';
 

blocks/procedures.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Procedure blocks for Blockly.
+ * @suppress {extraRequire|missingRequire|checkTypes|globalThis|visibility}
  */
 'use strict';
 

blocks/text.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Text blocks for Blockly.
+ * @suppress {extraRequire|missingRequire|checkTypes}
  */
 'use strict';
 

blocks/variables.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Variable blocks for Blockly.
+ * @suppress {extraRequire|missingRequire|checkTypes}
  */
 'use strict';
 

blocks/variables_dynamic.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Variable blocks for Blockly.
+ * @suppress {extraRequire|missingRequire|checkTypes}
  */
 'use strict';
 

closure/goog/base_minimal.js

@@ -0,0 +1,112 @@
+/**
+ * @license
+ * Copyright The Closure Library Authors.
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+/**
+ * @fileoverview A minimal implementation of base.js.
+ *
+ * This file is used in place of base.js (Closure library bootstrap
+ * code) when building Blockly using the Closure Compiler.  Refer to
+ * base.js for more information about items defined here.
+ *
+ * @provideGoog
+ */
+
+/** @define {boolean} Overridden to true by the compiler. */
+var COMPILED = false;
+
+/** @const */
+var goog = goog || {};
+
+/**
+ * Reference to the global object.  This is provided as 'root' by the
+ * UMD wrapper, but prefer globalThis if it is defined.
+ * 
+ * https://www.ecma-international.org/ecma-262/9.0/index.html#sec-global-object
+ *
+ * @const
+ * @type {!Global}
+ * @suppress {undefinedVars}
+ */
+goog.global = globalThis || root;
+
+/** @type {Object<string, (string|number|boolean)>|undefined} */
+goog.global.CLOSURE_DEFINES;
+
+/**
+ * Defines a named value.
+ * When compiled the default can be overridden using the compiler options or the
+ * value set in the CLOSURE_DEFINES object. Returns the defined value so that it
+ * can be used safely in modules. Note that the value type MUST be either
+ * boolean, number, or string.
+ *
+ * @param {string} name
+ * @param {T} defaultValue
+ * @return {T}
+ * @template T
+ */
+goog.define = function(name, defaultValue) {
+  return defaultValue;
+};
+
+/** @define {boolean} */
+goog.DEBUG = goog.define('goog.DEBUG', false);
+
+/** @define {boolean} */
+goog.DISALLOW_TEST_ONLY_CODE =
+    goog.define('goog.DISALLOW_TEST_ONLY_CODE', COMPILED && !goog.DEBUG);
+
+/**
+ * @param {string} name
+ */
+goog.provide = function(name) {};
+
+/**
+ * @param {string} name
+ * @return {void}
+ */
+goog.module = function(name) {};
+
+/**
+ * @param {string} name
+ * @return {?}
+ * @suppress {missingProvide}
+ */
+goog.module.get = function(name) {};
+
+/** @suppress {missingProvide} */
+goog.module.declareLegacyNamespace = function() {};
+
+/**
+ * Marks that the current file should only be used for testing, and never for
+ * live code in production.
+ *
+ * In the case of unit tests, the message may optionally be an exact namespace
+ * for the test (e.g. 'goog.stringTest'). The linter will then ignore the extra
+ * provide (if not explicitly defined in the code).
+ *
+ * @param {string=} opt_message Optional message to add to the error that's
+ *     raised when used in production code.
+ */
+goog.setTestOnly = function(opt_message) {
+  if (goog.DISALLOW_TEST_ONLY_CODE) {
+    opt_message = opt_message || '';
+    throw new Error(
+        'Importing test-only code into non-debug environment' +
+        (opt_message ? ': ' + opt_message : '.'));
+  }
+};
+
+/**
+ * @param {string} namespace
+ * @return {?}
+ */
+goog.require = function(namespace) {};
+
+/**
+ * @param {string} namespace
+ * @return {?}
+ */
+goog.requireType = function(namespace) {};

core/blockly.js

@@ -184,6 +184,11 @@ goog.require('Blockly.Events.VarCreate');
  */
 exports.VERSION = 'uncompiled';
 
+/**
+ * @define {boolean} Overridden to true by the compiler.
+ */
+const COMPILED = false;
+
 // Add a getter and setter pair for Blockly.alert, Blockly.confirm,
 // Blockly.mainWorkspace, Blockly.prompt and Blockly.selected for backwards
 // compatibility.
@@ -700,26 +705,22 @@ exports.zelos = zelos;
 //
 // This is only needed in uncompiled mode (see
 // google/blockly-samples#902); in compiled mode the exports object is
-// already the value of globalThis.Blockly.  Because
-// closure/goog/base.js is not included in the compiler input, we
-// can't use goog.global['COMPILED'] to check if we are running in
-// compiled mode.  Instead, use existence of globalThis.goog itself
-// for this purpose.
+// already the value of globalThis['Blockly'].
 //
 // Note that this code will still attempt to redefine accessors on a
 // previously-imported copy of the Blockly library if both are
 // imported in uncompiled mode.  This will fail with TypeError as the
 // accessors are nonconfigurable (which is good, as otherwise one
 // accessors on one copy would call get/set functions on the other
 // copy!)
-if (globalThis.goog && globalThis.Blockly &&
-    typeof globalThis.Blockly === 'object' && globalThis.Blockly !== exports) {
+if (!COMPILED && typeof globalThis['Blockly'] === 'object' &&
+    globalThis['Blockly'] !== exports) {
   const descriptors = Object.getOwnPropertyDescriptors(exports);
   const accessors = {};
   for (const key in descriptors) {
     if (descriptors[key].get || descriptors[key].set) {
       accessors[key] = descriptors[key];
     }
   }
-  Object.defineProperties(globalThis.Blockly, accessors);
+  Object.defineProperties(globalThis['Blockly'], accessors);
 }

core/utils/xml.js

@@ -36,7 +36,7 @@ exports.NAME_SPACE = NAME_SPACE;
  * jsdom package instead.
  * @type {!Document}
  */
-let xmlDocument = globalThis.document;
+let xmlDocument = globalThis['document'];
 
 /**
  * Get the document object to use for XML serialization.

generators/dart.js

@@ -6,6 +6,7 @@
 
 /**
  * @fileoverview Helper functions for generating Dart for blocks.
+ * @suppress {missingRequire|checkTypes|globalThis}
  */
 'use strict';