Buildtime increased by 50% in v12

[kemsky]

🐞 Bug report

Command (mark with an x)

• build

Description

This is v11.2.13:

$ ng build --prod
√ Browser application bundle generation complete.
√ Copying assets complete.
√ Index html generation complete.

Initial Chunk Files                   | Names         |      Size
main.1bbe17e891e0e81a6bae.js          | main          |   3.20 MB
scripts.5b3322e07a3cdcf6af69.js       | scripts       |   1.48 MB
styles.19406af932cc90ae2667.css       | styles        | 152.44 kB
polyfills-es5.2cb7395a5fd592fe7443.js | polyfills-es5 |  85.88 kB
runtime.23c71f07ba7ec5c3766a.js       | runtime       |   1.51 kB

                                      | Initial Total |   4.92 MB

Build at: 2021-05-14T11:00:10.476Z - Hash: 034e3176df64cbc0b60a - Time: 88357ms

Verbose log (logging itself causes slowdown, unfortunately...):
angular11.txt

Angular CLI: 11.2.13
Node: 14.17.0
OS: win32 x64

Angular: 11.2.13
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router
Ivy Workspace: Yes

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1102.13
@angular-devkit/build-angular   0.1102.13
@angular-devkit/core            11.2.13
@angular-devkit/schematics      11.2.13
@angular/cdk                    11.2.12
@angular/material               11.2.12
@schematics/angular             11.2.13
@schematics/update              0.1102.13
rxjs                            6.6.7
typescript                      4.1.5

This is v12 with yarn.lock and node_modules removed and reinstalled:

$ ng build --configuration production
Warning: Support was requested for IE 11 in the project's browserslist configuration. IE 11 support is deprecated since Angular v12.
For more information, see https://angular.io/guide/browser-support
√ Browser application bundle generation complete.
√ Copying assets complete.
√ Index html generation complete.

Initial Chunk Files                   | Names         |      Size
main.25755cb56d908f1ffa0c.js          | main          |   3.19 MB
scripts.5b3322e07a3cdcf6af69.js       | scripts       |   1.48 MB
styles.51baf6c2a9fbe6837aa1.css       | styles        | 154.28 kB
polyfills-es5.b787b17c5e8d81a4b18e.js | polyfills-es5 |  90.75 kB
runtime.bf3db8034d6e10a74a56.js       | runtime       |   1.20 kB

                                      | Initial Total |   4.92 MB

Build at: 2021-05-14T11:18:16.175Z - Hash: 8806f0a3a3b1271495e0 - Time: 157125ms

Verbose log (logging itself causes slowdown, unfortunately...):
angular12.txt

Angular CLI: 12.0.0
Node: 14.17.0
Package Manager: yarn 1.22.5
OS: win32 x64

Angular: 12.0.0
... animations, cdk, cli, common, compiler, compiler-cli, core
... forms, language-service, material, platform-browser
... platform-browser-dynamic, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1200.0
@angular-devkit/build-angular   12.0.0
@angular-devkit/core            12.0.0
@angular-devkit/schematics      12.0.0
@schematics/angular             12.0.0
rxjs                            6.6.7
typescript                      4.2.4

We have a single small *.scss file (overriding some material colors) and it was updated by migration. The application has only one module. I tried removing scss and css from angular.json, but it is still slower than v11.

On average it is 50% slower.

-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
TypeScript                    1011          10391           2652          54543
HTML                           311            629              7           7832
CSS                             66            344             30           2109
Sass                             1             15             14             63
JavaScript                       1              9              3             55
JSON                             1              0              0              3
-------------------------------------------------------------------------------
SUM:                          1391          11388           2706          64605
-------------------------------------------------------------------------------

---

Recorded profile using node 12:

https://link.nithins.me/s2Uca0Ti

[JoostK]

I am having little luck with loading the profiling events into chrome://tracing (it stays empty).

@kemsky if you are able to run with a debugger and then record a profiling session, that would be very helpful.

[kemsky]

@JoostK, I was not able to load it either. I tried recording on node 14.17 but it crashes at the very beginning. Ok, will try to do it using a debugger.

[JoostK]

After some debugging of Chrome's tracing tool, it appears that the +600MB of tracing data exceeds the maximum string length 🤣 v8/v8@ea56bf5. Importing the file itself just results in an empty string, whereas loading the zip crashes during decompression when all uncompressed bytes are converted into a single string.

[kemsky]

@JoostK, JSON contains a simple array, so you can cut it if necessary into two parts.

[joshwiens]

I'm having the same issue with build times, the initial report looks to be fairly thorough but feel free to @ me if you would like further tracing / info.

[JoostK]

With some sed magic to remove the common path prefix I managed to get the trace size down to fit:

Unfortunately I can't make a comparison with 11 as I don't have a trace for that run, but Webpack processing itself is now longer than the full build took before.

The pruned trace can be downloaded here can be downloaded as a much smaller zip here (only available for a week).

[JoostK]

I am not sure how to proceed here. We'd likely still need some runnable reproduction in order to establish what can be done to improve the situation here. If anyone can share a repro privately that would be super helpful.

[joshwiens]

@JoostK - Mine is a fairly large monorepo, I'll get to work on a repo you can access that is a bit more focused. Give me a hour or two

[JoostK]

@JoostK - Mine is a fairly large monorepo, I'll get to work on a repo you can access that is a bit more focused. Give me a hour or two

Thanks. Also take your time as it's almost 2 AM for me so I'm calling it a day!

[kemsky]

@JoostK, this is v11 profile: https://link.nithins.me/9NfVUS9T

I'm not sure if these are comparable...

Devtools are crashing also, I can only take a partial profile.

[joshwiens]

@kemsky - You can either give v8 more memory globally or chrome itself.

Globally ( imo the useful way ): NODE_OPTIONS=--max_old_space_size=8192
To Chrome: Chrome.app --args --js-flags="--max_old_space_size=8192"

[joshwiens]

@JoostK - I threw a new v11 CLI app up, added material and a basic component so it imports some styles. It's super minimal but should work for a reproduction, if not i'll just get you collab access to my work repo to poke around.

Main branch is cli v11.2.13 (~6000ms --prod) and upgrade-v12 is that branch upgraded. (~13200ms --prod).

https://github.com/EasyMetrics/v12-upgrade-performance

[JoostK]

@JoostK, this is v11 profile: https://link.nithins.me/9NfVUS9T

I'm not sure if these are comparable...

Devtools are crashing also, I can only take a partial profile.

Not sure how to interpret those results. The angular-compiler block stays the same across v11 vs v12 which is to be expected, but what follows is so vastly different that I can't say what the real problem is. Terser is reported surprisingly differently in a way that in v12 it spans 60s vs just 10s in v11, but that may just be because the profiling events are generated differently. Additionally, the v11 trace spans almost 6 minutes vs 3.5 minutes in the v12 trace, so that's suspicious as well (especially because the angular-compiler trace has stayed consistent)

[JoostK]

@JoostK - I threw a new v11 CLI app up, added material and a basic component so it imports some styles. It's super minimal but should work for a reproduction, if not i'll just get you collab access to my work repo to poke around.

Main branch is cli v11.2.13 (~6000ms --prod) and upgrade-v12 is that branch upgraded. (~13200ms --prod).

https://github.com/EasyMetrics/v12-upgrade-performance

Okay so I tried this and I'm actually getting similar times for v11 and v12, however in v11 the Terser processing is cached whereas in v12 it doesn't appear to be. This results in inconsistent results for v11, as cold runs are ~12s vs ~7s for cached runs; unlike v12 which always takes ~12s.

[alan-agius4]

terser-webpack plugin V2 will only cache to disk when using webpack 4. When used with webpack 5 it uses webpacks’ caching API and is dependent on this being set to filesystem to benefit 2nd cold builds.

I am currently experimenting with this and hoping to enable webpacks’ file system caching in the CLI in the near future.

Webpack 4, code path
https://github.com/webpack-contrib/terser-webpack-plugin/blob/version-2/src/Webpack4Cache.js

Webpack 5, code path
https://github.com/webpack-contrib/terser-webpack-plugin/blob/version-2/src/Webpack5Cache.js

[kemsky]

Is it possible to turn off caching in v11?

[alan-agius4]

Yeah, you can use the NG_BUILD_CACHE environment variable.

Ex: NG_BUILD_CACHE=0 ng build

[joshwiens]

If I don't cache my v11 builds in my big monorepo, the build times are almost identical.

tl&dr here is CLI v12 is using Webpack 5, terser-webpack-plugin isn't caching like it does in CLI v11 using Webpack v4 and it just sort of "is what it is" for the moment.

[anton-white]

Same increase of the build time after migrating from v11 to v12.
But here we can see more than 100% of time difference.

Before:

After:

[the-ult]

Same here. For a pretty small project. (2 pages with a couple of components)
V11 was slow.. But v12 is even slower atm.

Might be related to: nrwl/nx#5370

[kemsky]

@JoostK, @alan-agius4, I removed all scss files and set NG_BUILD_CACHE=0, results are below:

v11.2.13

>".\node_modules\.bin\ng.cmd" build --prod
√ Browser application bundle generation complete.
√ Copying assets complete.
√ Index html generation complete.

Initial Chunk Files                   | Names         |     Size
main.a383d4343fc769770869.js          | main          |  3.20 MB
scripts.5b3322e07a3cdcf6af69.js       | scripts       |  1.48 MB
polyfills-es5.66ac62d089ac586e9220.js | polyfills-es5 | 85.88 kB
runtime.76277ef9655d34c2d937.js       | runtime       |  1.51 kB

                                      | Initial Total |  4.77 MB

Build at: 2021-05-16T12:08:41.073Z - Hash: 6acba3c73318a8cd5749 - Time: 159782ms

v12.0.0

>".\node_modules\.bin\ng.cmd" build --prod
Option "--prod" is deprecated: Use "--configuration production" instead.
Warning: Support was requested for IE 11 in the project's browserslist configuration. IE 11 support is deprecated since Angular v12.
For more information, see https://angular.io/guide/browser-support
√ Browser application bundle generation complete.
√ Copying assets complete.
√ Index html generation complete.

Initial Chunk Files                   | Names         |     Size
main.1bd8f29d9fb0ec7900b5.js          | main          |  3.22 MB
scripts.5b3322e07a3cdcf6af69.js       | scripts       |  1.48 MB
polyfills-es5.b787b17c5e8d81a4b18e.js | polyfills-es5 | 90.75 kB
runtime.bf3db8034d6e10a74a56.js       | runtime       |  1.20 kB

                                      | Initial Total |  4.79 MB

Build at: 2021-05-16T12:02:59.182Z - Hash: a899c8b10f7d6415a015 - Time: 163759ms

After several runs +/- the same time.

Styles processing is slightly slower on v12.

[JonWallsten]

For me it's even worse! Build time for our example repo (available for public, but stripped of most content) is up almost 500%.
Angular v11: webpack 5.37.0 compiled with 3 warnings in 12455 ms
Angular v12: webpack 5.37.0 compiled with 5 warnings in 49177 ms

I replaced AngularCompilerPlugin with AngularWebpackPlugin when switching to v12.

I have not yet succeeded to build our real project since I've given up after a few minutes. But based on the time above I will probably take about 8-10 minutes. Will try it out now.

Edit:
Yeah, so it seems like it's around 1000% slower!
Angular v11: webpack 5.37.0 compiled with 3 warnings in 45357 ms
Angular v12: webpack 5.37.0 compiled with 5 warnings in 437131 ms

[JoostK]

@JonWallsten Can you share that repo, as you mention it's public?

[JonWallsten]

@JoostK: Yeah, sorry, I forgot to make it public after I wrote that:
https://github.com/JonWallsten/monorepo-new
I'm building with profiling enabled in Webpack as we speak. Will paste them ASAP.

Just open an issue in the repo if something doesn't work. I just updated it from Webpack v4 to v5, so I lot had changed in out real repo that it's based on.

Profile:

assets by path assets/ 556 KiB
  assets by path assets/fonts/ 535 KiB
    assets by path assets/fonts/*.woff2 214 KiB 7 assets
    3 assets
  assets by path assets/icons/*.svg 1.56 KiB 4 assets
  assets by path assets/*.png 18.2 KiB 3 assets
  asset assets/images/favicon.882ede6c184b86e8dd7bb2f6bb4f2990.png 1.18 KiB [emitted] [immutable] [from: src/assets/favicon.png]
assets by path *.js 35.6 MiB
  asset lib.js 22.4 MiB [emitted] (name: lib) (id hint: lib)
  asset main.js 6.51 MiB [emitted] (name: main)
  asset angular.js 5.13 MiB [emitted] (name: angular) (id hint: angular)
  asset fontawesome.js 998 KiB [emitted] (name: fontawesome) (id hint: fortawesome)
  asset vendor.js 556 KiB [emitted] (name: vendor) (id hint: vendor)
asset appentries.json 1.07 KiB [emitted] [from: src/appentries.json] [copied]
asset index.html 871 bytes [emitted]
Entrypoint main 35.6 MiB = angular.js 5.13 MiB fontawesome.js 998 KiB lib.js 22.4 MiB vendor.js 556 KiB main.js 6.51 MiB
runtime modules 3.43 KiB 7 modules
orphan modules 44 bytes [orphan] 1 module
modules by path ../../node_modules/ 5.73 MiB 690 modules
modules by path ./src/ 2.58 MiB
  modules by path ./src/app/components/ 1.93 MiB 67 modules
  modules by path ./src/app/common/ 531 KiB 53 modules
  modules by path ./src/app/*.ts 139 KiB 4 modules
  ./src/main.ts 998 bytes [built] [code generated]
    391788 ms (resolving: 2365 ms, restoring: 0 ms, integration: 0 ms, building: 389423 ms, storing: 0 ms)
  ./src/environments/environment.ts 636 bytes [built] [code generated]
    391788 ms ->
    71 ms (resolving: 6 ms, restoring: 0 ms, integration: 0 ms, building: 65 ms, storing: 0 ms)
modules by path ../../polyfills/ 27.6 KiB
  ../../polyfills/es6-es7.ts 347 bytes [built] [code generated]
    391788 ms ->
    66 ms (resolving: 65 ms, restoring: 0 ms, integration: 0 ms, building: 1 ms, storing: 0 ms)
  ../../polyfills/keyboardEvent.js 27.2 KiB [built] [code generated]
    391788 ms ->
    87 ms (resolving: 66 ms, restoring: 0 ms, integration: 0 ms, building: 21 ms, storing: 0 ms)
../web-lib-angular/dist/fesm2015/oas-web-lib-angular.js 376 KiB [built] [code generated]
  391788 ms -> 90 ms ->
  3075 ms (resolving: 2566 ms, restoring: 2 ms, integration: 0 ms, building: 507 ms, storing: 0 ms, additional resolving: 6 ms)
../web-lib-core/dist/index.js 8.39 MiB [built] [code generated]
  391788 ms -> 90 ms -> 104 ms ->
  0 ms (resolving: 0 ms, restoring: 0 ms, integration: 0 ms, building: 0 ms, storing: 0 ms)
../web-lib-common/dist/index.js 454 KiB [built] [code generated]
  391788 ms -> 90 ms -> 3075 ms ->
  0 ms (resolving: 0 ms, restoring: 0 ms, integration: 0 ms, building: 0 ms, storing: 0 ms)

LOG from webpack.Compilation.ModuleProfile
<e>  | 2365 ms resolve to new modules > ./src/main.ts
<e> 8197 ms resolve to new modules
<e>  |  | 389423 ms build modules > ./src/main.ts
<e>  | 392028 ms build modules > 147 x javascript/auto with ../../node_modules/@ngtools/webpack/src/index.js
<e> 393148 ms build modules
+ 164 hidden lines

WARNING in ../../node_modules/@angular/core/__ivy_ngcc__/fesm2015/core.js 29825:15-36
Critical dependency: the request of a dependency is an expression
 @ ./src/main.ts 10:15-39

WARNING in ../../node_modules/@angular/core/__ivy_ngcc__/fesm2015/core.js 29825:15-36
System.import() is deprecated and will be removed soon. Use import() instead.
For more info visit https://webpack.js.org/guides/code-splitting/
 @ ./src/main.ts 10:15-39

WARNING in ../../node_modules/@angular/core/__ivy_ngcc__/fesm2015/core.js 29837:15-102
Critical dependency: the request of a dependency is an expression
 @ ./src/main.ts 10:15-39

WARNING in ../../node_modules/@angular/core/__ivy_ngcc__/fesm2015/core.js 29837:15-102
System.import() is deprecated and will be removed soon. Use import() instead.
For more info visit https://webpack.js.org/guides/code-splitting/
 @ ./src/main.ts 10:15-39

WARNING in C:/Users/me/repo/oas-web/packages/web-app-edit/src/environments/environment.prod.ts is part of the TypeScript compilation but it's unused.
Add only entry points to the 'files' or 'include' properties in your tsconfig.

2 warnings have detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.

webpack 5.37.0 compiled with 5 warnings in 436241 ms

[MadeleineCodes]

Unfortunately, we do not see much improvement with NG_PERSISTENT_BUILD_CACHE, I tried with and without but compared to 11.0.2 it's still much slower. I could live with production or even development build to be slower, but the much slower watch really hurts during development... :/

command	11.0.2	12.1.0	12.1.0 with NG_PERSISTENT_BUILD_CACHE 1st	12.1.0 with NG_PERSISTENT_BUILD_CACHE 2nd
develop build	~83s	~98s	~100s	~106s
production build	~180s	~270s	~240s	~225s
watch	~92s	140s + 48s*	106s + 43s*	-
recompile (main bundle**)	~13s	~22s	~21s	-

* initial watch always immediately triggers a recompile for whatever reason
** main bundle has about 6-7mb in watch mode

angular.json:

{
  "version": 1,
  "projects": {
    "app": {
      "projectType": "application",
      "schematics": {
        "@nrwl/angular:component": {
          "style": "less",
          "changeDetection": "OnPush"
        }
      },
      "root": "apps/app",
      "sourceRoot": "apps/app/src",
      "prefix": "app",
      "architect": {
        "build": {
          "builder": "@angular-builders/custom-webpack:browser",
          "options": {
            "customWebpackConfig": {
              "path": "apps/app/extra-webpack.config.js",
              "mergeStrategies": {
                "optimization.minimizer": "replace",
                "externals": "replace"
              }
            },
            "outputPath": "dist/apps/app",
            "index": "apps/app/src/index.html",
            "main": "apps/app/src/main.ts",
            "polyfills": "apps/app/src/polyfills.ts",
            "tsConfig": "apps/app/tsconfig.app.json",
            "preserveSymlinks": true,
            "aot": true,
            "allowedCommonJsDependencies": [
              "jstimezonedetect",
              "css-element-queries",
              "fastdom",
              "dragula",
              "contra/emitter",
              "crossvent",
              "dom-autoscroller",
              "dom-plane",
              "dom-set",
              "pdfjs-dist",
              "zone.js/dist/zone-error"
            ],
            "assets": [
              {
                "glob": "**/*",
                "input": "apps/app/src/assets/images",
                "output": "/images"
              },
              {
                "glob": "**/*",
                "input": "apps/app/src/assets/icons",
                "output": "/icons"
              },
              {
                "glob": "**/*",
                "input": "apps/app/src/assets/static",
                "output": "/static"
              },
              {
                "glob": "**/*",
                "input": "apps/app/src/assets/translations/messages",
                "output": "/translations"
              },
              {
                "glob": "properties.js",
                "input": "apps/app/src/assets/",
                "output": "/"
              },
              {
                "glob": "**/*",
                "input": "libs/annotations/src/assets/translations/messages",
                "output": "/translations/annotations"
              },
              {
                "glob": "**/*",
                "input": "node_modules/pdfjs-dist/cmaps",
                "output": "/worker-libs/pdf-cmaps/"
              },
              {
                "glob": "pdf.worker.min.js",
                "input": "node_modules/pdfjs-dist/build",
                "output": "/worker-libs/"
              },
              {
                "glob": "**/*",
                "input": "libs/common-components/src/assets/translations/messages",
                "output": "/translations/common"
              },
              {
                "glob": "**/*",
                "input": "libs/common-components/src/assets/images",
                "output": "/images"
              },
              {
                "glob": "**/*",
                "input": "libs/common-components/src/assets/icons",
                "output": "/icons"
              },
              {
                "glob": "**/*",
                "input": "libs/internal-components/src/assets/images",
                "output": "/images"
              }
            ],
            "scripts": [
              "node_modules/pdfjs-dist/build/pdf.min.js",
              "libs/annotations/src/assets/libs/fabric.js",
              "node_modules/systemjs/dist/s.js",
              "node_modules/systemjs/dist/extras/named-register.js",
              "node_modules/systemjs/dist/extras/amd.js"
            ],
            "stylePreprocessorOptions": {
              "includePaths": [
                "libs/common-components/src/assets/styles",
                "apps/app/src/assets/styles",
                "libs/app/shared/src/lib/assets/styles"
              ]
            },
            "styles": [
              "node_modules/quill/dist/quill.core.css",
              "node_modules/quill/dist/quill.bubble.css",
              "node_modules/quill/dist/quill.snow.css",
              "node_modules/pdfjs-dist/web/pdf_viewer.css",
              "node_modules/dragula/dist/dragula.min.css",
              "libs/common-components/src/assets/styles/material-theme-custom.less",
              "libs/common-components/src/assets/styles/app.less",
              "apps/app/src/assets/styles/styles.less",
              "apps/app/src/assets/styles/app-material.less"
            ]
          },
          "configurations": {
            "production": {
              "fileReplacements": [
                {
                  "replace": "apps/app/src/environments/environment.ts",
                  "with": "apps/app/src/environments/environment.prod.ts"
                }
              ],
              "optimization": {
                "scripts": false,
                "styles": true,
                "fonts": true
              },
              "outputHashing": "all",
              "sourceMap": false,
              "extractCss": true,
              "namedChunks": false,
              "extractLicenses": true,
              "vendorChunk": false,
              "buildOptimizer": true,
              "budgets": [
                {
                  "type": "initial",
                  "maximumWarning": "4mb"
                },
                {
                  "type": "anyComponentStyle",
                  "maximumWarning": "6kb",
                  "maximumError": "10kb"
                }
              ]
            },
            "development": {
              "buildOptimizer": false,
              "optimization": false,
              "vendorChunk": true,
              "extractLicenses": false,
              "sourceMap": true,
              "namedChunks": true
            }
          },
          "defaultConfiguration": "development"
        },
        "serve": {
          "builder": "@angular-devkit/build-angular:dev-server",
          "options": {
            "browserTarget": "app:build"
          },
          "configurations": {
            "production": {
              "browserTarget": "app:build:production"
            }
          }
        },
        "lint": {
          "builder": "@angular-devkit/build-angular:tslint",
          "options": {
            "tsConfig": [
              "apps/app/tsconfig.app.json",
              "apps/app/tsconfig.spec.json"
            ],
            "exclude": [
              "**/node_modules/**",
              "!apps/app/**/*"
            ]
          }
        },
        "test": {
          "builder": "@nrwl/jest:jest",
          "options": {
            "jestConfig": "apps/app/jest.config.js",
            "passWithNoTests": true,
            "silent": true
          }
        }
      }
    },
    "common-components": {
      "projectType": "library",
      "root": "libs/common-components",
      "sourceRoot": "libs/common-components/src",
      "prefix": "cc",
      "architect": {
        "build": {
          "builder": "@nrwl/angular:package",
          "options": {
            "tsConfig": "libs/common-components/tsconfig.lib.json",
            "project": "libs/common-components/ng-package.json"
          },
          "configurations": {
            "production": {
              "tsConfig": "libs/common-components/tsconfig.lib.prod.json"
            }
          }
        },
        "lint": {
          "builder": "@angular-devkit/build-angular:tslint",
          "options": {
            "tsConfig": [
              "libs/common-components/tsconfig.lib.json",
              "libs/common-components/tsconfig.spec.json"
            ],
            "exclude": [
              "**/node_modules/**",
              "!libs/common-components/**/*"
            ]
          }
        },
        "test": {
          "builder": "@nrwl/jest:jest",
          "options": {
            "jestConfig": "libs/common-components/jest.config.js",
            "passWithNoTests": true,
            "silent": true
          }
        },
        "storybook": {
          "builder": "@nrwl/storybook:storybook",
          "options": {
            "uiFramework": "@storybook/angular",
            "port": 4400,
            "config": {
              "configFolder": "libs/common-components/.storybook"
            }
          },
          "configurations": {
            "ci": {
              "quiet": true
            }
          }
        },
        "build-storybook": {
          "builder": "@nrwl/storybook:build",
          "options": {
            "uiFramework": "@storybook/angular",
            "outputPath": "dist/storybook/common-components",
            "config": {
              "configFolder": "libs/common-components/.storybook"
            }
          },
          "configurations": {
            "ci": {
              "quiet": true
            }
          }
        }
      },
      "schematics": {
        "@nrwl/angular:component": {
          "style": "less",
          "changeDetection": "OnPush"
        }
      }
    },
    "internal-components": {
      "projectType": "library",
      "root": "libs/internal-components",
      "sourceRoot": "libs/internal-components/src",
      "prefix": "ic",
      "architect": {
        "build": {
          "builder": "@nrwl/angular:package",
          "options": {
            "tsConfig": "libs/internal-components/tsconfig.lib.json",
            "project": "libs/internal-components/ng-package.json"
          },
          "configurations": {
            "production": {
              "tsConfig": "libs/internal-components/tsconfig.lib.prod.json"
            }
          }
        },
        "lint": {
          "builder": "@angular-devkit/build-angular:tslint",
          "options": {
            "tsConfig": [
              "libs/internal-components/tsconfig.lib.json",
              "libs/internal-components/tsconfig.spec.json",
              "libs/internal-components/.storybook/tsconfig.json"
            ],
            "exclude": [
              "**/node_modules/**",
              "!libs/internal-components/**/*"
            ]
          }
        },
        "test": {
          "builder": "@nrwl/jest:jest",
          "options": {
            "jestConfig": "libs/internal-components/jest.config.js",
            "passWithNoTests": true,
            "silent": true
          }
        },
        "storybook": {
          "builder": "@nrwl/storybook:storybook",
          "options": {
            "uiFramework": "@storybook/angular",
            "port": 4400,
            "config": {
              "configFolder": "libs/internal-components/.storybook"
            }
          },
          "configurations": {
            "ci": {
              "quiet": true
            }
          }
        },
        "build-storybook": {
          "builder": "@nrwl/storybook:build",
          "options": {
            "uiFramework": "@storybook/angular",
            "outputPath": "dist/storybook/internal-components",
            "config": {
              "configFolder": "libs/internal-components/.storybook"
            }
          },
          "configurations": {
            "ci": {
              "quiet": true
            }
          }
        }
      },
      "schematics": {
        "@schematics/angular:component": {
          "style": "less",
          "changeDetection": "OnPush"
        }
      }
    },
     "annotations": {
      "projectType": "library",
      "root": "libs/annotations",
      "sourceRoot": "libs/annotations/src",
      "prefix": "app",
      "architect": {
        "build": {
          "builder": "@nrwl/angular:package",
          "options": {
            "tsConfig": "libs/annotations/tsconfig.lib.json",
            "project": "libs/annotations/ng-package.json"
          },
          "configurations": {
            "production": {
              "tsConfig": "libs/annotations/tsconfig.lib.prod.json"
            }
          }
        },
        "lint": {
          "builder": "@angular-devkit/build-angular:tslint",
          "options": {
            "tsConfig": [
              "libs/annotations/tsconfig.lib.json",
              "libs/annotations/tsconfig.spec.json"
            ],
            "exclude": [
              "**/node_modules/**",
              "!libs/annotations/**/*"
            ]
          }
        },
        "test": {
          "builder": "@nrwl/jest:jest",
          "outputs": [
            "coverage/libs/annotations"
          ],
          "options": {
            "jestConfig": "libs/annotations/jest.config.js",
            "passWithNoTests": true,
            "silent": true
          }
        }
      }
    }
  },
  "cli": {
    "defaultCollection": "@nrwl/angular"
  },
  "schematics": {
    "@nrwl/angular:application": {
      "unitTestRunner": "jest",
      "e2eTestRunner": "cypress"
    },
    "@nrwl/angular:library": {
      "unitTestRunner": "jest"
    },
    "@nrwl/angular:component": {
      "style": "less"
    }
  }
}

System info:

OS Name:                   Microsoft Windows 10 Enterprise
OS Version:                10.0.19041 N/A Build 19041
OS Build Type:             Multiprocessor Free
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 142 Stepping 10 GenuineIntel ~1792 Mhz
BIOS Version:              HP Q78 Ver. 01.06.00, 03.01.2019
Total Physical Memory:     32 611 MB

I could send the cpu profiles if needed, but I am not allowed to share them (or the repo) publicly. Just let me know where I should send them :)

[alan-agius4]

@mrucelum, you are using builders which are not supported by the Angular tooling team. This causes a migration that adjusts the options to provide equivalent behavior from prior to the update not to run. Ideally these 3rd party builders should provide their own migration.

You need to adjust the options by hand to retain the existing behavior. For more information, see the breaking changes section within the release notes: https://github.com/angular/angular-cli/releases/tag/v12.0.0

[MadeleineCodes]

@alan-agius4 Thanks for link!
So if I understand you correctly, I have to adjust the configuration section of my angular.json manually, because I am using custom builders, right?

I checked the breaking changes and if I understand them correctly, with this addition to the angular.json:

          "configurations": {
           [...]
            "development": {
              "optimization": false,
              "buildOptimizer": false,
              "sourceMap": true,
              "extractLicenses": false,
              "namedChunks": true,
              "vendorChunk": true
            }
          },

I have the old default values (we had "aot": true before) already in place (when it comes to develop build and watch).
But this is the exact configuration I did my tests with. So I guess I am missing something in addition? Or does the custom builder need to adapt something (I mean something I cannot influence by just adapting my angular.json)?

Thanks again :)

[alan-agius4]

In the above table, you mentioned watch are you using ng build --watch?, Can you also try to replace the 3rd party builder with a supported builder @angular-devkit/build-angular:browser?

You can send the CPU profiles to alan.agius4[at]gmail.com.

[MadeleineCodes]

It seems to be indeed the @angular-builders/custom-webpack:browser that causes the slow down 😲

ng build --watch

1. with @angular-devkit/build-angular:browser

Initial Chunk Files                    | Names         |      Size
vendor.js                              | vendor        |   7.86 MB
main.js                                | main          |   6.66 MB
scripts.js                             | scripts       | 677.03 kB
polyfills.js                           | polyfills     | 213.39 kB
styles.css                             | styles        | 150.33 kB
runtime.js                             | runtime       |  14.33 kB

                                       | Initial Total |  15.55 MB
Build at: 2021-07-01T11:06:38.058Z - Hash: cd684dc7fcbc67560f0f - Time: 76208ms

2. with @angular-builders/custom-webpack:browser

Initial Chunk Files                    | Names         |      Size
vendor.js                              | vendor        |   6.67 MB
main.js                                | main          |   6.66 MB
scripts.js                             | scripts       | 677.03 kB
polyfills.js                           | polyfills     | 213.39 kB
styles.css                             | styles        | 150.33 kB
runtime.js                             | runtime       |  13.67 kB

                                       | Initial Total |  14.36 MB

Build at: 2021-07-01T11:43:51.105Z - Hash: 34bdac20b292943b28c9 - Time: 107267ms
√ Browser application bundle generation complete.
√ Index html generation complete.

15 unchanged chunks

Build at: 2021-07-01T11:44:36.147Z - Hash: e7860c484b03f51d78f9 - Time: 43856ms

3. with @angular-builders/custom-webpack:browser and without custom terser config in the referenced webpack config

Initial Chunk Files                    | Names         |      Size
vendor.js                              | vendor        |   6.67 MB
main.js                                | main          |   6.66 MB
scripts.js                             | scripts       | 677.03 kB
polyfills.js                           | polyfills     | 213.39 kB
styles.css                             | styles        | 150.33 kB
runtime.js                             | runtime       |  13.67 kB

                                       | Initial Total |  14.36 MB

Build at: 2021-07-01T11:12:49.818Z - Hash: 1ac2e18b2a871e0bf8a0 - Time: 103078ms
√ Browser application bundle generation complete.
√ Index html generation complete.

15 unchanged chunks

Build at: 2021-07-01T11:13:32.438Z - Hash: b8d53eea43833e670b2b - Time: 41681ms

4. with @angular-builders/custom-webpack:browser with empty webpack config

Initial Chunk Files                   | Names         |      Size
vendor.js                             | vendor        |   7.86 MB
main.js                               | main          |   6.66 MB
scripts.js                            | scripts       | 677.03 kB
polyfills.js                          | polyfills     | 213.39 kB
styles.css                            | styles        | 150.33 kB
runtime.js                            | runtime       |  13.67 kB

                                      | Initial Total |  15.55 MB

Build at: 2021-07-01T11:17:38.033Z - Hash: 223bbb0d807f62d6e34f - Time: 109983ms
√ Browser application bundle generation complete.
√ Index html generation complete.

15 unchanged chunks

Build at: 2021-07-01T11:18:26.881Z - Hash: b76731d2fe86fcc023e6 - Time: 47707ms

btw this is the custom webpack config:

const webpack = require('webpack');
const TerserPlugin = require('terser-webpack-plugin');

module.exports = {
  externals: {
    'fabric': 'window'
  },

  plugins: [new webpack.ContextReplacementPlugin(/moment[\/\\]locale$/, /en|de/)],
  optimization: {
    minimizer: [new TerserPlugin({
                                   terserOptions: {
                                     ecma: undefined,
                                     warnings: false,
                                     parse: {},
                                     compress: {},
                                     mangle: true, // Note `mangle.properties` is `false` by default.
                                     module: false,
                                     output: null,
                                     toplevel: false,
                                     nameCache: null,
                                     ie8: false,
                                     keep_classnames: true,
                                     keep_fnames: false,
                                     safari10: false
                                   },
                                 })]
  }
};

Conclusion:

1. with the default angular builder
   a. times are the same as with v11 (or even a bit better)
   b. there is no immediate recompile on watch 🥳
2. with the custom-webpack builder
   a. it takes much longer even if the custom webpack config does not contain anything
   b. bundle is quite a bit smaller because of moment locale exclusion

However... we need the custom webpack config as we do not want to have all the unnecessary moment locales in our bundle and also need to be able to specify 'fabric' as external for some legacy lib we are using.
So I guess I will head over to the @angular-builders/custom-webpack:browser project and file an issue there.
Thanks a lot @alan-agius4 !

[DavidBowdoin]

Success! I found steps to get good performance when using @angular-builders/custom-webpack

TLDR; before upgrading angular, remove all traces of custom-webpack so the migrations run in full, and then re-install custom-webpack

Here is what worked for me, starting with an Angular 11 project:

1. In angular.json, replaced all traces of @angular-builders/custom-webpack:browser with @angular-devkit/build-angular:browser (and delete the customWebpackConfig block)
2. Removed @angular-builders/custom-webpack from package.json
3. Deleted node_modules and package_lock.json
4. Ran npm install
5. Ran ng update @angular/cdk @angular/cli @angular/core
6. Added v12 of @angular-builders/custom-webpack back into package.json
7. Ran npm install
8. In angular.json, changed back to @angular-builders/custom-webpack:browser and restored the customWebpackConfig block.

I confirmed that the migrations in angular.json (that were getting skipped) make the difference (2x slower ng serve build time)

Benchmarking gives similar results as with angular 11 and markedly better results when I use NG_PERSISTENT_BUILD_CACHE=1

Like @mrucelum our project uses @angular-builders/custom-webpack to specify several packages as externals (legacy code needs the package as well, so don't want to double load them) Is there any chance ng update could do migrations when custom-webpack is in use? Or the ability to specify external packages without using custom-webpack? Thanks everyone for the info that lead me to this solution!

[simeyla]

@mrucelum Thanks for all the stats. I'm actually using @angular-builders/custom-webpack:browser too and for the exact same reason (moment.js size). However I just ran a dozen builds and with or without the custom builder it was taking 38-40 seconds each time. And it definitely did change because it told me to remove the part of the config that was due to the custom builder.

However... as I was writing this though I saw you managed to get them working.....

But, could you confirm that running the migrations did not in fact disable AOT compilation in your angular.json file. Depending upon your exact version of the CLI it's possible to end up disabling it when running migrations (due to a change in what was considered the default). If you in fact did disable AOT then that would explain the much faster compilation. Again though I'm using the same builder and wasn't seeing any noticeable difference between them. (Also version 12.1.0 of the custom builder was just published today - not sure if there were changes or just a version bump)

[terencehonles]

• NG_BUILD_CACHE is enabled by default, and is mainly used to opt-out all build caching, this supersedes NG_PERSISTENT_BUILD_CACHE.
• NG_PERSISTENT_BUILD_CACHE is disable by default, and can be used to opt-in to cache the entire build on disk, which will make the 2nd build warm. Typically you don't want this on CI since typically the "cache" will be discarded with node_modules after every CI run.

That being said, this issue is about build times and not memory, hence if you are experiencing memory issues please file a new issue providing all the necessary profiles and/or a reproduction.

Thanks @alan-agius4 for the explanation and sorry you'll probably end up marking this "off topic", but maybe not since it is about NG_PERSISTENT_BUILD_CACHE and I do care about any build speed improvements I can get since my build does seem slow to me.

I reviewed #20756 to confirm I understood and I wasn't taking a corner case that was unexpected. I use NG_BUILD_CACHE to point to a directory that is external to the node modules and in a location the CI will preserve, and this looks like it's still OK to do based on

angular-cli/packages/angular_devkit/build_angular/src/utils/environment-options.ts

Lines 70 to 81 in 736a5f8

	const cacheVariable = process.env['NG_BUILD_CACHE'];
	export const cachingDisabled = isPresent(cacheVariable) && isDisabled(cacheVariable);
	export const cachingBasePath = (() => {
	if (cachingDisabled || !isPresent(cacheVariable) || isEnabled(cacheVariable)) {
	return null;
	}
	if (!path.isAbsolute(cacheVariable)) {
	throw new Error('NG_BUILD_CACHE path value must be absolute.');
	}
	return cacheVariable;
	})();

does that mean I should also be able to turn on NG_PERSISTENT_BUILD_CACHE in order to persist the webpack cache? I noticed this was said to be experimental, but any reason I might not want to turn it on in this context? It looks like the cache key is pretty comprehensive and I would think the serialized build options should probably be good 🤔

Edit: After thinking about it more I definitely have places I can enable NG_PERSISTENT_BUILD_CACHE and hopefully it speeds up those paths, so thanks again regardless!

[terencehonles]

This actually looks to really help a lot for my build time 🎉 , but I've found a few errors and I'm reporting them (and fixing the one I can fix simply)

[whyboris]

After upgrading from Angular 11 to Angular 12 the build time went up from ~1 second to 22 seconds 🤯

whyboris/Video-Hub-App#648

[JoostK]

@whyboris It doesn't for me:

Is there a particular change I should be making?

[whyboris]

@JoostK thank you for testing! I was so frustrated I quit for the day, but thanks to your message I had new hope 😅

I ran rm -rf node_modules and npm install again and it looks like I'm back to fast 🚀

Build at: 2021-07-07T20:46:38.298Z - Hash: f2c33b8ef37564ec1ef0 - Time: 1713ms 🙇

Sorry for the scare 🙇

update: 🤦 turns out it still takes ~22 seconds when you just add console.log in some component

[JoostK]

@whyboris Thanks for getting back. Good to hear it's fast again, hopefully it stays that way. If you do run into it again please report back so we can have another look.

[whyboris]

@JoostK 🤦 sorry -- the reload was fast because I simply saved the file with no changes ... but when I add console.log somewhere - thereby changing the file, the compilation takes a long time again:

Build at: 2021-07-07T20:50:32.591Z - Hash: 0a430155980e29d73d08 - Time: 23324ms 😢

It seems to hang on the ... bundles (phase: sealing)... step

[JoostK]

@whyboris Oh, it's just a misconfiguration. In Angular 12 some builder defaults have changed in support of the new production builds by default for ng build. I am not sure how you updated but I don't see any changes to angular.json in the pull request; ng update should have updated it to pin the configuration options to their prior defaults to retain the previous behavior. If I run ng update @angular/cli --migrate-only --from 11 --to 12 then the angular.json file is indeed updated. Please review the changes as I notice in your case that aot will be set to false, which may or may not be desirable.

[whyboris]

@JoostK thank you very much for the suggestion!
I went back to my main branch (before my attempt to upgrade to angular 12) and ran the command you gave:
ng update @angular/cli --migrate-only --from 11 --to 12
It updated my angular.json file and now the app recompiles quickly after every save 🚀
https://github.com/whyboris/Video-Hub-App/pull/666/files
Thank you 🤝 🙇 ❤️

[the-ult]

@whyboris it seems like you set "aot": false which should be set to true (if I'm not misstaken)

[MadeleineCodes]

Just wanted to let you know that I did the migration again the way @DavidBowdoin suggested (remove custom builder before ng update).
Aaaand there is no slowdown anymore! 🥳

Not sure what the issue was, maybe it was actually fixed with the newest version of the custom webpack builder (as @simeyla suggested). My angular.json was not touched by the migration at all (although I removed the custom builder before the migration!), so I guess that was not the problem.

But anyway, I am happy that we can update now!
Thanks again everyone! :)

[vugar005]

Maybe this can solve this problem https://dev.to/brandontroberts/speeding-up-the-development-serve-after-upgrading-to-angular-v12-5db5

[alan-agius4]

It appears that the root cause of the problem is misconfigurations and not an issue in the Angular CLI.

If you are still experiencing slow production builds please open a new issue with a production.

[zeshan321]

Removing "emitDecoratorMetadata": true from tsconfig.json reduced our build time significantly for production.

[monsieur-ricky]

Success! I found steps to get good performance when using @angular-builders/custom-webpack

TLDR; before upgrading angular, remove all traces of custom-webpack so the migrations run in full, and then re-install custom-webpack

Here is what worked for me, starting with an Angular 11 project:

1. In angular.json, replaced all traces of @angular-builders/custom-webpack:browser with @angular-devkit/build-angular:browser (and delete the customWebpackConfig block)
2. Removed @angular-builders/custom-webpack from package.json
3. Deleted node_modules and package_lock.json
4. Ran npm install
5. Ran ng update @angular/cdk @angular/cli @angular/core
6. Added v12 of @angular-builders/custom-webpack back into package.json
7. Ran npm install
8. In angular.json, changed back to @angular-builders/custom-webpack:browser and restored the customWebpackConfig block.

I confirmed that the migrations in angular.json (that were getting skipped) make the difference (2x slower ng serve build time)

Benchmarking gives similar results as with angular 11 and markedly better results when I use NG_PERSISTENT_BUILD_CACHE=1

Like @mrucelum our project uses @angular-builders/custom-webpack to specify several packages as externals (legacy code needs the package as well, so don't want to double load them) Is there any chance ng update could do migrations when custom-webpack is in use? Or the ability to specify external packages without using custom-webpack? Thanks everyone for the info that lead me to this solution!

This solved my issue! Thanks :)

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}