-
-
Notifications
You must be signed in to change notification settings - Fork 156
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Since 2.3.0, webpack builds on CircleCI fail with "Error: Call retries were exceeded" #202
Comments
Actually I think my analysis was incorrect. It seems the issue also reproduces with version 2.2.3 (it just doesn't happen when debugging a failing build on CircleCI with SSH, only when running a "real" CircleCI build). Will close this issue as it is probably the same as #143 . |
Can you try https://github.com/webpack-contrib/terser-webpack-plugin/releases/tag/v2.3.3? If problem still exists i will reopen the problem, thanks |
@evilebottnawi I upgraded in my project to |
Can you provide minimum reproducible test repo and information about environment - real cores and memory, config file for CircleCI |
I cannot publish the code 😞The app that I'm talking about is quite large (+/- 4k files, 600+ react components, I don't count any external dependencies here). While building, we compile server and client bundle with Job's resource: circleci config.yml version: 2
jobs:
build_dependencies:
docker:
- image: circleci/node:10.16.2-stretch
working_directory: ~/app-frontend
steps:
- checkout
- run: sudo apt-get update && sudo apt-get install -y autoconf libtool pkg-config nasm
- restore_cache:
keys:
- npm-dependencies-{{ checksum "package.json" }}
# fallback to using the latest cache if no exact match is found
- npm-dependencies-
- run: yarn --silent --frozen-lockfile
- save_cache:
key: npm-dependencies-{{ checksum "package.json" }}
paths:
- node_modules
- save_cache:
key: repo-{{ .Environment.CIRCLE_SHA1 }}
paths:
- ~/app-frontend
build:
docker:
- image: circleci/node:10.16.2-stretch
working_directory: ~/app-frontend
steps:
- restore_cache:
key: repo-{{ .Environment.CIRCLE_SHA1 }}
- run: CONFIG=dev yarn build
workflows:
version: 2
build:
jobs:
- build_dependencies:
filters:
branches:
ignore:
- develop
- master
tags:
ignore: /^v.*/
- build:
requires:
- build_dependencies
filters:
branches:
ignore:
- develop
- master
tags:
ignore: /^v.*/ package.json {
"build": "npm-run-all --parallel build:*",
"build:client": "NODE_ENV=production webpack --config webpack/webpack.client.js --env.prod",
"build:server": "NODE_ENV=production webpack --config webpack/webpack.server.js --env.prod"
} webpack terser config const terserPlugin = new TerserJSPlugin({
terserOptions: {
parse: {
ecma: 8,
},
compress: {
ecma: 5,
warnings: false,
comparisons: false,
inline: 2,
drop_console: true,
},
mangle: {
safari10: true,
},
output: {
ecma: 5,
comments: false,
ascii_only: true,
},
},
parallel: isCircleCi() ? 2 : true,
cache: true,
sourceMap: true,
extractComments: true,
}); |
Thanks for information, what is result of |
circleci@<hash>:~$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 36
On-line CPU(s) list: 0-35
Thread(s) per core: 2
Core(s) per socket: 18
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
Stepping: 4
CPU MHz: 3399.925
BogoMIPS: 6000.00
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 25344K
NUMA node0 CPU(s): 0-35
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke > os.cpus() [
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3397,
times: {
user: 26940600,
nice: 141900,
sys: 6673300,
idle: 84844700,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3400,
times: {
user: 27778400,
nice: 153300,
sys: 7061100,
idle: 85539900,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3389,
times: {
user: 28683800,
nice: 128700,
sys: 7030100,
idle: 85136100,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3440,
times: {
user: 27663600,
nice: 121100,
sys: 6981800,
idle: 86086700,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3116,
times: {
user: 29731200,
nice: 133300,
sys: 7820900,
idle: 83304900,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3085,
times: {
user: 30912800,
nice: 134600,
sys: 8423700,
idle: 80364100,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3355,
times: {
user: 30386700,
nice: 129000,
sys: 7978900,
idle: 82519400,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3400,
times: {
user: 30117100,
nice: 140200,
sys: 8219200,
idle: 81339400,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3287,
times: {
user: 29964800,
nice: 122800,
sys: 7932900,
idle: 82947300,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3204,
times: {
user: 30102100,
nice: 134300,
sys: 8368700,
idle: 81286800,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3287,
times: {
user: 30583800,
nice: 139200,
sys: 8267000,
idle: 82011400,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3051,
times: {
user: 31621100,
nice: 118100,
sys: 8382300,
idle: 79922800,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3399,
times: {
user: 30451100,
nice: 133400,
sys: 7990200,
idle: 82395800,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3150,
times: {
user: 30496800,
nice: 152600,
sys: 7863100,
idle: 82457000,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3400,
times: {
user: 30808600,
nice: 124900,
sys: 8456000,
idle: 81011800,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3361,
times: {
user: 30607600,
nice: 137500,
sys: 7916100,
idle: 82274600,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3399,
times: {
user: 30067600,
nice: 140000,
sys: 8477200,
idle: 81668900,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3399,
times: {
user: 31656800,
nice: 118100,
sys: 8421500,
idle: 80762400,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3395,
times: {
user: 29729600,
nice: 122000,
sys: 9043100,
idle: 80276600,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3399,
times: {
user: 27666100,
nice: 125000,
sys: 8233300,
idle: 84423800,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3394,
times: {
user: 28174800,
nice: 121000,
sys: 8073200,
idle: 84135900,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3400,
times: {
user: 27988900,
nice: 125000,
sys: 8748100,
idle: 82338500,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3286,
times: {
user: 26778200,
nice: 133100,
sys: 7375700,
idle: 86398500,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3282,
times: {
user: 26562400,
nice: 124500,
sys: 7311000,
idle: 86705800,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3282,
times: {
user: 27483200,
nice: 124300,
sys: 7304700,
idle: 85339500,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3370,
times: {
user: 25486000,
nice: 123700,
sys: 7560800,
idle: 87532900,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3370,
times: {
user: 26600400,
nice: 127400,
sys: 7662600,
idle: 85278300,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3395,
times: {
user: 26358500,
nice: 131400,
sys: 7278300,
idle: 86987600,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3399,
times: {
user: 25715800,
nice: 131500,
sys: 7525700,
idle: 86858100,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3375,
times: {
user: 25976400,
nice: 142500,
sys: 6956500,
idle: 87683600,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3379,
times: {
user: 25610900,
nice: 137300,
sys: 7254800,
idle: 87717200,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3247,
times: {
user: 25417900,
nice: 136900,
sys: 7160800,
idle: 88048800,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3340,
times: {
user: 26041700,
nice: 121100,
sys: 6681500,
idle: 87907000,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3400,
times: {
user: 25104700,
nice: 138200,
sys: 6924000,
idle: 88608600,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3398,
times: {
user: 25153000,
nice: 130900,
sys: 6785800,
idle: 88700200,
irq: 0
}
},
{
model: 'Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz',
speed: 3400,
times: {
user: 24982900,
nice: 151500,
sys: 6926600,
idle: 88630400,
irq: 0
}
}
] |
Looks CircleCI doesn't provide real available count of CPUs, here the request for improvement https://ideas.circleci.com/ideas/CCI-I-264 |
Unfortunately, we have no way to get the real value of available 😞 Your solution seems to be the only one right at the moment |
👍 It'd be good to document that issue in the docs as a short explanation for Basically the trick with Thanks for confirming that it's the proper way to resolve that. |
@hinok Yes, will be great to documented that, I will do in near future, but you can be a champion 😄 |
This bug has been haunting us for some time in the https://github.com/automattic/wp-calypso project, so I did a deep dive trying to find the root cause. And I figured out that the CircleCI container where the build runs is running out of memory, so the Linux kernel will select a "bad" process and kill it with The There are other errors you can get that are caused by the same condition. For example, I've also seen this:
Here the attempt to restart the process fails: the kernel outright refuses to spawn a new process, because there' not enough memory for it. The |
@jsnajdr thanks for feedback, can you open an issue in jest repo? |
We are having exactly the same issue and cannot figure out. Thanks for clarifying what is going on there... |
I think that this is the best explanation for the problem. I am also getting this error in my current project. |
Expected Behavior
Building webpack JS bundles on CircleCI should be working the same for version 2.2.3 and versions >= 2.3.0.
Actual Behavior
Builds succeeds for terser-webpack-plugin v2.2.3 and are not working for terser-webpack-plugin >= 2.3.0 (tested with v2.3.0 and v2.3.1).
The error returned when running a webpack command such as:
webpack --mode production -p --progress --config webpack.config.js
with terser-webpack-plugin v2.3.0 produces this error:and this happens for multiple of the bundles that webpack tries to build.
Code
How Do We Reproduce?
The project I am building has more than 30 JS bundles and few of them are rather heavy (the heaviest is 1.6MB, average size is 600KB). Providing a reproducible example would be complicated to simulate the high number of JS files used and the overall heaviness of the bundles (maybe the issue is triggered in the source-map generation step).
But since the issue was introduced in a specific recent release (2.3.0), I hope that it will be easier to pinpoint the issue.
Please not that this issue is not reproducible on my local machine (a very recent MBP running Mac Mojave 10.14.6)
Thanks!
The text was updated successfully, but these errors were encountered: