Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(NA): creation of multiple processes on production by splitting no_transpilation when setting up node env #115246

Merged
Merged
Show file tree
Hide file tree
Changes from 56 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
3859c56
fix(NA): adds no_transpilation_dist to avoid preserve_symlinks on dist
mistic Oct 14, 2021
02696c4
Merge remote-tracking branch 'upstream/master' into fix-no-transpilat…
mistic Oct 14, 2021
5728f66
chore(NA): setup node env correctly on functional tests
mistic Oct 14, 2021
f81cf44
Merge remote-tracking branch 'upstream/master' into fix-no-transpilat…
mistic Oct 14, 2021
e447bae
chore(NA): try to fix tests
mistic Oct 14, 2021
ae9860a
chore(NA): correctly separate split
mistic Oct 14, 2021
3851301
Merge branch 'master' into fix-no-transpilation-by-adding-dist-version
kibanamachine Oct 14, 2021
0d533cf
chore(NA): check ensure preserve symlinks need
mistic Oct 14, 2021
7b9d5fa
Merge branch 'fix-no-transpilation-by-adding-dist-version' of github.…
mistic Oct 14, 2021
eea42f9
chore(NA): investigate path resolve result
mistic Oct 14, 2021
988f5ba
chore(NA): investigate path resolve result #2
mistic Oct 14, 2021
3e4ba6d
chore(NA): comment out preserve symlinks
mistic Oct 14, 2021
9f6aa9f
chore(NA): apply fs.realpathSync into the calculated REPO_ROOT paths …
mistic Oct 15, 2021
5a37205
chore(NA): removes debug code
mistic Oct 15, 2021
783bdee
chore(NA): move array definition
mistic Oct 15, 2021
02be1a3
chore(NA): correctly import fs
mistic Oct 15, 2021
0053253
Merge branch 'master' into fix-no-transpilation-by-adding-dist-version
kibanamachine Oct 15, 2021
8c3a250
Merge remote-tracking branch 'upstream/master' into fix-no-transpilat…
mistic Oct 15, 2021
8c14f98
Merge branch 'fix-no-transpilation-by-adding-dist-version' of github.…
mistic Oct 15, 2021
34dd32c
Merge branch 'master' into fix-no-transpilation-by-adding-dist-version
kibanamachine Oct 15, 2021
35ed596
chore(NA): add debug code
mistic Oct 15, 2021
77440a0
chore(NA): some more debug statements
mistic Oct 15, 2021
b1fd263
chore(NA): remove ensure symlinks
mistic Oct 15, 2021
6120e78
chore(NA): trying to solve double symlinking
mistic Oct 15, 2021
7d1a329
chore(NA): test mappings
mistic Oct 15, 2021
2f5308b
chore(NA): process path
mistic Oct 15, 2021
d4fe028
chore(NA): test a second map
mistic Oct 15, 2021
ba4a150
chore(NA): using a different mappings
mistic Oct 16, 2021
ebb456f
chore(NA): more debug cases
mistic Oct 16, 2021
04170a1
chore(NA): more debug logic
mistic Oct 16, 2021
247412b
chore(NA): more debug cases
mistic Oct 16, 2021
1a19a23
chore(NA): more debug cases
mistic Oct 16, 2021
a1e94e5
chore(NA): more debug cases
mistic Oct 16, 2021
3747912
chore(NA): try to add realpathSync into require
mistic Oct 16, 2021
5be4490
chore(NA): try to add realpathSync into require
mistic Oct 16, 2021
539edea
fix(NA): jenkins and buildkite run
mistic Oct 17, 2021
3275ef0
chore(NA): add debug logs
mistic Oct 17, 2021
2e9cdb9
chore(NA): correct path
mistic Oct 17, 2021
ab46de8
chore(NA): correct path
mistic Oct 18, 2021
957165d
chore(NA): add more test maps
mistic Oct 18, 2021
f96576c
chore(NA): add more test maps
mistic Oct 18, 2021
45fab4f
chore(NA): add some more test maps experiments
mistic Oct 18, 2021
790576e
chore(NA): try to remove another test map dep
mistic Oct 18, 2021
3311ede
chore(NA): try to remove another test map dep
mistic Oct 18, 2021
375d70a
chore(NA): try to remove another test map dep
mistic Oct 18, 2021
61eed1b
chore(NA): try to remove another test map dep
mistic Oct 18, 2021
07373f4
chore(NA): try to remove another test map dep
mistic Oct 19, 2021
109cd42
Merge remote-tracking branch 'upstream/master' into fix-no-transpilat…
mistic Oct 19, 2021
e92fe2d
chore(NA): try to remove another test map dep
mistic Oct 19, 2021
29b7d29
chore(NA): try to remove another test map dep
mistic Oct 20, 2021
46ec91b
chore(NA): try to remove another test map dep
mistic Oct 20, 2021
abd391c
chore(NA): include all correct transpilations for each jenkins path
mistic Oct 20, 2021
05c2ed5
chore(NA): include all correct transpilations for each used asset path
mistic Oct 20, 2021
8d97a6a
chore(NA): include all correct transpilations for each used asset path
mistic Oct 20, 2021
7a615b8
Merge remote-tracking branch 'upstream/master' into fix-no-transpilat…
mistic Oct 20, 2021
aa58429
Merge branch 'master' into fix-no-transpilation-by-adding-dist-version
kibanamachine Oct 20, 2021
44ece12
Merge branch 'master' into fix-no-transpilation-by-adding-dist-version
mistic Oct 20, 2021
cde712c
chore(NA): remove jenkins support
mistic Oct 20, 2021
033dbf3
Merge branch 'fix-no-transpilation-by-adding-dist-version' of github.…
mistic Oct 20, 2021
7dec8b3
Merge branch 'master' into fix-no-transpilation-by-adding-dist-version
kibanamachine Oct 21, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,50 @@
* Side Public License, v 1.
*/

const Fs = require('fs');
const Path = require('path');

const { REPO_ROOT } = require('@kbn/dev-utils');
const { REPO_ROOT: REPO_ROOT_FOLLOWING_SYMLINKS } = require('@kbn/dev-utils');
const BASE_REPO_ROOT = Path.resolve(
Fs.realpathSync(Path.resolve(REPO_ROOT_FOLLOWING_SYMLINKS, 'package.json')),
'..'
);

// Jenkins use a special symlink setup between a main checkout at
mistic marked this conversation as resolved.
Show resolved Hide resolved
// /dev/shm/workspace/kibana and a workers on at /dev/shm/workspace/parallel/X/kibana
// We have jobs running at both and as such we need to transpile for both paths.
// Once we no longer run on Jenkins our REPO_ROOT can just be calculated as
// Fs.realpathSync($PATH) where $PATH is any Path.resolve(REPO_ROOT, TRANSPILE_KBN_PATH)
const REPO_ROOT =
process.env.JENKINS_HOME && !BASE_REPO_ROOT.includes('parallel')
? Path.join(
Path.dirname(BASE_REPO_ROOT),
'parallel',
process.env.CI_PARALLEL_PROCESS_NUMBER,
Path.basename(BASE_REPO_ROOT)
)
: BASE_REPO_ROOT;

const transpileKbnBasePaths = [
'test',
'x-pack/test',
'examples',
'x-pack/examples',
// TODO: should should probably remove this link back to the source
'x-pack/plugins/task_manager/server/config.ts',
'src/core/utils/default_app_categories.ts',
];

const transpileKbnPaths = transpileKbnBasePaths.reduce(
(prev, curr) => prev.concat([Path.resolve(REPO_ROOT, curr), Path.resolve(BASE_REPO_ROOT, curr)]),
[]
);

// modifies all future calls to require() to automatically
// compile the required source with babel
require('@babel/register')({
ignore: [/[\/\\](node_modules|target|dist)[\/\\]/],
only: [
Path.resolve(REPO_ROOT, 'test'),
Path.resolve(REPO_ROOT, 'x-pack/test'),
Path.resolve(REPO_ROOT, 'examples'),
Path.resolve(REPO_ROOT, 'x-pack/examples'),
// TODO: should should probably remove this link back to the source
Path.resolve(REPO_ROOT, 'x-pack/plugins/task_manager/server/config.ts'),
Path.resolve(REPO_ROOT, 'src/core/utils/default_app_categories.ts'),
],
only: transpileKbnPaths,
babelrc: false,
presets: [require.resolve('@kbn/babel-preset/node_preset')],
extensions: ['.js', '.ts', '.tsx'],
Expand Down
2 changes: 1 addition & 1 deletion src/setup_node_env/dist.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@
* Side Public License, v 1.
*/

require('./no_transpilation');
require('./no_transpilation_dist');
require('./polyfill');
10 changes: 1 addition & 9 deletions src/setup_node_env/no_transpilation.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,4 @@
*/

require('./ensure_node_preserve_symlinks');

// The following require statements MUST be executed before any others - BEGIN
require('./exit_on_warning');
require('./harden');
// The following require statements MUST be executed before any others - END

require('symbol-observable');
require('source-map-support/register');
require('./node_version_validator');
require('./no_transpilation_dist');
16 changes: 16 additions & 0 deletions src/setup_node_env/no_transpilation_dist.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

// The following require statements MUST be executed before any others - BEGIN
require('./exit_on_warning');
require('./harden');
// The following require statements MUST be executed before any others - END

require('symbol-observable');
require('source-map-support/register');
require('./node_version_validator');