[build][quality] more efficient CI, builds, improved linter config #272

…onfig This is a significant refactoring of the repo and CI. 1) Jenkins CI: more efficient, quicker builds I propose that we adopt a "run often, run quickly" approach to our Jenkins CI. ATM we mostly run Jenkins when it's time to release new Blueprint installers (when a PR is merged on master). The other case is if "jenkins" is detected in the PR branch or PR title, then almost a full Jenkins build is done, just stopping short of publishing new Blueprint installers (full signing and notarization is done, which can take a couple of hours). I made it so we run a very light version of the Jenkins CI by default, that builds the "dev" version of the app and pre-packages it using electron builder, generating a directory that contains the app but no installers. We really ought to run tests then, on the unpacked pre-packaged app, but it will need to be done separately. Until then, we do not require the built-in extensions (plugins), so I gave openvsx.org a rest and skip fetching them, by default. When it's detected that the Jenkins CI is processing the result of a merge on master or if the new "release dry-run" mode is enabled [1], a production version of the Blueprint apps will be produced, including built-in plugins, which will then be packaged for "real", producing genuine OS-specific installers, signed and notarized as required. This will take a lot longer to run, of course, but may be required only a minority of the time. Conclusion: with this PR, running Jenkins CI now takes about 12 minutes, and since this is relatively quick, we can afford to run this when any PR branch is created or updated. Running the same CI in "release dry-run mode" takes about an hour. A release will take longer, a bit over an hour, since additional steps are executed, like copying the various installers to a release folder. [1] About the "release dry-run" mode, it can be enabled in JenkinsFile, "pipeline" definition near the top, in "environment": BLUEPRINT_JENKINS_RELEASE_DRYRUN = 'true' I wish I found a better mechanism to enable this "release dry-run" mode. But this will serve for now. The "release dry-run" mode should be disabled before merging a PR to master, but can in the meantime be used to test or troubleshoot the whole pipeline, in particular signing and notarizing. 2) tsconfig/eslint configs: Added configs, to improve linter coverage. This made it possible for some source files, not previously covered, to get ts/linter feedback, both while editing and when running `yarn lint`. This will help keep code in-line with our standards. The config is not perfect and I would welcome further improvements. But for now I think it's a nice improvement. 3) Build "scripts" (package.json) Refactored the build commands ("scripts" section in package.json) - previously, merely running 'yarn" in the repo's root would rebuild every application from scratch. This prevents running a quick "yarn install", e.g. just to re-install build dependencies - the new version permits a granular build, with simple defaults - inspired from a similar change not so long ago in the main repo - see updated README for some examples Other misc items: - renamed extensions / applications - made the browser application a first class citizen, equal to the Electron application - all applications now share a common 'plugins' folder rather than each having their own. Moved the plugin-related entries to root package.json - to gain flexibility about which `yarn workspaces` are invoked for a given `lerna` command, using the `--scope=` CLI option.I renamed the repo's extensions and applications. This permits easily composing commands that target only the extensions or only the applications. e.g.: ``` json "build:extensions": "lerna run --scope=\"blueprint*ext\" build", "build:applications": "lerna run --scope=\"blueprint*app\" build --concurrency 1"," ``` - renamed the extensions folders, made them more straightforward - For systems with limited RAM, like on a Raspberry Pi 4B board with 4GB of RAM, it's now possible to successfully build Blueprint. e.g. use the following cmd: `yarn && yarn build:dev`, optionally followed by a packing command like: `yarn electron package` - (build:dev will build the Blueprint app in dev mode, which can be achieved in 4GB RAM) - [windows][jenkins] stash only dist folder: Currently, and only for Windows, we stash the whole git repo, which is very big and takes long to stash and un-stash. For the other OS, we only stash the dist folder, that contains the platform-specific installer, that we built. Let's try that for Windows too, and see if it works. - [jenkins] Decrease timeout from 5h to 3h" Looking at the build history, all recent builds that succeeded, did do under 2h30. OTOH, when a build hangs, it needs to wait for the timeout to expire, wasting time at the current value of 5h. Let's compromise at 3 hours and see how it goes? - [jenkins][installer build] exclude browser app for now: We do not yet publish the browser app, so let's skip building it to save time/resources. We may revisit when we use the new browser app bundling, recently made available upstream in the Theia framework. Signed-off-by: Marc Dumais <marc.dumais@ericsson.com>

#272 (review) Signed-off-by: Marc Dumais <marc.dumais@ericsson.com>

Oops - I had forgotten to build the Theia extensions, which caused a problem at runtime. Signed-off-by: Marc Dumais <marc.dumais@ericsson.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[build][quality] more efficient CI, builds, improved linter config #272

[build][quality] more efficient CI, builds, improved linter config #272

Commits on Aug 7, 2023