watch: add initial implementation for `node --watch` #44366

MoLow · 2022-08-23T18:03:48Z

Fixes: #40429

This is currently still WIP, opening this PR to gather some initial feedback

TODO

write documentation
add tests
explore use-cases and if they should be part of the initial implementation
- error handling / crashing
- debugging experience
- ~~making things configurable?~~
- please comment if you come up with any interesting use case

nodejs-github-bot · 2022-08-23T18:03:52Z

Review requested:

@nodejs/startup

lib/internal/main/watch_mode.js

doc/api/cli.md

lib/internal/main/watch_mode.js

lib/internal/modules/esm/loader.js

lib/internal/watch_mode/files_watcher.js

watch mode (experimental): Running in 'watch' mode using `node --watch` restarts the process when an imported file is changed. Contributed by Moshe Atlow in #44366 Other notable changes: * fs: * (SEMVER-MINOR) add `FileHandle.prototype.readLines` (Antoine du Hamel) #42590 * http: * (SEMVER-MINOR) add writeEarlyHints function to ServerResponse (Wing) #44180 * http2: * (SEMVER-MINOR) make early hints generic (Yagiz Nizipli) #44820 * lib: * (SEMVER-MINOR) refactor transferable AbortSignal (flakey5) #44048 * src: * (SEMVER-MINOR) add detailed embedder process initialization API (Anna Henningsen) #44121 * util: * (SEMVER-MINOR) add default value option to parsearg (Manuel Spigolon) #44631 PR-URL: #44968

Notable Changes: cli: * (SEMVER-MINOR) add `--watch` (Moshe Atlow) #44366 esm: * remove specifier resolution flag (Geoffrey Booth) #44859 * convert `resolve` hook to synchronous (Jacob Smith) #43363 http: * (SEMVER-MINOR) add writeEarlyHints function to ServerResponse (Wing) #44180 * (SEMVER-MAJOR) use Keep-Alive by default in global agents (Paolo Insogna) #43522 build: * (SEMVER-MAJOR) remove dtrace & etw support (Ben Noordhuis) #43652 * (SEMVER-MAJOR) remove systemtap support (Ben Noordhuis) #43651 Deprecation and Removals: * (SEMVER-MAJOR) runtime deprecate exports double slash maps (Guy Bedford) #44495 Semver-Major Commits: * [aa3a572] - (SEMVER-MAJOR) build: remove dtrace & etw support (Ben Noordhuis) #43652 * [38f1e27] - (SEMVER-MAJOR) build: remove systemtap support (Ben Noordhuis) #43651 * [2849283] - (SEMVER-MAJOR) crypto: remove non-standard `webcrypto.Crypto.prototype.CryptoKey` (Antoine du Hamel) #42083 * [a1653ac] - (SEMVER-MAJOR) crypto: do not allow to call setFips from the worker thread (Sergey Petushkov) #43624 * [a4fa526] - (SEMVER-MAJOR) fs: add directory autodetection to fsPromises.symlink() (Livia Medeiros) #42894 * [bb4891d] - (SEMVER-MAJOR) fs: add validateBuffer to improve error (Hirotaka Tagawa / wafuwafu13) #44769 * [950a441] - (SEMVER-MAJOR) fs: remove coercion to string in writing methods (Livia Medeiros) #42796 * [41a6d82] - (SEMVER-MAJOR) fs: harden fs.readSync(buffer, options) typecheck (LiviaMedeiros) #42772 * [2275faa] - (SEMVER-MAJOR) fs: harden fs.read(params, callback) typecheck (LiviaMedeiros) #42772 * [29953a0] - (SEMVER-MAJOR) fs: harden filehandle.read(params) typecheck (LiviaMedeiros) #42772 * [4267b92] - (SEMVER-MAJOR) http: use Keep-Alive by default in global agents (Paolo Insogna) #43522 * [f529f73] - (SEMVER-MAJOR) lib: brand check event handler property receivers (Chengzhong Wu) #44483 * [6de2673] - (SEMVER-MAJOR) lib: enable global WebCrypto by default (Antoine du Hamel) #42083 * [73ba883] - (SEMVER-MAJOR) lib: use private field in AbortController (Joyee Cheung) #43820 * [7dd2f41] - (SEMVER-MAJOR) module: runtime deprecate exports double slash maps (Guy Bedford) #44495 * [587367d] - (SEMVER-MAJOR) perf_hooks: expose webperf global scope interfaces (Chengzhong Wu) #44483 * [364c0e1] - (SEMVER-MAJOR) perf_hooks: fix webperf idlharness (Chengzhong Wu) #44483 * [e0ab8dd] - (SEMVER-MAJOR) process: make process.config read only (Sergey Petushkov) #43627 * [481a959] - (SEMVER-MAJOR) readline: remove `question` method from `InterfaceConstructor` (Antoine du Hamel) #44606 * [77e5856] - (SEMVER-MAJOR) src: turn embedder api overload into default argument (Alena Khineika) #43629 * [dabda03] - (SEMVER-MAJOR) src: per-environment time origin value (Chengzhong Wu) #43781 * [2b32985] - (SEMVER-MAJOR) stream: use null for the error argument (Luigi Pinca) #44312 * [57ff476] - (SEMVER-MAJOR) test: remove duplicate test (Luigi Pinca) #44051 * [77def91] - (SEMVER-MAJOR) tls,http2: send fatal alert on ALPN mismatch (Tobias Nießen) #44031 PR-URL: #44626 Co-authored-by: Ruy Adorno <ruyadorno@google.com>

Using `path.join(os.tmpdir(), 'file')` on a multi-user system can lead to file collisions where the file already exists but is owned by a different user and cannot be removed or overwritten. Other tests in `parallel/test-watch-mode-files_watcher` use `common/tmpdir` instead so switch to that for consistency. PR-URL: #45211 Refs: #44366 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>

PR-URL: #44366 Backport-PR-URL: #44976 Fixes: #40429 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

BobFrankston · 2022-12-04T02:20:42Z

Watch mode seems wonderful but I need a way to say I really want to exit without a restart. Is there an API or an option on Exit?

MoLow · 2022-12-04T06:56:44Z

Watch mode seems wonderful but I need a way to say I really want to exit without a restart. Is there an API or an option on Exit?

@BobFrankston can you kindly open an issue with an example describing your use case?

BobFrankston · 2022-12-04T13:50:37Z

Since the documentation is sparse, I conflated **nodemon'**s automatic restart with the restart on change. I realize --watch may not only do a restart on change but not assure the program continues to run after failure. Both are useful but distinct. If it only restarts on change, then it's worth thinking about a --persist or related option.

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [node](https://github.com/nodejs/node) | stage | minor | `16.18.1-bullseye` -> `16.19.0-bullseye` | --- ### Release Notes <details> <summary>nodejs/node</summary> ### [`v16.19.0`](https://github.com/nodejs/node/releases/tag/v16.19.0): 2022-12-13, Version 16.19.0 'Gallium' (LTS), @richardlau [Compare Source](nodejs/node@v16.18.1...v16.19.0) ##### Notable Changes ##### OpenSSL 1.1.1s This update is a bugfix release and does not address any security vulnerabilities. ##### Root certificates updated to NSS 3.85 Certificates added: - Autoridad de Certificacion Firmaprofesional CIF [`A626340`](nodejs/node@A62634068) - Certainly Root E1 - Certainly Root R1 - D-TRUST BR Root CA 1 2020 - D-TRUST EV Root CA 1 2020 - DigiCert TLS ECC P384 Root G5 - DigiCert TLS RSA4096 Root G5 - E-Tugra Global Root CA ECC v3 - E-Tugra Global Root CA RSA v3 - HiPKI Root CA - G1 - ISRG Root X2 - Security Communication ECC RootCA1 - Security Communication RootCA3 - Telia Root CA v2 - vTrus ECC Root CA - vTrus Root CA Certificates removed: - Cybertrust Global Root - DST Root CA X3 - GlobalSign Root CA - R2 - Hellenic Academic and Research Institutions RootCA 2011 ##### Time zone update to 2022f Time zone data has been updated to 2022f. This includes changes to Daylight Savings Time (DST) for Fiji and Mexico. For more information, see <https://mm.icann.org/pipermail/tz-announce/2022-October/000075.html>. ##### Other Notable Changes - \[[`33707dcd03`](nodejs/node@33707dcd03)] - **dgram**: add dgram send queue info (theanarkh) [#44149](nodejs/node#44149) Dependency updates: - \[[`3b2b70d792`](nodejs/node@3b2b70d792)] - **deps**: upgrade npm to 8.19.3 (npm team) [#45322](nodejs/node#45322) Experimental features: - \[[`1e0dcd1ee0`](nodejs/node@1e0dcd1ee0)] - **cli**: add `--watch` (Moshe Atlow) [#44366](nodejs/node#44366) - \[[`8c73279ebb`](nodejs/node@8c73279ebb)] - **util**: add default value option to parsearg (Manuel Spigolon) [#44631](nodejs/node#44631) ##### Commits - \[[`bbef3c42f6`](nodejs/node@bbef3c42f6)] - **build**: add version info to timezone update PR (Darshan Sen) [#45021](nodejs/node#45021) - \[[`cc2c7648e0`](nodejs/node@cc2c7648e0)] - **build**: support Python 3.11 (Luigi Pinca) [#45191](nodejs/node#45191) - \[[`ac24c80663`](nodejs/node@ac24c80663)] - **build**: remove redundant condition from common.gypi (Richard Lau) [#45076](nodejs/node#45076) - \[[`03dcbe3030`](nodejs/node@03dcbe3030)] - **build**: fix bad upstream merge (Stephen Gallagher) [#44642](nodejs/node#44642) - \[[`1e0dcd1ee0`](nodejs/node@1e0dcd1ee0)] - **cli**: add `--watch` (Moshe Atlow) [#44366](nodejs/node#44366) - \[[`96d131665e`](nodejs/node@96d131665e)] - **cluster**: use inspector utils (Moshe Atlow) [#44592](nodejs/node#44592) - \[[`704836033a`](nodejs/node@704836033a)] - **crypto**: update root certificates (Luigi Pinca) [#45490](nodejs/node#45490) - \[[`5a776d4a69`](nodejs/node@5a776d4a69)] - **deps**: update timezone to 2022f (Richard Lau) [#45613](nodejs/node#45613) - \[[`3b2b70d792`](nodejs/node@3b2b70d792)] - **deps**: upgrade npm to 8.19.3 (npm team) [#45322](nodejs/node#45322) - \[[`9fbc8b21db`](nodejs/node@9fbc8b21db)] - **deps**: update corepack to 0.15.1 (Node.js GitHub Bot) [#45331](nodejs/node#45331) - \[[`87e3d002ca`](nodejs/node@87e3d002ca)] - **deps**: update corepack to 0.15.0 (Node.js GitHub Bot) [#45235](nodejs/node#45235) - \[[`e972ff7b13`](nodejs/node@e972ff7b13)] - **deps**: V8: backport [`bbd800c`](nodejs/node@bbd800c6e359) (Chengzhong Wu) [#44947](nodejs/node#44947) - \[[`af9d8217c0`](nodejs/node@af9d8217c0)] - **deps**: V8: cherry-pick [`b953542`](nodejs/node@b95354290941) (Chengzhong Wu) [#44947](nodejs/node#44947) - \[[`38202d321b`](nodejs/node@38202d321b)] - **deps**: update undici to 5.12.0 (Node.js GitHub Bot) [#45236](nodejs/node#45236) - \[[`7c0da6adf9`](nodejs/node@7c0da6adf9)] - **deps**: update archs files for OpenSSL-1.1.1s (RafaelGSS) [#45274](nodejs/node#45274) - \[[`1149ead6f7`](nodejs/node@1149ead6f7)] - **deps**: upgrade openssl sources to OpenSSL\_1\_1\_1s (RafaelGSS) [#45274](nodejs/node#45274) - \[[`cd54bce4f5`](nodejs/node@cd54bce4f5)] - **deps**: update timezone (Node.js GitHub Bot) [#44950](nodejs/node#44950) - \[[`2901abe4f0`](nodejs/node@2901abe4f0)] - **deps**: update undici to 5.11.0 (Node.js GitHub Bot) [#44929](nodejs/node#44929) - \[[`c80cf97033`](nodejs/node@c80cf97033)] - **deps**: update corepack to 0.14.2 (Node.js GitHub Bot) [#44775](nodejs/node#44775) - \[[`33707dcd03`](nodejs/node@33707dcd03)] - **dgram**: add dgram send queue info (theanarkh) [#44149](nodejs/node#44149) - \[[`c708d9bb94`](nodejs/node@c708d9bb94)] - **doc**: fix typo in parseArgs default value (Tobias Nießen) [#45083](nodejs/node#45083) - \[[`5a0efa05d2`](nodejs/node@5a0efa05d2)] - **node-api**: handle no support for external buffers (Michael Dawson) [#45181](nodejs/node#45181) - \[[`db31de634e`](nodejs/node@db31de634e)] - **readline**: refactor to avoid unsafe regex primordials (Antoine du Hamel) [#43475](nodejs/node#43475) - \[[`fbc52e5729`](nodejs/node@fbc52e5729)] - **src**: disambiguate terms used to refer to builtins and addons (Joyee Cheung) [#44135](nodejs/node#44135) - \[[`953072d3db`](nodejs/node@953072d3db)] - **src**: let http2 streams end after session close (Santiago Gimeno) [#45153](nodejs/node#45153) - \[[`54608d8dc3`](nodejs/node@54608d8dc3)] - **src**: split property helpers from node::Environment (Chengzhong Wu) [#44056](nodejs/node#44056) - \[[`6733556783`](nodejs/node@6733556783)] - **test**: add test to validate changelogs for releases (Richard Lau) [#45325](nodejs/node#45325) - \[[`821d832cef`](nodejs/node@821d832cef)] - **test**: mark test-watch-mode\* as flaky on all platforms (Pierrick Bouvier) [#45049](nodejs/node#45049) - \[[`02a18eac69`](nodejs/node@02a18eac69)] - **test**: fix test-runner-inspect (Moshe Atlow) [#44620](nodejs/node#44620) - \[[`197df63f74`](nodejs/node@197df63f74)] - **test**: add a test to ensure the correctness of timezone upgrades (Darshan Sen) [#45299](nodejs/node#45299) - \[[`42e9d8016a`](nodejs/node@42e9d8016a)] - **test**: fix textdecoder test for small-icu builds (Richard Lau) [#45225](nodejs/node#45225) - \[[`6d736a56d8`](nodejs/node@6d736a56d8)] - **test**: fix watch mode test flake (Moshe Atlow) [#44739](nodejs/node#44739) - \[[`543d3d2bf3`](nodejs/node@543d3d2bf3)] - **test**: deflake watch mode tests (Moshe Atlow) [#44621](nodejs/node#44621) - \[[`97f6caf4eb`](nodejs/node@97f6caf4eb)] - **test**: split watch mode inspector tests to sequential (Moshe Atlow) [#44551](nodejs/node#44551) - \[[`499750ff7a`](nodejs/node@499750ff7a)] - **test**: update list of known globals (Antoine du Hamel) [#45255](nodejs/node#45255) - \[[`64d343af74`](nodejs/node@64d343af74)] - **test_runner**: support using `--inspect` with `--test` (Moshe Atlow) [#44520](nodejs/node#44520) - \[[`99ee5e484d`](nodejs/node@99ee5e484d)] - **test_runner**: fix `duration_ms` to be milliseconds (Moshe Atlow) [#44450](nodejs/node#44450) - \[[`37e909251c`](nodejs/node@37e909251c)] - **test_runner**: support programmatically running `--test` (Moshe Atlow) [#44241](nodejs/node#44241) - \[[`0ae5694f88`](nodejs/node@0ae5694f88)] - **tools**: update certdata.txt (Luigi Pinca) [#45490](nodejs/node#45490) - \[[`891368cefd`](nodejs/node@891368cefd)] - **tools**: remove faulty early termination logic from update-timezone.mjs (Darshan Sen) [#44870](nodejs/node#44870) - \[[`543493c242`](nodejs/node@543493c242)] - **tools**: fix timezone update tool (Darshan Sen) [#44870](nodejs/node#44870) - \[[`c77f660b75`](nodejs/node@c77f660b75)] - **tools**: fix `create-or-update-pull-request-action` hash on GHA (Antoine du Hamel) [#45166](nodejs/node#45166) - \[[`58c30dd049`](nodejs/node@58c30dd049)] - **tools**: update gr2m/create-or-update-pull-request-action (Luigi Pinca) [#45022](nodejs/node#45022) - \[[`749a4b3e5e`](nodejs/node@749a4b3e5e)] - **tools**: use Python 3.11 in GitHub Actions workflows (Luigi Pinca) [#45191](nodejs/node#45191) - \[[`6f541d99a5`](nodejs/node@6f541d99a5)] - **tools**: have test-asan use ubuntu-20.04 (Filip Skokan) [#45581](nodejs/node#45581) - \[[`e7ed56f501`](nodejs/node@e7ed56f501)] - **tools**: make license-builder.sh comply with shellcheck 0.8.0 (Rich Trott) [#41258](nodejs/node#41258) - \[[`cc819b4bf8`](nodejs/node@cc819b4bf8)] - **tools**: fix typo in `avoid-prototype-pollution` lint rule (Antoine du Hamel) [#44446](nodejs/node#44446) - \[[`254358c81e`](nodejs/node@254358c81e)] - **tools**: refactor `avoid-prototype-pollution` lint rule (Antoine du Hamel) [#43476](nodejs/node#43476) - \[[`8c73279ebb`](nodejs/node@8c73279ebb)] - **util**: add default value option to parsearg (Manuel Spigolon) [#44631](nodejs/node#44631) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.walbeck.it/mwalbeck/docker-cyberchef/pulls/143 Co-authored-by: renovate-bot <bot@walbeck.it> Co-committed-by: renovate-bot <bot@walbeck.it>

Using `path.join(os.tmpdir(), 'file')` on a multi-user system can lead to file collisions where the file already exists but is owned by a different user and cannot be removed or overwritten. Other tests in `parallel/test-watch-mode-files_watcher` use `common/tmpdir` instead so switch to that for consistency. PR-URL: #45211 Refs: #44366 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>

PR-URL: nodejs/node#44366 Backport-PR-URL: nodejs/node#44976 Fixes: nodejs/node#40429 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Using `path.join(os.tmpdir(), 'file')` on a multi-user system can lead to file collisions where the file already exists but is owned by a different user and cannot be removed or overwritten. Other tests in `parallel/test-watch-mode-files_watcher` use `common/tmpdir` instead so switch to that for consistency. PR-URL: #45211 Refs: #44366 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>

mule-stand · 2023-04-01T22:48:38Z

Maybe it would be good to add the ability to handle multiple entry points?

cinderblock · 2023-04-24T23:33:51Z

I am experiencing an issue where on some restarts, it seems watch mode restarts constantly for up to about 60 seconds - Eventually it recovers and runs normally.

I'm running on Windows 11 x64, Node v19.8.1 (not quite latest). Process Explorer confirmed that it was rapidly restarting and making 100s (or 1000s) of new processes that were dying quickly. My app does not modify local files. I believe my editor is also not repeatedly editing files.

I'm not even sure where/how to start debugging this... I should probably make a new issue.Uuntil I pull all the details into a real issue, this will have to suffice ;)

@mule-stand Running node ... is by definition one entry point. What are you imagining would change? In any case, this discussion would probably be better in a new Feature Request issue.

@BobFrankston You can add an infinite sleep at the end of your own program to keep node alive and watching for changes. This is what nodemon does internally. You need some process still running to be notified of any changes to the file system. You might even check process.execArgv.includes('--watch') and only run the infinite sleep if watch mode is enabled.

tniessen · 2023-04-25T10:56:52Z

@cinderblock Please open an issue. This PR is not the right place to discuss such problems in depth.

richterdennis · 2023-11-26T10:35:27Z

@cinderblock Did you open an issue for this restart thing? I am facing the same issue with Win 11 and Node 20. I try to reproduce this with a small code example, but it is kinda random and I don't know what causing the issue.

Edit:
After some research I found out that fs.watch is reporting change events when a file gets read. I opened an issue for this: #50920

claudioc · 2023-12-29T14:27:59Z

lib/internal/main/watch_mode.js

+}
+
+async function restart() {
+  process.stdout.write(`${clear}${green}Restarting ${kCommandStr}${white}\n`);


This "clear" should be at least configurable, as the --preserveWatchOutput option with tsc

Krinkle · 2024-07-30T20:34:03Z

doc/api/cli.md

+$ node --watch-path=./src --watch-path=./tests index.js
+```
+
+This option is only supported on macOS and Windows.


What part of this is disabled or partially functional on Linux? fs.watch() has caveats but certainly doesn't advertise as limited to Windows and macOS.

Is there an open issue for making this supported on Linux?

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Aug 23, 2022

MoLow changed the title ~~watch: add initial implementation~~ watch: add initial implementation for node --watch Aug 23, 2022

aduh95 reviewed Aug 23, 2022

View reviewed changes