Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert 5541300 and add coverage to pipeline #56278

Closed
wants to merge 2 commits into from

Conversation

jakecastelli
Copy link
Member

@jakecastelli jakecastelli commented Dec 17, 2024

#55096 introduced a regression which causes the callback in the pipeline not called when error is thrown in read from readable. Please note this is on the current main not released yet.

I think this would cause a massive breakage as I found this out through some user land applications / packages by using the nightly build. My suggestion is:

  1. revert 5541300 (as it currently causes regression on nightly build)
  2. adding a test coverage / some test coverages
  3. let the origin author to reopen a PR and we run a CITGM to see if there are more potential breakages

Take this code snippet as repro:

const { Readable, pipeline, Duplex, Writable } = require("node:stream");

async function* myAsyncGenerator(ag) {
  for await (const data of ag) {
    yield data;
  }
}

const duplexStream = Duplex.from(myAsyncGenerator);

const r = new Readable({
  read() {
    this.push("data1\n");
    throw new Error("booom");
  },
});

const w = new Writable({
  write(chunk, encoding, callback) {
    callback();
  },
});

pipeline(r, duplexStream, w, (err) => {
  console.log("Error in pipeline:", err); // you won't be able to see this line as the cb never get called
});

This PR reverted 5541300 and added a new test coverage.

cc @matthieusieben

jakecastelli and others added 2 commits December 17, 2024 15:12
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem. labels Dec 17, 2024
@jakecastelli jakecastelli added commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. stream Issues and PRs related to the stream subsystem. needs-ci PRs that need a full CI run. and removed stream Issues and PRs related to the stream subsystem. needs-ci PRs that need a full CI run. labels Dec 17, 2024
Copy link

codecov bot commented Dec 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.54%. Comparing base (80e3ef3) to head (175b3aa).
Report is 95 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #56278      +/-   ##
==========================================
- Coverage   88.55%   88.54%   -0.01%     
==========================================
  Files         657      657              
  Lines      190265   190220      -45     
  Branches    36541    36534       -7     
==========================================
- Hits       168483   168430      -53     
- Misses      14970    14971       +1     
- Partials     6812     6819       +7     
Files with missing lines Coverage Δ
lib/internal/streams/duplexify.js 96.56% <100.00%> (-0.60%) ⬇️

... and 24 files with indirect coverage changes

@jakecastelli jakecastelli added the request-ci Add this label to start a Jenkins CI on a PR. label Dec 17, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Dec 17, 2024
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@matthieusieben
Copy link
Contributor

matthieusieben commented Dec 17, 2024

Isn't this due to the fact that autoDestroy is not set in the readable options ?

apparently not.

@matthieusieben
Copy link
Contributor

matthieusieben commented Dec 17, 2024

I adressed this issue in #56287. Streams are hard 😅

@jakecastelli
Copy link
Member Author

Can we get this landed first?

First this is breaking on nightly.
Second, @matthieusieben can take time to work on opening a new PR with the origin change and his fix in #56287 in a single commit and we run CITGM. What do you folks think?

This PR is ready with a green CI. cc @nodejs/streams

@matthieusieben
Copy link
Contributor

I agree that this is probably the best course of action

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Member

@matthieusieben @nodejs/streams may i get another LGTM so we can land?

@jakecastelli jakecastelli added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Dec 23, 2024
@mcollina mcollina added the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 24, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 24, 2024
@nodejs-github-bot
Copy link
Collaborator

Landed in a9e65f6...5ea6fa7

nodejs-github-bot pushed a commit that referenced this pull request Dec 24, 2024
This reverts commit 5541300.

PR-URL: #56278
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
nodejs-github-bot pushed a commit that referenced this pull request Dec 24, 2024
co-authored-by: jazelly <xzha4350@gmail.com>
PR-URL: #56278
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
@aduh95 aduh95 added the dont-land-on-v18.x PRs that should not land on the v18.x-staging branch and should not be released in v18.x. label Jan 2, 2025
@aduh95 aduh95 added dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. dont-land-on-v22.x PRs that should not land on the v22.x-staging branch and should not be released in v22.x. dont-land-on-v23.x PRs that should not land on the v23.x-staging branch and should not be released in v23.x. labels Jan 2, 2025
aduh95 pushed a commit that referenced this pull request Jan 2, 2025
co-authored-by: jazelly <xzha4350@gmail.com>
PR-URL: #56278
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
@aduh95 aduh95 added backported-to-v23.x PRs backported to the v23.x-staging branch. and removed dont-land-on-v23.x PRs that should not land on the v23.x-staging branch and should not be released in v23.x. labels Jan 6, 2025
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Jan 8, 2025
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [node](https://nodejs.org) ([source](https://github.com/nodejs/node)) | minor | `23.5.0` -> `23.6.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>nodejs/node (node)</summary>

### [`v23.6.0`](https://github.com/nodejs/node/releases/tag/v23.6.0): 2025-01-07, Version 23.6.0 (Current), @&#8203;marco-ippolito

[Compare Source](nodejs/node@v23.5.0...v23.6.0)

##### Notable Changes

##### Unflagging --experimental-strip-types

This release enables the flag `--experimental-strip-types` by default.
Node.js will be able to execute TypeScript files without additional configuration:

```bash
node file.ts
```

There are some limitations in the supported syntax documented at <https://nodejs.org/api/typescript.html#type-stripping>
This feature is experimental and is subject to change.

Contributed by Marco Ippolito in [#&#8203;56350](nodejs/node#56350)

##### Other Notable Changes

-   \[[`c1023284c3`](nodejs/node@c1023284c3)] - **(SEMVER-MINOR)** **lib**: add typescript support to STDIN eval (Marco Ippolito) [#&#8203;56359](nodejs/node#56359)
-   \[[`8dc39e5e2e`](nodejs/node@8dc39e5e2e)] - **(SEMVER-MINOR)** **process**: add process.ref() and process.unref() methods (James M Snell) [#&#8203;56400](nodejs/node#56400)
-   \[[`8b20cc212b`](nodejs/node@8b20cc212b)] - **(SEMVER-MINOR)** **worker**: add eval ts input (Marco Ippolito) [#&#8203;56394](nodejs/node#56394)

##### Commits

-   \[[`7b4d288116`](nodejs/node@7b4d288116)] - **assert**: make partialDeepStrictEqual throw when comparing \[0] with \[-0] (Giovanni) [#&#8203;56237](nodejs/node#56237)
-   \[[`0ec2ed0a0b`](nodejs/node@0ec2ed0a0b)] - **build**: fix GN build for ngtcp2 (Cheng) [#&#8203;56300](nodejs/node#56300)
-   \[[`ab3e64630b`](nodejs/node@ab3e64630b)] - **build**: test macos-13 on GitHub actions (Michaël Zasso) [#&#8203;56307](nodejs/node#56307)
-   \[[`46fb69daca`](nodejs/node@46fb69daca)] - **build**: build v8 with -fvisibility=hidden on macOS (Joyee Cheung) [#&#8203;56275](nodejs/node#56275)
-   \[[`9d4930b993`](nodejs/node@9d4930b993)] - **deps**: update simdutf to 5.7.2 (Node.js GitHub Bot) [#&#8203;56388](nodejs/node#56388)
-   \[[`6afe36397e`](nodejs/node@6afe36397e)] - **deps**: update amaro to 0.2.1 (Node.js GitHub Bot) [#&#8203;56390](nodejs/node#56390)
-   \[[`195990a0ee`](nodejs/node@195990a0ee)] - **deps**: update googletest to [`7d76a23`](nodejs/node@7d76a23) (Node.js GitHub Bot) [#&#8203;56387](nodejs/node#56387)
-   \[[`b9c0852fc6`](nodejs/node@b9c0852fc6)] - **deps**: update googletest to [`e54519b`](nodejs/node@e54519b) (Node.js GitHub Bot) [#&#8203;56370](nodejs/node#56370)
-   \[[`eaefd90128`](nodejs/node@eaefd90128)] - **deps**: update ngtcp2 to 1.10.0 (Node.js GitHub Bot) [#&#8203;56334](nodejs/node#56334)
-   \[[`06de0c65cf`](nodejs/node@06de0c65cf)] - **deps**: update simdutf to 5.7.0 (Node.js GitHub Bot) [#&#8203;56332](nodejs/node#56332)
-   \[[`03df76cdec`](nodejs/node@03df76cdec)] - **doc**: add example for piping ReadableStream (Gabriel Schulhof) [#&#8203;56415](nodejs/node#56415)
-   \[[`38ce249b07`](nodejs/node@38ce249b07)] - **doc**: expand description of `parseArg`'s `default` (Kevin Gibbons) [#&#8203;54431](nodejs/node#54431)
-   \[[`ecc718cef2`](nodejs/node@ecc718cef2)] - **doc**: use `<ul>` instead of `<ol>` in `SECURITY.md` (Antoine du Hamel) [#&#8203;56346](nodejs/node#56346)
-   \[[`3db4809130`](nodejs/node@3db4809130)] - **doc**: clarify that WASM is trusted (Matteo Collina) [#&#8203;56345](nodejs/node#56345)
-   \[[`384ccbacd5`](nodejs/node@384ccbacd5)] - **doc**: update macOS and Xcode versions for releases (Michaël Zasso) [#&#8203;56337](nodejs/node#56337)
-   \[[`3943986e88`](nodejs/node@3943986e88)] - **doc**: fix the `crc32` documentation (Kevin Toshihiro Uehara) [#&#8203;55898](nodejs/node#55898)
-   \[[`710b8fc6ed`](nodejs/node@710b8fc6ed)] - **doc**: add entry to changelog about SQLite Session Extension (Bart Louwers) [#&#8203;56318](nodejs/node#56318)
-   \[[`4c978b4d77`](nodejs/node@4c978b4d77)] - **doc**: fix links in `module.md` (Antoine du Hamel) [#&#8203;56283](nodejs/node#56283)
-   \[[`cdb631efe7`](nodejs/node@cdb631efe7)] - **esm**: add experimental support for addon modules (Chengzhong Wu) [#&#8203;55844](nodejs/node#55844)
-   \[[`db83d2f0ee`](nodejs/node@db83d2f0ee)] - ***Revert*** "**events**: add hasEventListener util for validate" (origranot) [#&#8203;56282](nodejs/node#56282)
-   \[[`c2baae84ce`](nodejs/node@c2baae84ce)] - **lib**: refactor execution.js (Marco Ippolito) [#&#8203;56358](nodejs/node#56358)
-   \[[`c1023284c3`](nodejs/node@c1023284c3)] - **(SEMVER-MINOR)** **lib**: add typescript support to STDIN eval (Marco Ippolito) [#&#8203;56359](nodejs/node#56359)
-   \[[`e4b795ec4a`](nodejs/node@e4b795ec4a)] - **lib**: optimize `prepareStackTrace` on builtin frames (Chengzhong Wu) [#&#8203;56299](nodejs/node#56299)
-   \[[`d1b009b623`](nodejs/node@d1b009b623)] - **lib**: suppress source map lookup exceptions (Chengzhong Wu) [#&#8203;56299](nodejs/node#56299)
-   \[[`c2837f0805`](nodejs/node@c2837f0805)] - **meta**: move one or more collaborators to emeritus (Node.js GitHub Bot) [#&#8203;56342](nodejs/node#56342)
-   \[[`72336233f2`](nodejs/node@72336233f2)] - **meta**: move MoLow to TSC regular member (Moshe Atlow) [#&#8203;56276](nodejs/node#56276)
-   \[[`4f77920a9d`](nodejs/node@4f77920a9d)] - **module**: fix async resolution error within the sync `findPackageJSON` (Jacob Smith) [#&#8203;56382](nodejs/node#56382)
-   \[[`e5ba216501`](nodejs/node@e5ba216501)] - **(SEMVER-MINOR)** **module**: unflag --experimental-strip-types (Marco Ippolito) [#&#8203;56350](nodejs/node#56350)
-   \[[`959f133a22`](nodejs/node@959f133a22)] - **module**: support eval with ts syntax detection (Marco Ippolito) [#&#8203;56285](nodejs/node#56285)
-   \[[`717cfa4fac`](nodejs/node@717cfa4fac)] - **module**: use buffer.toString base64 (Chengzhong Wu) [#&#8203;56315](nodejs/node#56315)
-   \[[`c2f4d8d688`](nodejs/node@c2f4d8d688)] - **node-api**: define version 10 (Gabriel Schulhof) [#&#8203;55676](nodejs/node#55676)
-   \[[`417a8ebdec`](nodejs/node@417a8ebdec)] - **node-api**: remove deprecated attribute from napi_module_register (Vladimir Morozov) [#&#8203;56162](nodejs/node#56162)
-   \[[`8dc39e5e2e`](nodejs/node@8dc39e5e2e)] - **(SEMVER-MINOR)** **process**: add process.ref() and process.unref() methods (James M Snell) [#&#8203;56400](nodejs/node#56400)
-   \[[`d194f1ab5f`](nodejs/node@d194f1ab5f)] - **sqlite**: pass conflict type to conflict resolution handler (Bart Louwers) [#&#8203;56352](nodejs/node#56352)
-   \[[`29f5d70452`](nodejs/node@29f5d70452)] - **src**: use v8::LocalVector consistently with other minor cleanups (James M Snell) [#&#8203;56417](nodejs/node#56417)
-   \[[`2a5543b78e`](nodejs/node@2a5543b78e)] - **src**: use starts_with in fs_permission.cc (ishabi) [#&#8203;55811](nodejs/node#55811)
-   \[[`3a3f5c9a64`](nodejs/node@3a3f5c9a64)] - **stream**: validate undefined sizeAlgorithm in WritableStream (Jason Zhang) [#&#8203;56067](nodejs/node#56067)
-   \[[`6e6f6b071a`](nodejs/node@6e6f6b071a)] - **test**: add ts eval snapshots (Marco Ippolito) [#&#8203;56358](nodejs/node#56358)
-   \[[`8a87e39052`](nodejs/node@8a87e39052)] - **test**: remove empty lines from snapshots (Marco Ippolito) [#&#8203;56358](nodejs/node#56358)
-   \[[`510649f617`](nodejs/node@510649f617)] - **test**: use unusual chars in the path to ensure our tests are robust (Antoine du Hamel) [#&#8203;48409](nodejs/node#48409)
-   \[[`54f6d681a0`](nodejs/node@54f6d681a0)] - **test**: remove flaky designation (Luigi Pinca) [#&#8203;56369](nodejs/node#56369)
-   \[[`20ace0bb01`](nodejs/node@20ace0bb01)] - **test**: remove test-worker-arraybuffer-zerofill flaky designation (Luigi Pinca) [#&#8203;56364](nodejs/node#56364)
-   \[[`b757e40525`](nodejs/node@b757e40525)] - **test**: remove test-net-write-fully-async-hex-string flaky designation (Luigi Pinca) [#&#8203;56365](nodejs/node#56365)
-   \[[`64556baddc`](nodejs/node@64556baddc)] - **test**: improve abort signal dropping test (Edy Silva) [#&#8203;56339](nodejs/node#56339)
-   \[[`accbdad329`](nodejs/node@accbdad329)] - **test**: enable ts test on win arm64 (Marco Ippolito) [#&#8203;56349](nodejs/node#56349)
-   \[[`4188ee00d1`](nodejs/node@4188ee00d1)] - **test**: deflake test-watch-file-shared-dependency (Luigi Pinca) [#&#8203;56344](nodejs/node#56344)
-   \[[`079cee0609`](nodejs/node@079cee0609)] - **test**: skip `test-sqlite-extensions` when SQLite is not built by us (Antoine du Hamel) [#&#8203;56341](nodejs/node#56341)
-   \[[`96a38044ee`](nodejs/node@96a38044ee)] - **test**: increase spin for eventloop test on s390 (Michael Dawson) [#&#8203;56228](nodejs/node#56228)
-   \[[`c062ffc242`](nodejs/node@c062ffc242)] - **test**: add coverage for pipeline (jakecastelli) [#&#8203;56278](nodejs/node#56278)
-   \[[`d4404f0d0e`](nodejs/node@d4404f0d0e)] - **test**: migrate message eval tests from Python to JS (Yiyun Lei) [#&#8203;50482](nodejs/node#50482)
-   \[[`9369942745`](nodejs/node@9369942745)] - **test**: check typescript loader (Marco Ippolito) [#&#8203;54657](nodejs/node#54657)
-   \[[`4930244484`](nodejs/node@4930244484)] - **test**: remove async-hooks/test-writewrap flaky designation (Luigi Pinca) [#&#8203;56048](nodejs/node#56048)
-   \[[`7819bfec69`](nodejs/node@7819bfec69)] - **test**: deflake test-esm-loader-hooks-inspect-brk (Luigi Pinca) [#&#8203;56050](nodejs/node#56050)
-   \[[`e9762bf005`](nodejs/node@e9762bf005)] - **test**: add test case for listeners (origranot) [#&#8203;56282](nodejs/node#56282)
-   \[[`c1627e9d19`](nodejs/node@c1627e9d19)] - **test**: make `test-permission-sqlite-load-extension` more robust (Antoine du Hamel) [#&#8203;56295](nodejs/node#56295)
-   \[[`97d854e1d5`](nodejs/node@97d854e1d5)] - **test_runner,cli**: mark test isolation as stable (Colin Ihrig) [#&#8203;56298](nodejs/node#56298)
-   \[[`a4f336fdd4`](nodejs/node@a4f336fdd4)] - **tools**: fix `require-common-first` lint rule from subfolder (Antoine du Hamel) [#&#8203;56325](nodejs/node#56325)
-   \[[`dc3dafcb50`](nodejs/node@dc3dafcb50)] - **tools**: add release line label when opening release proposal (Antoine du Hamel) [#&#8203;56317](nodejs/node#56317)
-   \[[`2a5ac932ac`](nodejs/node@2a5ac932ac)] - **url**: use resolved path to convert UNC paths to URL (Antoine du Hamel) [#&#8203;56302](nodejs/node#56302)
-   \[[`8b20cc212b`](nodejs/node@8b20cc212b)] - **(SEMVER-MINOR)** **worker**: add eval ts input (Marco Ippolito) [#&#8203;56394](nodejs/node#56394)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS45MS40IiwidXBkYXRlZEluVmVyIjoiMzkuOTEuNCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiUmVub3ZhdGUgQm90Il19-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. backported-to-v23.x PRs backported to the v23.x-staging branch. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. dont-land-on-v18.x PRs that should not land on the v18.x-staging branch and should not be released in v18.x. dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. dont-land-on-v22.x PRs that should not land on the v22.x-staging branch and should not be released in v22.x. needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants