First parameter has member 'readable' that is not a ReadableStream

[jaschaio]

What version of Remix are you using?

1.15.0

Are all your remix dependencies & dev-dependencies using the same version?

• Yes

Steps to Reproduce

Using response.body.pipeThrough( new TextDecoderStream() ).getReader() throws an error if executed through a Remix Loader:

// where url is a streamable resource
const response = await fetch( url );

const reader = response.body.pipeThrough( new TextDecoderStream() ).getReader();

while ( true ) {

    const { value, done } = await reader.read();

    if ( done )
        break;

    console.log( value );

}

Expected Behavior

response.body.pipeThrough( new TextDecoderStream() ).getReader() should work as expected

Actual Behavior

Instead it throws an error:

TypeError: First parameter has member 'readable' that is not a ReadableStream.
      at assertReadableStream (/Volumes/Code/Ipsun/app/node_modules/web-streams-polyfill/src/lib/validators/readable-stream.ts:5:11)
      at convertReadableWritablePair (/Volumes/Code/Ipsun/app/node_modules/web-streams-polyfill/src/lib/validators/readable-writable-pair.ts:15:3)
      at ReadableStream.pipeThrough (/Volumes/Code/Ipsun/app/node_modules/web-streams-polyfill/src/lib/readable-stream.ts:211:23)

[Kalkut]

I have the same issue with TransformStream in version 1.17.1

It also seems related to #3862

[mindblight]

I followed a couple of threads on the @Kalkut provided. It looks like it's because remix uses https://github.com/MattiasBuelens/web-streams-polyfill/tree/master under the hood to polyfill ReadableStream on the server side. If you're running node >= 16.5, opening the CLI and running the same line works fine.

I tried new ReadableStream(response.body) to get around the polyfill, but no luck. Putting new ReadableStream().pipeThrough(new TextDecoderStream()); at the root of any route file causes the error. Using the same code in the browser also works fine, so I suspect that it's either 1) not being loaded, or 2) polyfilled differently between the environments.

Node 16 is at EOL (< 1 month left in security support at the time of this comment), and the vast majority of browsers support most of the streams API (https://caniuse.com/streams). It seems pretty safe to remove the polyfill at this point.

vercel/ai#199 (comment) provides one solution. I'm going to see if there's a simpler option. And post back if I find one.

[mindblight]

Alright, I figured something out:

// stream-compat.server.ts
import {
  ReadableStream as NodeReadableStream,
  TextDecoderStream as NodeTextDecoderStream,
  TextEncoderStream as NodeTextEncoderStream,
  TransformStream as NodeTransformStream,
  WritableStream as NodeWritableStream,
} from "node:stream/web";

async function* iterStream<T>(body?: ReadableStream<T> | null) {
  if (!body) {
    return;
  }
  const reader = body.getReader();

  while (true) {
    const r = await reader.read();
    if (r.done) {
      break;
    } else {
      yield r.value;
    }
  }
}

// Ensures that custom streams are compatible with node streams
(global as any).ReadableStream = NodeReadableStream;
(global as any).TransformStream = NodeTransformStream;
(global as any).WritableStream = NodeWritableStream;
(global as any).TextDecoderStream = NodeTextDecoderStream;
(global as any).TextEncoderStream = NodeTextEncoderStream;

export const toNodeReadableStream = <T>(
  polyStream: ReadableStream<T> | null | undefined
) => {
  const iterator = iterStream(polyStream);

  return new NodeReadableStream<T>({
    async pull(controller) {
      const r = await iterator.next();
      if (r.done) {
        controller.close();
      } else {
        controller.enqueue(r.value);
      }
    },
    // This is *mostly* correct. There are slight type issues between the two streams
  }) as ReadableStream<T>;
};

export const StreamCompat = {
  toReadable: toNodeReadableStream,
  // lib.dom.d.ts and web.d.ts typings are apparently incompatible.
  // Remix routes use web.d.ts and .server.ts files use lib.dom.d.ts
  // This exposes lib.dom.d.ts typings for the routes
  TextEncoderStream,
  TextDecoderStream,
};

In an action, you can now do

StreamCompat.toReadable(response.body).pipeThrough(...)

And everything should JustWork™️. I'm using TextDecoderStream with a number of custom streams to make handling the data easier, so I wanted to be able to use and define whatever I feel like.

In theory, this problem goes away with Remix@v2, so hopefully this solves the issue for people in the meantime.

[jacob-ebey]

See #3862 (comment) for why supporting fromWeb / toWeb isn't going to be doable.

If you need a native node ReadableStream you can use something like:

const passthrough = new PassThrough();
writeReadableStreamToWritable(polyfilledReadable, passthrough);
const nativeReadable = Readable.toWeb(passthrough);

[sidlak-c137]

Ran into this error when using the vercel ai package. Fixed by commenting out remix's fetch (using patch-package)

diff --git a/node_modules/@remix-run/node/dist/globals.js b/node_modules/@remix-run/node/dist/globals.js
index b96820b..7277c69 100644
--- a/node_modules/@remix-run/node/dist/globals.js
+++ b/node_modules/@remix-run/node/dist/globals.js
@@ -28,7 +28,7 @@ function installGlobals({
     global.Headers = UndiciHeaders;
     global.Request = UndiciRequest;
     global.Response = UndiciResponse;
-    global.fetch = undiciFetch;
+    // global.fetch = undiciFetch;
     global.FormData = UndiciFormData;
   } else {
     let {
@@ -43,7 +43,7 @@ function installGlobals({
     global.Headers = RemixHeaders;
     global.Request = RemixRequest;
     global.Response = RemixResponse;
-    global.fetch = RemixFetch;
+    // global.fetch = RemixFetch;
     global.FormData = RemixFormData;
   }
 }

[anthonyringoet]

What fixed it for me without patching: vercel/ai#199 (comment)