Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: Body is unusable when using Remix(experimental-netlify-edge) Actions #3003

Closed
nickytonline opened this issue Apr 27, 2022 · 12 comments
Labels
bug Something isn't working feat:deno Issues related to Deno support

Comments

@nickytonline
Copy link
Contributor

nickytonline commented Apr 27, 2022

What version of Remix are you using?

0.0.0-experimental-fd9fa7f4

Steps to Reproduce

  1. Use Node 16.x or 18.x, the result is the same.
  2. Ensure that you have the Netlify CLI installed.
  3. Run npx create-remix --template https://github.com/netlify/remix-edge-template. Use the default answers when the Remix CLI asks you questions. Note: You can create a TypeScript project (default) or a JavaScript project. It doesn't matter, the result is the same in regards to the error.
  4. Modify the file /app/routes/index.js with the following code:
import type { LoaderFunction, ActionFunction } from "@remix-run/server-runtime"
import { useLoaderData } from "@remix-run/react";

let projectName: string | null = null;

export const loader: LoaderFunction = async () => {
  return { name: projectName ?? 'Create a Project Name' };
};

export const action: ActionFunction = async({ request }) => {
  const body = await request.formData();

  console.log('body',body);
  projectName = body.get("projectName") as string;
  console.log('projectName pulled from form data', projectName)
  console.log(`request body is used ${request.bodyUsed}`)

  console.log('action ran')

  return null
}

export default function Index() {
  let project = useLoaderData();

  return (
    <div style={{ fontFamily: "system-ui, sans-serif", lineHeight: "1.4" }}>
      <h1>Welcome to Remix {project.name}</h1>
      <ul>
        <li>
          <a
            target="_blank"
            href="https://remix.run/tutorials/blog"
            rel="noreferrer"
          >
            15m Quickstart Blog Tutorial
          </a>
        </li>
        <li>
          <a
            target="_blank"
            href="https://remix.run/tutorials/jokes"
            rel="noreferrer"
          >
            Deep Dive Jokes App Tutorial
          </a>
        </li>
        <li>
          <a target="_blank" href="https://remix.run/docs" rel="noreferrer">
            Remix Docs
          </a>
        </li>
      </ul>
    </div>
  );
}
  1. run ntl dev to start the application in development mode.
  2. Things build without any failures.
  3. Navigate to http://localhost:3000 and the Welcome to Remix page loads.

CleanShot 2022-04-27 at 14 15 40

  1. To test out the action, use a tool like Postman or Thunder Client (if using VS Code).

  2. Create a POST for the URL to http://localhost:3000/?index. Pass in some form data

This is how it would look in Postman

CleanShot 2022-04-29 at 16 56 46

  1. The tools above will respond with Unexpected Server Error (HTTP CODE 500). If you look at the logs in your terminal where you ran ntl dev, you'll see the following.
Watching Remix app in development mode...
◈ Loaded edge function server
💿 Built in 237ms
body FormData {
  [Symbol("entry list")]: [ { name: "projectName", value: "yolo" } ],
  [Symbol("[[webidl.brand]]")]: Symbol("[[webidl.brand]]")
}
projectName pulled from form data yolo
request body is used true
action ran
There was an error running the data loader for route routes/index
TypeError: Body is unusable.
    at EdgeRequest.clone (deno:ext/fetch/23_request.js:362:15)
    at handleDocumentRequest (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13507:51)
    at async requestHandler (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13667:22)
    at async Object.function (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13924:26)
    at async FunctionChain.runFunction (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/function_chain.ts:151:24)
    at async FunctionChain.run (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/function_chain.ts:130:22)
    at async handleRequest (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/handler.ts:35:22)
    at async Server.#respond (https://deno.land/std@0.114.0/http/server.ts:350:24)

There is also a discussion in Discord for more context if need be. See https://discord.com/channels/770287896669978684/778004294673760266/968514769223049326

Expected Behavior

No errors should occur when a Remix action is called, e.g. a POST to http://localhost:3000/?index when Netlify Edge functions are enabled.

Actual Behavior

An error occurs when a Remix action is called, e.g. a POST to http://localhost:3000/?index when Netlify Edge functions are enabled.

There was an error running the data loader for route routes/index
TypeError: Body is unusable.
    at EdgeRequest.clone (deno:ext/fetch/23_request.js:362:15)
    at handleDocumentRequest (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13507:51)
    at async requestHandler (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13667:22)
    at async Object.function (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13924:26)
    at async FunctionChain.runFunction (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/function_chain.ts:151:24)
    at async FunctionChain.run (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/function_chain.ts:130:22)
    at async handleRequest (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/handler.ts:35:22)
    at async Server.#respond (https://deno.land/std@0.114.0/http/server.ts:350:24)
@nickytonline
Copy link
Contributor Author

nickytonline commented Apr 27, 2022

As discussed on Discord, the reason TypeError: Body is unusable. is thrown is because the request is consumed before it's cloned and accessed again. This only occurs with Netlify Edge functions because they are streaming requests unlike regular Netlify functions, non-streamed requests, where the issue does not occur.

It's still unclear to me why this is only happening with Netlify Edge functions, but works on other platforms that support edge functions. Maybe the ReadableStream implemented on Deno is different than V8s in terms of expected behaviour?

The solution I had proposed was to do a request clone where the issue occurs, but as @mcansh mentioned on Discord, there used to be a request.clone(), but it was removed in #1766 because it was causing issues with larger payloads.

@nickytonline
Copy link
Contributor Author

I'm going to continue to look into this for the time being.

@nickytonline
Copy link
Contributor Author

nickytonline commented Apr 29, 2022

I updated the initial message as I had misunderstood the flow of the error.

  1. The action is being called and form data or JSON is accessible via await request.formData() or await request.json().
  2. The error occurs after the action is called.

It's still not clear to me what the issue is as the same thing works with regular Netlify functions. In handleDocumentRequest, there are matches to load that run via

  let matchesToLoad = matches || [];
  if (appState.catch) {
    matchesToLoad = getMatchesUpToDeepestBoundary(matchesToLoad.slice(0, -1), "CatchBoundary");
  } else if (appState.error) {
    matchesToLoad = getMatchesUpToDeepestBoundary(matchesToLoad.slice(0, -1), "ErrorBoundary");
  }
  console.log('matches to load')
  console.dir(matchesToLoad)
  let routeLoaderResults = await Promise.allSettled(matchesToLoad.map((match) => match.route.module.loader ? callRouteLoader({
    loadContext,
    match,
    request
  }) : Promise.resolve(void 0)));

For Edge functions the matches to load are the following:

matches to load

[
  {
    params: {},
    pathname: "/",
    route: {
      id: "root",
      parentId: undefined,
      path: "",
      index: undefined,
      caseSensitive: undefined,
      module: { default: [Getter], meta: [Getter] },
      children: [ [Object] ]
    }
  },
  {
    params: {},
    pathname: "/",
    route: {
      id: "routes/index",
      parentId: "root",
      path: undefined,
      index: true,
      caseSensitive: undefined,
      module: { action: [Getter], default: [Getter], loader: [Getter] },
      children: []
    }
  }
]

and when those Promises have settled, they result in our error in one of the entries

[
  { status: "fulfilled", value: undefined },
  {
    status: "rejected",
    reason: TypeError: Body is unusable.
    at EdgeRequest.clone (deno:ext/fetch/23_request.js:362:15)
    at callRouteLoader (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:12664:55)
    at file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13407:110
    at Array.map (<anonymous>)
    at handleDocumentRequest (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13407:67)
    at async requestHandler (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13670:22)
    at async Object.function (file:///Users/nicktaylor/dev/remix-edge-demo/.netlify/edge-functions/server.js:13927:26)
    at async FunctionChain.runFunction (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/function_chain.ts:151:24)
    at async FunctionChain.run (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/function_chain.ts:130:22)
    at async handleRequest (https://625d32be1b90870009edfc99--edge-bootstrap.netlify.app/bootstrap/handler.ts:35:22)
  }
]

For regular Netlify functions, as mentioned there is no error. The matches to load are the following:

[
  {
    params: {},
    pathname: '/',
    route: {
      id: 'root',
      parentId: undefined,
      path: '',
      index: undefined,
      caseSensitive: undefined,
      module: [Object],
      children: [Array]
    }
  },
  {
    params: {},
    pathname: '/',
    route: {
      id: 'routes/index',
      parentId: 'root',
      path: undefined,
      index: true,
      caseSensitive: undefined,
      module: [Object],
      children: []
    }
  }
]

and when the promises have settled, no errors:

[
  { status: 'fulfilled', value: undefined },
  {
    status: 'fulfilled',
    value: Response {
      size: 0,
      timeout: 0,
      [Symbol(Body internals)]: [Object],
      [Symbol(Response internals)]: [Object]
    }
  }
]

The flow in the code from what I can tell is the same whether it's Edge functions or Netlify functions, so I'm still not sure what's up.

In handleDocumentRequest when this is called,

      actionResponse = await data.callRouteAction({
        loadContext,
        match: actionMatch,
        request: request
      });

it will call the action, i.e.

    result = await action({
      request: stripDataParam(stripIndexParam(request)),
      context: loadContext,
      params: match.params
    });

see https://github.com/remix-run/remix/blob/main/packages/remix-server-runtime/data.ts#L35-L39

For Netlify functions request.bodyUsed is false right after the action is called, whereas for Netlify Edge functions, the request.bodyUsed is true. The only difference I can tell of is the main one, we have a Netlify Edge createRequestHandler, and it's a ReadableStream in deno, not Node.js

I'm going to chat with @ascorbic next week some more about this and will post more findings here.

Here is the source code that I've been using to debug this, https://github.com/nickytonline/debug-remix-netlify-edge-from

@mcansh mcansh added bug Something isn't working adapter:netlify-edge feat:deno Issues related to Deno support and removed bug:unverified labels May 3, 2022
@nickytonline
Copy link
Contributor Author

nickytonline commented May 3, 2022

Alright, so I did some more digging @mcansh and it turns out it's throwing in deno because deno is spec compliant for Request. I did a minimal repro and since Remix is using node-fetch's Request, there is something in there that is not spec compliant that masks the issue when using regular Netlify functions. I imagine you'll run into this once you move to native fetch in Node. Shoutout to @eduardoboucas for the assist on this!

Below is what happens, but I have a repository you can check out for yourself, https://github.com/nickytonline/remix-node-fetch-request-vs-deno-request-bug-repro

I'm simulating what this bit of code does:

function stripIndexParam(request) {
  let url = new URL(request.url);
  let indexValues = url.searchParams.getAll("index");
  url.searchParams.delete("index");
  let indexValuesToKeep = [];
  for (let indexValue of indexValues) {
    if (indexValue) {
      indexValuesToKeep.push(indexValue);
    }
  }
  for (let toKeep of indexValuesToKeep) {
    url.searchParams.append("index", toKeep);
  }
  return new Request(url.href, request);
}
function stripDataParam(request) {
  let url = new URL(request.url);
  url.searchParams.delete("_data");
  return new Request(url.href, request);
}

async function callRouteAction({
  loadContext,
  match,
  request
}) {
...

  let result;

  try {
    result = await action({
      request: stripDataParam(stripIndexParam(request)),
      context: loadContext,
      params: match.params
    });
  } catch (error) {
    if (!isResponse(error)) {
      throw error;
    }

...
}

As soon as that request is used later, request.bodyUsed is true, even though a new request was made via e.g. new Request(url.href, request).

Clear skies in Node.js with node-fetch

/** Run node index.js and you should see the following output:

body used: false
{ json: { message: 'Hello world!' } }
body used: false
{ json: { message: 'Hello world!' } }

 */
(async () => {
  const { default: fetch, Request} = await import('node-fetch');
  const firstRequest = new Request("https://post.deno.dev", {
    method: "POST",
    body: JSON.stringify({
      message: "Hello world!",
    }),
    headers: {
      "content-type": "application/json",
    },
  });
  const secondRequest = new Request('https://post.deno.dev', firstRequest);

  try {
    console.log(`body used: ${firstRequest.bodyUsed}`);
    const firstResponse = await fetch(firstRequest);
    const firstJson = await firstResponse.json();
    console.log(firstJson);
    console.log(`body used: ${firstRequest.bodyUsed}`);

    const secondResponse = await fetch(secondRequest);
    const secondJson = await secondResponse.json(); // No boom. All good because secondResponse.bodyUsed is false.
    console.log(secondJson);
  } catch (error) {
    console.error(error);
  }
})();

Rough waters as expected in specs compliant land with deno and the native Request object.

/** Run deno run --allow-all --unstable ./index.ts and you should see the following output:

body used: false
{ json: { message: "Hello world!" } }
body used: true
TypeError: Input request's body is unusable.
    at new Request (deno:ext/fetch/23_request.js:325:17)
    at deno:ext/fetch/26_fetch.js:422:29
    at new Promise (<anonymous>)
    at fetch (deno:ext/fetch/26_fetch.js:418:20)
    at file:///Users/nicktaylor/dev/deno-request-demo/index.ts:18:34

 */
const firstRequest = new Request("https://post.deno.dev", {
  method: "POST",
  body: JSON.stringify({
    message: "Hello world!",
  }),
  headers: {
    "content-type": "application/json",
  },
});
const secondRequest = new Request('https://post.deno.dev', firstRequest);

try {
  console.log(`body used: ${firstRequest.bodyUsed}`)
  const firstResponse = await fetch(firstRequest)
  const firstJson = await firstResponse.json()
  console.log(firstJson)
  console.log(`body used: ${firstRequest.bodyUsed}`)

  const secondResponse = await fetch(secondRequest)
  const secondJson = await secondResponse.json() // 💥 boom!
  console.log(secondJson)
} catch (error) {
  console.error(error);
}

@nickytonline
Copy link
Contributor Author

Since we don't want to clone because of #1766, I'm curious how we should proceed. Happy to help with this still.

@mcansh
Copy link
Collaborator

mcansh commented May 3, 2022

hey @nickytonline, thanks so much for digging into this!!

we're in the process of swapping node-fetch for @web-std/fetch (#2736) so i'll check that it works correctly there - im assuming once we switch we can freely clone the action request again

edit: over there, i get the following

body used: false
{ json: { message: 'Hello world!' } }
body used: false
TypeError [ERR_INVALID_STATE]: Invalid state: ReadableStream is locked

and cloning the secondRequest works as expected :)

const secondRequest = new Request("https://post.deno.dev", firstRequest.clone());

//=> body used: false
//=> { json: { message: 'Hello world!' } }
//=> body used: false
//=> { json: { message: 'Hello world!' } }

@ascorbic
Copy link
Contributor

ascorbic commented May 4, 2022

Great detective work 🕵🏻 ! As Deno and newer versions of Node support fetch natively, could we use the native version in those cases instead of the polyfill?

@nickytonline
Copy link
Contributor Author

@mcansh, I see that #2736 got merged. 🥳 For the request cloning, would you like me to put up a PR, or is the core team working on that?

@mcansh
Copy link
Collaborator

mcansh commented May 14, 2022

@mcansh, I see that #2736 got merged. 🥳 For the request cloning, would you like me to put up a PR, or is the core team working on that?

interestingly enough without any cloning it works https://netlify-thinks-mcansh-is-great.netlify.app/

https://github.com/mcansh/gdshfd-sjfbdsh

i'll verify again in the morning after the nightly goes out

@mcansh
Copy link
Collaborator

mcansh commented Jun 8, 2022

fixed by #3207

@mcansh mcansh closed this as completed Jun 8, 2022
@nickytonline
Copy link
Contributor Author

Woohoo! Nice work @jacob-ebey! 🔥

@SaiRev0
Copy link

SaiRev0 commented Dec 16, 2023

Writing this for others who will be coming here to for this error

I had the same issue while using react-hook-form. The issue with me was that I was doing this.

const { phone, hostel, roomNumber, gender, branch, year } = await req.json()
console.log(await req.json())

these two lines used together throws this error Body is unusable

So, to fix this, I just removed any one of them, and it's fixed. I don't know why it's like this, If anyone knows, I would love to know

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working feat:deno Issues related to Deno support
Projects
None yet
Development

No branches or pull requests

4 participants