Fetch shim breaks in vercel edge environment

[dzhng]

When I'm using this SDK in a Vercel edge function, on LOCAL only, the request will refuse to send with the below error.

If I change the function back to nodejs environment, it works. OR, if I deploy the actual edge function (e.g. running on the real edge env, not the local one), it will also work.

IMO it's probably as much a Vercel issue as it is an issue with this SDK, but an easy way to fix is to just let users be able to define their own fetch implementation. I see that you already have the property for it in core.js, just add it as an option in the constructor.

[rattrayalex]

Which version of the library are you using? Can you share a full code sample, along with your vercel config?

[dzhng]

Latest version of this lib.

Here's a codesandbox to repro:
https://codesandbox.io/p/sandbox/serene-sammet-6dd394

Go to /app/api/route.ts, and point the browser to the /api route to run the code.

When runtime is set to edge, it fails with the error in the screenshot (this is not the exact same error I got, but I think it's because I'm using WSL whereas codesandbox runs another linux distro).

If you change the runtime to nodejs, it will be successful (it does throw a bunch of module not found errors, but I think that's just codesandbox issues).

One weird thing is - if I run another simpler prompt (like the one I commented out), it will still work in the edge environment. This has me wondering if it's a text encoding issue here.

[dzhng]

I think @jaredpalmer is looking into adding Anthropic support to Vercel edge streaming as part of their AI push. I wonder if they ran into similar issues.

[dzhng]

OK, it IS an encoding issue, specifically, with the em dash character (although I suspect it basically applies to all multi-byte UTF-8 encoded characters).

For example, even if I just send the em dash char (not normal dash e.g. -, but em dash e.g. —), it'll break the SDK.

[dzhng]

Last update - overriding the Content-Length in the header by passing in the correct one that takes into account multi-byte characters works.

Wrong length calculation on this line:
https://github.com/anthropics/anthropic-sdk-typescript/blob/main/src/core.ts#L132

The reason why it fails on edge is because Vercel uses undici in their edge runtime, which defaults to strict re Content-Length on write:
https://github.com/nodejs/undici/blob/22bdbd8c7820035276b4e876daccef513c29f5c4/lib/client.js#L1359

[rattrayalex]

Thank you so much for the repro and the terrific debugging here, @dzhng ! We'll aim to get a fix out for this today.

It may be a little complicated since Buffer isn't available in all runtimes, and we'll want to make sure that our fix is accurate everywhere.

[jedwards1211]

I think in the non-node case using Web API fetch it can set the Content-Length header automatically? But should check

[jedwards1211]

Yes from the fetch spec it looks like it automatically adds Content-Length: https://fetch.spec.whatwg.org/#http-network-or-cache-fetch, steps 8.5-9

[RobertCraigie]

This should be fixed in v0.5.4 - implementation in #47.

Please try it out and let us know if your issue is resolved!