Module not found: Can't resolve 'path' on NextJS edge runtime.

[homanp]

Summary

When running langchainjs on the edge runtime in NextJS I see the following exception:

Error: Module not found: Can't resolve 'path'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/langchain/dist/base_language/count_tokens.js
./node_modules/langchain/dist/llms/openai.js
./node_modules/langchain/llms/openai.js

Currently running version 0.0.63.

[nfcampos]

See https://github.com/hwchase17/langchainjs/tree/main/test-exports-vercel for a working example

[homanp]

yep, I use the Next13 appDir route handlers. Perhaps that env differs somehow.

https://beta.nextjs.org/docs/routing/route-handlers

[mushan0x0]

Me too.

[Nedomas]

I believe it's because of this line:
https://github.com/hwchase17/langchainjs/blob/main/langchain/src/base_language/count_tokens.ts#L62
Additionally imported modules raise error on this line too: https://github.com/hwchase17/langchainjs/blob/main/langchain/src/text_splitter.ts#L1, there might be more related to tiktoken.

Not sure why - maybe Next.js 13 app dir route handlers with runtime: 'edge' import files differently?

Commenting this line fixes the error.

@nfcampos should tiktoken work with langchain on the edge or should we be skipping imports of it?

Version: 0.0.64

[nfcampos]

Ah interesting, thanks for investigating!

@dqbd any thoughts on this one? Would it be fixed/fixable with #847

[dqbd]

Hello! That definitely sounds like an issue with @dqbd/tiktoken, will address it as soon as possible.
This does to be a different issue than found in #847

[nfcampos]

Cool, thank you!

[dqbd]

Added a quick fix for both @dqbd/tiktoken and langchain
#1034

Please note, that until #847 is done and merged, the bundle size will be larger than the recommended 1 MB limit. WIll finish the PR in the upcoming days.

[nfcampos]

Thanks @dqbd ! This has been released as 0.0.66