Can't resolve 'bufferutil' and 'utf-8-validate' #179

NickyMeuleman · 2019-08-23T18:12:36Z

on netlify-lambda build

WARNING in /home/nicky/projects/jason-api/node_modules/subscriptions-transport-ws/node_modules/ws/lib/buffer-util.js
not found: Error: Can't resolve 'bufferutil' in '/home/nicky/projects/jason-api/node_modules/subscriptions-transport-ws/node_modules/ws/lib'

WARNING in /home/nicky/projects/jason-api/node_modules/subscriptions-transport-ws/node_modules/ws/lib/validation.js
not found: Error: Can't resolve 'utf-8-validate' in '/home/nicky/projects/jason-api/node_modules/subscriptions-transport-ws/node_modules/ws/lib'

Despite those errors, ./netlify/functions/graphql still works locally after running netlify-lambda serve

The repo I'm having this issue in is also the mininal reproduction https://github.com/NickyMeuleman/jason-api

The text was updated successfully, but these errors were encountered:

glennreyes · 2019-09-14T10:28:32Z

This can be solved by configuring your webpack externals:

//webpack.functions.js
const nodeExternals = require('webpack-node-externals');

module.exports = {
  externals: [nodeExternals()],
};

However, imho it is more reasonable to have this set up by default so there's no hassle with a custom webpack config when setting up a graphql lambda.

@sw-yx Any thoughts or objections on having this config by default? I'd be happy to open a PR adding it!

glennreyes · 2019-09-14T10:31:07Z

One thing I forgot to note, you'll also have to point to your config file in the netlify-lambda command, eg:

netlify-lambda build src/lambda --config webpack.functions.js

NickyMeuleman · 2019-09-25T17:03:44Z

Thanks so much for this!
Didn't make the connection I would need this.
I swear I read the README when swapping from netlify dev to netlify lambda...briefly 😊

PavelMGs · 2021-11-05T11:13:44Z

If you facing this problem with Next.JS, you just need to add isServer condition for it gonna work

if (isServer) { config.externals = [nodeExternals()] }

dquanghuy4444 · 2021-12-08T05:02:01Z

If you facing this problem with Next.JS, you just need to add isServer condition for it gonna work

if (isServer) { config.externals = [nodeExternals()] }

hi there , could u public your file , i have problem with nextjs

PavelMGs · 2021-12-10T12:36:32Z

If you facing this problem with Next.JS, you just need to add isServer condition for it gonna work
if (isServer) { config.externals = [nodeExternals()] }

hi there , could u public your file , i have problem with nextjs

@dquanghuy4444 I cant do it because of NDA, sorry. You can just show your config

dquanghuy4444 · 2022-01-02T13:02:45Z

If you facing this problem with Next.JS, you just need to add isServer condition for it gonna work
if (isServer) { config.externals = [nodeExternals()] }

hi there , could u public your file , i have problem with nextjs

@dquanghuy4444 I cant do it because of NDA, sorry. You can just show your config

thank bro , i install fabricjs (https://www.npmjs.com/package/fabric) , and deploy nextjs by netlify and i have this error
7:57:31 PM: ModuleNotFoundError: Module not found: Error: Can't resolve 'bufferutil' in '/opt/build/repo/node_modules/ws/lib'
7:57:31 PM: > Build error occurred
7:57:31 PM: Error: > Build failed because of webpack errors
7:57:31 PM: at /opt/build/repo/node_modules/next/dist/build/index.js:15:924
7:57:31 PM: at async Span.traceAsyncFn (/opt/build/repo/node_modules/next/dist/telemetry/trace/trace.js:6:584)

My next.config.js

const path = require("path")

const withSass = require("@zeit/next-sass")
const nodeExternals = require("webpack-node-externals")

module.exports = withSass({
    /* bydefault config  option Read For More Optios
     here https://github.com/vercel/next-plugins/tree/master/packages/next-sass*/

    cssModules: true
})

module.exports = {
    /* Add Your Scss File Folder Path Here */
    sassOptions: {
        includePaths: [path.join(__dirname, "styles")]
    }
}

module.exports = {
    productionBrowserSourceMaps: false
}

module.exports = {
    webpack: (config, { isServer }) => {
        // eslint-disable-next-line space-before-blocks
        if (isServer) {
            config.externals = nodeExternals()
        }

        return config
    }
}

module.exports = {
    images: {
        domains: ["cdn.shopify.com"]
    }
}

Jesuspinarte · 2022-05-04T16:00:57Z

The solution for me was add this ti my next.config.js:

const nodeExternals = require('webpack-node-externals');

const nextConfig = {
  reactStrictMode: true,
};

module.exports = {
  ...nextConfig,
  webpack: (config, { isServer }) => {
    if (!isServer) {
      config.externals = [nodeExternals()];
    }

    return config;
  },
};

I'm using Next 12.1.5

ThatGuySam · 2022-11-09T20:36:25Z

external_node_modules for netlify.toml is supposed to fix this but for some reason, it wouldn't work, however, when I tried including the node_module paths that fixed it.

# netlify.toml
[functions]
  included_files = [
    # For some reason this works to fix "bufferutil" error on
    # Netlify Function and external_node_modules does not
    "node_modules/bufferutil/**", 
    "node_modules/utf-8-validate/**"
  ]

irzko · 2023-03-01T16:50:16Z

module.exports = { ...nextConfig, webpack: ( config, { buildId, dev, isServer, defaultLoaders, nextRuntime, webpack } ) => { // Important: return the modified config if (isServer) { config.externals.push({ bufferutil: "bufferutil", "utf-8-validate": "utf-8-validate", }); } return config; }, };

kapysta-xxl · 2023-06-29T13:27:19Z

next version 13.4.7

npm i bufferutil@4.0.7 --save-dev
npm i utf-8-validate@5.0.10 --save-dev
npm i supports-color@8.1.1 --save-dev

// next.config.js

const nextConfig = {
  reactStrictMode: true,
};

module.exports = {
  ...nextConfig,
  webpack: (config, { isServer }) => {
    if (!isServer) {
      config.externals.push({ bufferutil: "bufferutil", "utf-8-validate": "utf-8-validate", "supports-color": "supports-color" }); 
    }

    return config;
  },
};

naingaungphyo · 2023-09-17T09:03:26Z

next version 13.4.7

npm i bufferutil@4.0.7 --save-dev npm i utf-8-validate@5.0.10 --save-dev npm i supports-color@8.1.1 --save-dev

// next.config.js
const nextConfig = {
  reactStrictMode: true,
};

module.exports = {
  ...nextConfig,
  webpack: (config, { isServer }) => {
    if (!isServer) {
      config.externals.push({ bufferutil: "bufferutil", "utf-8-validate": "utf-8-validate", "supports-color": "supports-color" }); 
    }

    return config;
  },
};

I have no idea what those packages are used for. Can someone clarify it please?

And my another concern is that don't we need those packages in production as we are installing with dev flag? Is not that something Nextjs needs to handle internally for their config?

when you try to use `@web3-storage/access` in a new next.js repository you get errors like this: ``` Import trace for requested module: ./node_modules/ipfs-utils/src/http/fetch.js ./node_modules/@web3-storage/upload-client/src/store.js ./node_modules/@web3-storage/upload-client/src/index.js ./node_modules/@web3-storage/w3up-client/src/client.js ./node_modules/@web3-storage/w3up-client/src/index.node.js ./app/utils/utils.ts ./app/page.tsx ./node_modules/ws/lib/buffer-util.js Module not found: Can't resolve 'bufferutil' in '/Users/travis/dev/pl/sample-w3/node_modules/ws/lib' Import trace for requested module: ./node_modules/ws/lib/buffer-util.js ./node_modules/ws/lib/websocket.js ./node_modules/ws/index.js ./node_modules/isomorphic-ws/node.js ./node_modules/@web3-storage/access/src/utils/ws.js ./node_modules/@web3-storage/access/src/agent-use-cases.js ./node_modules/@web3-storage/access/src/agent.js ./node_modules/@web3-storage/w3up-client/src/index.node.js ./app/utils/utils.ts ./app/page.tsx ./node_modules/ws/lib/validation.js Module not found: Can't resolve 'utf-8-validate' in '/Users/travis/dev/pl/sample-w3/node_modules/ws/lib' Import trace for requested module: ./node_modules/ws/lib/validation.js ./node_modules/ws/lib/receiver.js ./node_modules/ws/index.js ./node_modules/isomorphic-ws/node.js ./node_modules/@web3-storage/access/src/utils/ws.js ./node_modules/@web3-storage/access/src/agent-use-cases.js ./node_modules/@web3-storage/access/src/agent.js ./node_modules/@web3-storage/w3up-client/src/index.node.js ./app/utils/utils.ts ./app/page.tsx ``` Which looks like the same thing as netlify/netlify-lambda#179 and shows up a few other times in similar repos. While we could instruct our users to add some build configuration to go away, we don't use websockets in production in any of our flows and imho we should just remove this dependency. The easiest way to do this is to remove the cli as well, and since we don't use that anywhere that also seems like a good idea.

xalteropsx · 2024-01-09T05:29:05Z

what is for nuxt ?

muckee · 2025-01-09T23:27:10Z

I solved this by adding the following .yarnrc.yml file in the project root directory:

nodeLinker: pnp
pnpMode: loose

NickyMeuleman closed this as completed Sep 25, 2019

kimadeline mentioned this issue Nov 13, 2020

"Module not found" errors in prePublishBundle step but CI not failing microsoft/vscode-jupyter#2124

Closed

samepant mentioned this issue Feb 3, 2022

Adds webpack modification to build on netlify cosmos/cosmos-multisig-ui#34

Closed

hackinhikikomori mentioned this issue Dec 13, 2022

Can't Resolve 'utf-8-validate' or 'butterutil' in ws Library cosmos/cosmjs#1350

Closed

jaime-ez mentioned this issue Jan 10, 2023

NextJS 13 - PNPM - bufferuntil deepstreamIO/deepstream.io-client-js#586

Closed

penberg mentioned this issue May 4, 2023

Can't resolve 'utf-8-validation' on Windows tursodatabase/libsql-client-ts#33

Closed

nori26 mentioned this issue Jun 3, 2023

ホットリロード時に謎のmodule notfound s-xix98/trc-prot#306

Closed

travis mentioned this issue Oct 10, 2023

fix!: remove websocket support storacha/w3up#966

Merged

syssys131 mentioned this issue Dec 17, 2023

[SETTING] bufferutil, utf-8-validate 관련 에러 해결 BrokenRocketKR/near-2023#204

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't resolve 'bufferutil' and 'utf-8-validate' #179

Can't resolve 'bufferutil' and 'utf-8-validate' #179

NickyMeuleman commented Aug 23, 2019 •

edited

Loading

glennreyes commented Sep 14, 2019 •

edited

Loading

glennreyes commented Sep 14, 2019

NickyMeuleman commented Sep 25, 2019

PavelMGs commented Nov 5, 2021

dquanghuy4444 commented Dec 8, 2021

PavelMGs commented Dec 10, 2021 •

edited

Loading

dquanghuy4444 commented Jan 2, 2022 •

edited

Loading

Jesuspinarte commented May 4, 2022

ThatGuySam commented Nov 9, 2022 •

edited

Loading

irzko commented Mar 1, 2023

kapysta-xxl commented Jun 29, 2023 •

edited

Loading

naingaungphyo commented Sep 17, 2023 •

edited

Loading

xalteropsx commented Jan 9, 2024

muckee commented Jan 9, 2025

Can't resolve 'bufferutil' and 'utf-8-validate' #179

Can't resolve 'bufferutil' and 'utf-8-validate' #179

Comments

NickyMeuleman commented Aug 23, 2019 • edited Loading

glennreyes commented Sep 14, 2019 • edited Loading

glennreyes commented Sep 14, 2019

NickyMeuleman commented Sep 25, 2019

PavelMGs commented Nov 5, 2021

dquanghuy4444 commented Dec 8, 2021

PavelMGs commented Dec 10, 2021 • edited Loading

dquanghuy4444 commented Jan 2, 2022 • edited Loading

Jesuspinarte commented May 4, 2022

ThatGuySam commented Nov 9, 2022 • edited Loading

irzko commented Mar 1, 2023

kapysta-xxl commented Jun 29, 2023 • edited Loading

naingaungphyo commented Sep 17, 2023 • edited Loading

xalteropsx commented Jan 9, 2024

muckee commented Jan 9, 2025

NickyMeuleman commented Aug 23, 2019 •

edited

Loading

glennreyes commented Sep 14, 2019 •

edited

Loading

PavelMGs commented Dec 10, 2021 •

edited

Loading

dquanghuy4444 commented Jan 2, 2022 •

edited

Loading

ThatGuySam commented Nov 9, 2022 •

edited

Loading

kapysta-xxl commented Jun 29, 2023 •

edited

Loading

naingaungphyo commented Sep 17, 2023 •

edited

Loading