Request for Prebuilt Packages for duckdb-node

[azmy60]

Hi,

First of all, thanks for the efforts in maintaining duckdb-node! We've noticed the yarn install/add process is quite time-consuming. I'm running on WSL, AMD Ryzen 5 5600H, laptop and the time it takes to run yarn after adding duckdb-node is > 20 minutes, and the number grows even more each time I run yarn add. This also happens during test automation on Github Actions.

Could prebuilt packages be made available to cover this?

Thanks!

[rathboma]

For context, Azmi and I work on Beekeeper Studio - github.com/beekeeper-studio/beekeeper-studio

The WASM build is great for front end, but because of electron we can use the 'full' duckdb.

Hoping something similar can be provided to how better-sqlite3 does their pre builds?

No idea how hard this is.

Love DuckDB, would be thrilled to add it to BKS ❤️

[maxzhuo]

Execute commands in the following order:

D:\app> npm i duckdb

D:\app> npm i @electron-forge/cli

D:\app> npx electron-forge import

D:\app> npm i electron-rebuild

D:\app> electron-rebuild -f -w duckdb

then... some bug errors, is my method incorrect? I am using the Win11 system.

[carlopi]

We distribute precompiled packages for a bunch of platform/arch/version combinations.

For details, this is the CI job that generated the duckdb-node npm package: https://github.com/duckdb/duckdb-node/actions/runs/8388373879.

Can you share the result of:

$ node
> process.arch
'arm64'
> process.platform
'darwin'
> process.version
'v21.7.1'

or directly look up whether your platform is currently being built?

[rathboma]

Ah, yes it's because we use Electron. Is there any chance you could add Electron to your prebuilds, in a similar way to how better-sqlite3 does it?

https://github.com/WiseLibs/better-sqlite3/blob/master/.github/workflows/build.yml#L17

Once it's added once we can help maintain it when new electron versions come out.

[rathboma]

@carlopi if we make a PR that adds electron pre builds would that be something you'd consider merging?

[RyanChapman2x]

@rathboma: perhaps try using electron-builder over electron-forge.

I was having trouble getting electron-forge to package my project with duckdb-node in any kind of reasonable timeframe (I waited for hours and never saw any progress). I searched around and found that a lot of people were having trouble with electron-forge for some packages like duckdb-node, so I made the switch. My build times now clock in at about 1-2 minutes.

[carlopi]

@carlopi if we make a PR that adds electron pre builds would that be something you'd consider merging?

It would be welcome a blueprint on how to do so / what changes are required.

[rathboma]

@RyanChapman2x we use Electron builder already, but the build times are very slow still. It's because there are no precompiled packages. Electron 18 is node 16 basically, so it shouldn't be too hard to figure out how to add this to the existing pipelines.

[rathboma]

@carlopi We've made a PR for this. Let us know what you think. Right now it's just for a couple of Electron versions, but it could be easily extended to automatically build for all published electron versions.