Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request for Prebuilt Packages for duckdb-node #70

Open
azmy60 opened this issue Mar 24, 2024 · 9 comments
Open

Request for Prebuilt Packages for duckdb-node #70

azmy60 opened this issue Mar 24, 2024 · 9 comments

Comments

@azmy60
Copy link
Contributor

azmy60 commented Mar 24, 2024

Hi,

First of all, thanks for the efforts in maintaining duckdb-node! We've noticed the yarn install/add process is quite time-consuming. I'm running on WSL, AMD Ryzen 5 5600H, laptop and the time it takes to run yarn after adding duckdb-node is > 20 minutes, and the number grows even more each time I run yarn add. This also happens during test automation on Github Actions.

Could prebuilt packages be made available to cover this?

Thanks!

@rathboma
Copy link

For context, Azmi and I work on Beekeeper Studio - github.com/beekeeper-studio/beekeeper-studio

The WASM build is great for front end, but because of electron we can use the 'full' duckdb.

Hoping something similar can be provided to how better-sqlite3 does their pre builds?

No idea how hard this is.

Love DuckDB, would be thrilled to add it to BKS ❤️

@maxzhuo
Copy link

maxzhuo commented Mar 26, 2024

Execute commands in the following order:

D:\app> npm i duckdb

D:\app> npm i @electron-forge/cli

D:\app> npx electron-forge import

D:\app> npm i electron-rebuild

D:\app> electron-rebuild -f -w duckdb

then... some bug errors, is my method incorrect? I am using the Win11 system.

@carlopi
Copy link
Collaborator

carlopi commented Apr 2, 2024

We distribute precompiled packages for a bunch of platform/arch/version combinations.

For details, this is the CI job that generated the duckdb-node npm package: https://github.com/duckdb/duckdb-node/actions/runs/8388373879.

Can you share the result of:

$ node
> process.arch
'arm64'
> process.platform
'darwin'
> process.version
'v21.7.1'

or directly look up whether your platform is currently being built?

@rathboma
Copy link

rathboma commented Apr 3, 2024

Ah, yes it's because we use Electron. Is there any chance you could add Electron to your prebuilds, in a similar way to how better-sqlite3 does it?

https://github.com/WiseLibs/better-sqlite3/blob/master/.github/workflows/build.yml#L17

Once it's added once we can help maintain it when new electron versions come out.

@rathboma
Copy link

@carlopi if we make a PR that adds electron pre builds would that be something you'd consider merging?

@RyanChapman2x
Copy link

@rathboma: perhaps try using electron-builder over electron-forge.

I was having trouble getting electron-forge to package my project with duckdb-node in any kind of reasonable timeframe (I waited for hours and never saw any progress). I searched around and found that a lot of people were having trouble with electron-forge for some packages like duckdb-node, so I made the switch. My build times now clock in at about 1-2 minutes.

@carlopi
Copy link
Collaborator

carlopi commented Apr 19, 2024

@carlopi if we make a PR that adds electron pre builds would that be something you'd consider merging?

It would be welcome a blueprint on how to do so / what changes are required.

@rathboma
Copy link

@RyanChapman2x we use Electron builder already, but the build times are very slow still. It's because there are no precompiled packages. Electron 18 is node 16 basically, so it shouldn't be too hard to figure out how to add this to the existing pipelines.

@rathboma
Copy link

@carlopi We've made a PR for this. Let us know what you think. Right now it's just for a couple of Electron versions, but it could be easily extended to automatically build for all published electron versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants