Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The DuckDB version in framework cannot read the tables of recent duckdb native file #1223

Closed
fcontrepois opened this issue Apr 11, 2024 · 3 comments

Comments

@fcontrepois
Copy link

This is my first issue on GitHub so if I am missing something let me know.

The latest version of DuckDB is 0.10.1 and is the one that is installed by default. File .duckdb created by version 0.10.1 are not compatible with version 0.9.1 used in Framework.

By forcing the data loader to use a 0.9.1 makes it work.

It would be great if either Framework were updated to use the latest version of DuckDB or to allow users to update the framework package can be updated or both.

More details on file format compatibility is available here
Quoting for clarity:

Forward Compatibility
Forward compatibility refers to the ability of an older DuckDB version to read storage files produced by a newer DuckDB version. DuckDB v0.9 is partially forward compatible with DuckDB v0.10. Certain files created by DuckDB v0.10 can be read by DuckDB v0.9.
Forward compatibility is provided on a best effort basis. While stability of the storage format is important – there are still many improvements and innovations that we want to make to the storage format in the future. As such, forward compatibility may be (partially) broken on occasion.

@fcontrepois fcontrepois added the bug Something isn’t working label Apr 11, 2024
@Lawiss
Copy link

Lawiss commented Apr 11, 2024

I have the same problem. Currently I convert my db file into parquet to be able to use it. But duckdb native format is more efficient in term of compression.

@Fil
Copy link
Contributor

Fil commented Apr 11, 2024

Yes, it's a known issue that we hope to be able to solve once duckdb-wasm publishes a new release (it's not going to be easy, since there are lots of issues around duckdb and arrow versions).

See #1007 #1009 #1070, there are already ways to choose a more recent duckdb-wasm (1.28.1-dev179.0 just appeared 10 hours ago, see https://www.npmjs.com/package/@duckdb/duckdb-wasm?activeTab=versions).

@mbostock mbostock removed the bug Something isn’t working label Apr 11, 2024
@mbostock
Copy link
Member

Support for DuckDB 0.10.1 hasn’t yet been released in DuckDB-Wasm yet; you can use the prerelease (-dev) 1.28.1 release using the hints in Fil’s message. Generally speaking, you’ll automatically get the latest versions of libraries in Framework when they’re released without us having to do anything. We had to pin DuckDB-Wasm to 1.28.0 because they unintentionally marked the 1.28.1 prerelease as the latest tag (and for a while this prerelease was broken on jsDelivr). We can remove this when 1.28.1 is released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants