Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OPFS (Origin Private File System) Support #1856

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

e1arikawa
Copy link

Description:

This PR implements OPFS (Origin Private File System) support in the latest version of duckdb-wasm based on PR #1490.
This allows database files to be read and written to the OPFS.

API:
When opening a database file, you can now use the following API:

await db.open({
    path: 'opfs://test.db',
    accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
});
const conn = await db.connect();

Changes:

  • Added support for OPFS.
    • Bug Fixes:
      • Resolved issues related to COPY TO functionality.
      • Fixed support for signed S3 URLs.

@carlopi
Copy link
Collaborator

carlopi commented Sep 18, 2024

This is very very welcome, thanks.

I am in the process of releasing a new version of duckdb-wasm, connected to duckdb 1.1.0.
Once that is out, I will properly review this PR.

@e1arikawa
Copy link
Author

@carlopi
Thank you very much for your update and for taking the time to review this PR.
I look forward to your feedback once the release is out.
Please let me know if there's anything I can assist with in the meantime.

@carlopi
Copy link
Collaborator

carlopi commented Sep 18, 2024

One comment would be if you could have a look at the failing test

@e1arikawa e1arikawa force-pushed the feature/opfs_support branch 2 times, most recently from bbb2067 to 383ff1b Compare September 23, 2024 02:57
@e1arikawa
Copy link
Author

e1arikawa commented Sep 24, 2024

@carlopi
I have merged the latest main branch and confirmed that the tests using DuckDB 1.1.1 pass successfully.

https://github.com/AKABANAKK/duckdb-wasm/actions/runs/11016245572

@carlopi
Copy link
Collaborator

carlopi commented Sep 25, 2024

Thanks, I gave a proper look, this looks solid, thanks a lot for the contribution.

This PR does introduces some API changes that might be somewhat unexpected, so I think the proper way forward would be:

  • tag v1.29.0 (later today)
  • merge this PR / iterate
  • tag v1.30.0 in the next weeks

@e1arikawa
Copy link
Author

e1arikawa commented Sep 25, 2024

@carlopi
Thank you for reviewing my contribution. Regarding the API changes, I will carefully test them after tagging v1.29.0.
Please proceed with the work as per your proposed plan. I appreciate your continued support.

@e1arikawa e1arikawa changed the title Add OPFS (Origin Private File System) Support to the Latest Version of duckdb-wasm Add OPFS (Origin Private File System) Support Oct 5, 2024
@e1arikawa
Copy link
Author

@carlopi
I will be using the release version that includes this PR for my work. Therefore, I would like to ask when the PR will be merged. If there is anything I can do to assist with the merge process, please let me know. Thank you for your time and support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants