Run dbt on WebAssembly using Pyodide #5803
Conversation
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, don't hesitate to ping @drewbanin. CLA has not been signed by users: @arieldbt |
|
Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide. |
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, don't hesitate to ping @drewbanin. CLA has not been signed by users: @arieldbt |
arieldbt
force-pushed
the
arieldbt/pyodide
branch
from
79a62db to
4e79436
Compare
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, don't hesitate to ping @drewbanin. CLA has not been signed by users: @arieldbt |
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
|
@arieldbt looks great! I probably will have more time to play with it in October! |
core/setup.py
Outdated
| if "DBT_WASM_BUILD" in os.environ and int(os.environ["DBT_WASM_BUILD"]) == 1: | ||
| # binary dependency not supported in pyodide | ||
| pass | ||
| else: | ||
| _install_requires.insert(14, "dbt-extractor~=0.4.1") |
There was a problem hiding this comment.
Is there any cleaner way for us to manage this? Would this add another point of complexity that would prevent us from moving to pyproject.toml in the future? (#4674)
|
Ariel has moved on, so unless he wants to contribute to this as an external contributor... I've run pre-commit on it, and there are a number of complaints from mypy. I didn't look closely at all of them, but I think the code in flags.py for Pyodide should probably go somewhere else at least. |
|
👋 - been a bit busy but happy to see if I can help with the pre-commit / code-quality checks. Probably should have had the pre-commit hooks already set up on my machine. My bad. |
|
Closing in favor of #6146 |
Description
Changes needed to run dbt in the browser (using WebAssembly via Pyodide). For more context on the rationale see the discussion in github and our community slack. Here is a demo of the jaffle shop example project using sqlite and dbt in the browser:
https://s3.amazonaws.com/com.getdbt.cloud.wasm/index.html
To enable running dbt on WebAssembly I needed to refactor two parts of the codebase into clients since they behave differently in a browser compared to a normal operating system:
Checklist
changie newto create a changelog entry