Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: extract storages into sub crates #6981

Merged
merged 17 commits into from
Aug 5, 2022

Conversation

dantengsky
Copy link
Member

@dantengsky dantengsky commented Aug 4, 2022

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

subtask of issue

All the refactoring work except explicitly noted, are pure renaming/moving things, no API/Implementation changed.

extract storages crates from crate databend-query

new crates extracted:

  • common-storages-hive (common/storages/hive)
    hive catalog, tables

  • common-storages-fuse (common/storages/fuse)
    fuse table

  • common-storages-index (common/storages/index)
    min/max index, bloom filter index

  • common-storages-preludes (common/storages/preludes)

    • github, memory, null, random, view
    • tables belong to information_schema
    • all the system tables except cluster

    stage, and result are kept left in crate databend-query, there are some unresolved dependency issues.

other extracted crates:

other crates extracted "on-demand"

  • common-storages-util (common/storages/preludes)

    • the Retry trait and its default impl
    • def of StorageContext
    • constants of table option key names

    better name?

  • common-pipeline-core (common/pipeline/core)

    former common-pipeline

  • common-pipeline-sinks (common/pipeline/sink)

  • common-pipeline-sources (common/pipeline/sources)

    decoupled from QueryContext, use trait TableContext instead

  • common-pipeline-transforms (common/pipeline/transforms)

    base types, and part of the transforms that table engine impls needed

  • common-legacy-parser (common/legacy-parser)

    the expression parser extracted from the legacy parser

about unit tests:

unit tests(tests/it/*) depends concrete type(especially the QueryContext), are not migratable yet

Fixes #issue

@vercel
Copy link

vercel bot commented Aug 4, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Updated
databend ⬜️ Ignored (Inspect) Aug 5, 2022 at 2:57AM (UTC)

@dantengsky dantengsky changed the title refat: split query refactor: split query Aug 4, 2022
@mergify mergify bot added the pr-refactor this PR changes the code base without new features or bugfix label Aug 4, 2022
@dantengsky dantengsky changed the title refactor: split query refactor: extract storages into sub crates Aug 4, 2022
@dantengsky dantengsky marked this pull request as ready for review August 5, 2022 02:35
Copy link
Member

@PsiACE PsiACE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@mergify mergify bot merged commit 490b243 into databendlabs:main Aug 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-refactor this PR changes the code base without new features or bugfix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants