Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: multiple sharding keys implementation #681

Merged
merged 2 commits into from
Jul 1, 2023

Conversation

jjeffcaii
Copy link
Contributor

@jjeffcaii jjeffcaii commented May 4, 2023

What this PR does:

  1. tow low-level modules includes logic and calculus.
  2. sharding columns configuration refinement.

Which issue(s) this PR fixes:

Fixes #425

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


@jjeffcaii jjeffcaii added the WIP Work in progress label May 4, 2023
@codecov-commenter
Copy link

codecov-commenter commented May 4, 2023

Codecov Report

Patch coverage: 55.05% and project coverage change: +2.12 🎉

Comparison is base (c696476) 36.42% compared to head (a9c4d9c) 38.54%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #681      +/-   ##
==========================================
+ Coverage   36.42%   38.54%   +2.12%     
==========================================
  Files         228      235       +7     
  Lines       23759    26035    +2276     
==========================================
+ Hits         8654    10035    +1381     
- Misses      14106    14916     +810     
- Partials      999     1084      +85     
Impacted Files Coverage Δ
pkg/boot/discovery.go 59.90% <ø> (+10.61%) ⬆️
pkg/config/equals.go 7.40% <0.00%> (+0.99%) ⬆️
pkg/config/model.go 39.08% <ø> (+4.62%) ⬆️
pkg/config/tenant.go 0.00% <0.00%> (ø)
pkg/runtime/ast/create_table.go 0.00% <ø> (ø)
pkg/runtime/ast/select_element.go 16.66% <0.00%> (-0.47%) ⬇️
pkg/runtime/calc/logic/bool.go 0.00% <0.00%> (ø)
pkg/runtime/optimize/dml/insert.go 1.94% <0.00%> (+0.44%) ⬆️
pkg/runtime/optimize/dml/update.go 5.79% <0.00%> (+1.95%) ⬆️
pkg/runtime/plan/dal/show_database_rules.go 0.00% <0.00%> (ø)
... and 23 more

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@dongzl dongzl changed the title [WIP] feat: multiple sharding keys implementation feat: multiple sharding keys implementation May 13, 2023
- columns:
- name: uid
- name: friend_id
expr: parseInt(($0*31+$1) % 32 / 8)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the meaing of $0+31 ? 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The $index means the column of target index, and $0*31+$1 is equals with valueOf(uid) * 31 + valueOf(friend_id). BTW, it's just an example which show how the new expression works.

- column: uid
type: scriptExpr
- columns:
- name: uid
expr: parseInt($value % 32 / 8)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the placeholder $0 not used here, but $value

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be compatible with the old expression design, $value is the alias of $0.

pkg/proto/rule/rule.go Outdated Show resolved Hide resolved
pkg/runtime/ast/create_table.go Outdated Show resolved Hide resolved
@AlexStocks
Copy link
Contributor

@maronghe hey, guy, the ci failed.

@jjeffcaii jjeffcaii force-pushed the feat/issue-425 branch 6 times, most recently from f69708c to a1559e3 Compare June 18, 2023 13:42
@jjeffcaii jjeffcaii added kernel kernel and removed WIP Work in progress labels Jun 18, 2023
@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 18 Code Smells

No Coverage information No Coverage information
4.0% 4.0% Duplication

idea Catch issues before they fail your Quality Gate with our IDE extension sonarlint SonarLint

@@ -104,6 +108,16 @@ func (to *Topology) Enumerate() DatabaseTables {
return dt
}

func (to *Topology) Exists(dbIdx, tbIdx int) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not Exist?

@AlexStocks AlexStocks merged commit 9dfe877 into arana-db:master Jul 1, 2023
@maronghe
Copy link
Contributor

maronghe commented Jul 8, 2023

done

Lvnszn pushed a commit to Lvnszn/arana that referenced this pull request Jul 22, 2023
* composite_sharding (arana-db#665)

* feat: support multiple shard keys

---------

Co-authored-by: binbin.zhang <binbin0325@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kernel kernel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Sharding] Support multiple columns complex sharding key.
6 participants