Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support table options #1044

Merged
merged 5 commits into from
Feb 21, 2023

Conversation

v0y4g3r
Copy link
Contributor

@v0y4g3r v0y4g3r commented Feb 20, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Allow user to specify table options when creating table. These options are also persisted to manifest/metasrv.

We can now creating table like:

CREATE TABLE monitor (
    host STRING,
    ts TIMESTAMP,
    cpu DOUBLE DEFAULT 0,
    memory DOUBLE,
    TIME INDEX (ts),
    PRIMARY KEY(host)
) ENGINE=mito WITH(regions=1, ttl='1day', write_buffer_size='32KB');

The table options are persisted to table manifest:

$ cat  /tmp/greptimedb/data/greptime/public/1024/manifest/00000000000000000000.json | jq
...
        "options": {
          "write_buffer_size": "32KiB",
          "ttl": "1day",
          "extra_options": {
            "regions": "1"
          }
        },
        "created_on": "2023-02-20T09:58:50.390371Z"
      },
      "table_type": "Base"
    }
  }
}

Also in this PR, regions now respect user specified max_write_buffer_size when creating table. Table engine will create a SizeBasedStrategy with that value for each region.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

@v0y4g3r v0y4g3r force-pushed the feat/table-ttl-options branch from 90afb4d to d6f4a43 Compare February 20, 2023 10:03
@v0y4g3r v0y4g3r marked this pull request as ready for review February 20, 2023 11:37
@v0y4g3r v0y4g3r requested review from killme2008 and evenyag and removed request for killme2008 February 20, 2023 11:38
@v0y4g3r v0y4g3r force-pushed the feat/table-ttl-options branch from 67939cf to 4650d5a Compare February 20, 2023 11:42
@v0y4g3r v0y4g3r changed the title feat: table ttl options feat: support table options Feb 20, 2023
@v0y4g3r v0y4g3r force-pushed the feat/table-ttl-options branch from 4650d5a to 3d165ab Compare February 20, 2023 11:54
@codecov
Copy link

codecov bot commented Feb 20, 2023

Codecov Report

Merging #1044 (0427d64) into develop (23092a5) will decrease coverage by 0.11%.
The diff coverage is 91.58%.

@@             Coverage Diff             @@
##           develop    #1044      +/-   ##
===========================================
- Coverage    85.82%   85.72%   -0.11%     
===========================================
  Files          446      446              
  Lines        64897    65073     +176     
===========================================
+ Hits         55698    55781      +83     
- Misses        9199     9292      +93     
Flag Coverage Δ
rust 85.72% <91.58%> (-0.11%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/storage/src/flush.rs 79.86% <0.00%> (-3.36%) ⬇️
src/store-api/src/storage/engine.rs 100.00% <ø> (ø)
src/table/src/metadata.rs 98.44% <ø> (ø)
src/table/src/requests.rs 85.43% <88.88%> (+23.89%) ⬆️
src/datanode/src/sql/create.rs 90.71% <96.42%> (+0.44%) ⬆️
src/catalog/src/system.rs 94.76% <100.00%> (ø)
src/common/grpc-expr/src/alter.rs 94.21% <100.00%> (+0.06%) ⬆️
src/frontend/src/expr_factory.rs 92.72% <100.00%> (+1.29%) ⬆️
src/frontend/src/instance/distributed.rs 87.07% <100.00%> (ø)
src/meta-client/src/client.rs 94.61% <100.00%> (ø)
... and 19 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

src/datanode/src/sql/create.rs Show resolved Hide resolved
src/frontend/src/expr_factory.rs Outdated Show resolved Hide resolved
src/table/src/requests.rs Show resolved Hide resolved
src/table/src/requests.rs Show resolved Hide resolved
src/table/src/requests.rs Show resolved Hide resolved
src/table/src/requests.rs Show resolved Hide resolved
src/common/grpc-expr/src/error.rs Show resolved Hide resolved
src/datanode/src/error.rs Show resolved Hide resolved
src/datanode/src/sql/create.rs Outdated Show resolved Hide resolved
src/datanode/src/sql/create.rs Show resolved Hide resolved
src/datanode/src/sql/create.rs Show resolved Hide resolved
src/frontend/src/error.rs Show resolved Hide resolved
src/store-api/src/storage/engine.rs Show resolved Hide resolved
@MichaelScofield MichaelScofield enabled auto-merge (squash) February 21, 2023 07:37
@v0y4g3r v0y4g3r force-pushed the feat/table-ttl-options branch from 5e7caa5 to 69ddc0d Compare February 21, 2023 07:40
@v0y4g3r v0y4g3r force-pushed the feat/table-ttl-options branch from 69ddc0d to 0427d64 Compare February 21, 2023 07:47
@MichaelScofield MichaelScofield merged commit e17d5a1 into GreptimeTeam:develop Feb 21, 2023
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
* feat: change table options from string map to a struct, add ttl and write_buffer_size

* fix: also pass table options to table meta

* feat: pass table options when opening/creating regions

* fix: CR comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants