Local results and perf comparisons for queries #6206
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
facebook-github-bot
added
the
CLA Signed
clee2000
force-pushed
the
csl/ch_query_perf_local
branch
2 times, most recently
from
3d18cc9 to
e93971e
Compare
| "params": { | ||
| "shas": "String" | ||
| }, | ||
| "tests": [] |
There was a problem hiding this comment.
We will probably need a linter for this later to enforce this structure I guess, maybe put up a warning about having no test case
There was a problem hiding this comment.
Added a simple linter for the structure but it doesn't check much more than that. No warnings for test cases yet
| for i, test in enumerate(tests): | ||
| new_results = query_clickhouse(query, test) | ||
| base_results = query_clickhouse(base_query, test) | ||
| if new_results != base_results: |
There was a problem hiding this comment.
This comparison might be brittle if the results returned by the query is not sorted. But I think we could keep it simple with the current implementation.
A more complex approach would be to provide an option to compare the results in strict mode (same order) and relax mode (in any order)
There was a problem hiding this comment.
Weirdly, a lot of clickhouse results are ordered, probably because the table themselves have orderings.
Added an arg to sort the results when comparing
| print(table) | ||
|
|
||
|
|
||
| def results_compare(args: argparse.Namespace) -> None: |
There was a problem hiding this comment.
For the comparison, an edge case might be when adding a new query or, in rarer case, to delete one.
clee2000
force-pushed
the
csl/ch_query_perf_local
branch
from
e93971e to
bcddc84
Compare
clee2000
force-pushed
the
csl/ch_query_perf_local
branch
from
0fb6559 to
e367365
Compare
Second commit onwards are the actual changes. The first commit just changes all the formatting for the existing files.
This changes the formatting of the params.json file to allow for example inputs for the queries that can be used to compare results and performance to be used in the script mentioned below. The new format can be seen in the README.
This also adds a script to get compare time and memory a query uses between local and a specific commit. It can also compare what the query returns.