Add nightly benchmark and trigger in keyword #740
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR runs Nicolas' benchmark suite nightly and stores the results in a github artifact so anyone on the team can inspect them. It's also possible to trigger the benchmark suite manually by typing
RUN_BENCHMARK
in the PR description which is how this benchmark run was actually triggeredEventually we'd need to add more benchmarks, log them to the same directory, maybe do a dashboard and of course integrate it into an AWS infra but at least this way the team has a way of easily running benchmarks now without needing to manually spin up machine
h/t @xuzhao9