Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update kong and get rid of internal helpprinter #14

Merged
merged 1 commit into from
Apr 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 47 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,8 @@ It essentially combines multiple `git`, `go test -bench` and `benchstat` command

These are the basic steps `benchdiff` performs:
- Runs `go test -bench ...` on your current worktree and saves the results to a cache directory.
- Creates a git stash with the current state of your worktree
- Switches to the base ref (usually "HEAD" or "master").
- Runs `go test -bench ...` and saves the results to cache again.
- Switches back to the original git HEAD and restores the state from stash
- Creates a new worktree at the base ref
- Runs `go test -bench ...` on the base worktree and saves the results to cache again.
- Runs `benchstat` to compare the base and head results from cache.

## GitHub Action
Expand All @@ -42,51 +40,51 @@ Flags:
--version Output the benchdiff version and exit.
--debug write verbose output to stderr

--base-ref="HEAD" The git ref to be used as a baseline.
--cooldown=100ms How long to pause for cooldown between head and base runs.
--force-base Rerun benchmarks on the base reference even if the output already exists.
--git-cmd="git" The executable to use for git commands.
--json Format output as JSON.
--on-degrade=0 Exit code when there is a statistically significant degradation in the
results.
--tolerance=10.0 The minimum percent change before a result is considered degraded.

benchmark command line:
--bench="." Run only those benchmarks matching a regular expression. To run all
benchmarks, use '--bench .'.
--benchmark-args=args Override the default args to the go command. This may be a template.
See https://github.com/willabides/benchdiff for details."
--benchmark-cmd="go" The command to use for benchmarks.
--benchmem Memory allocation statistics for benchmarks.
--benchtime=STRING Run enough iterations of each benchmark to take t, specified as a
time.Duration (for example, --benchtime 1h30s). The default is 1
second (1s). The special syntax Nx means to run the benchmark N times
(for example, -benchtime 100x).
--count=10 Run each benchmark n times. If --cpu is set, run n times for each
GOMAXPROCS value.'
--cpu=GOMAXPROCS,... Specify a list of GOMAXPROCS values for which the benchmarks should
be executed. The default is the current value of GOMAXPROCS.
--packages="./..." Run benchmarks in these packages.
--show-bench-cmdline Instead of running benchmarks, output the command that would be used
and exit.
--tags=STRING Set the -tags flag on the go test command
--warmup-count=INT Run benchmarks with -count=n as a warmup
--warmup-time=STRING When warmups are run, set -benchtime=n

benchstat options:
--alpha=0.05 consider change significant if p < α
--benchstat-output="text" format for benchstat output (csv,html,markdown or text)
--delta-test="utest" significance test to apply to delta: utest, ttest, or none
--geomean print the geometric mean of each file
--norange suppress range columns (CSV and markdown only)
--reverse-sort reverse sort order
--sort="none" sort by order: delta, name, none
--split="pkg,goos,goarch" split benchmarks by labels

benchmark result cache:
--cache-dir=STRING Override the default directory where benchmark output is kept.
--clear-cache Remove benchdiff files from the cache dir.
--show-cache-dir Output the cache dir and exit.
--base-ref="HEAD" The git ref to be used as a baseline.
--cooldown=100ms How long to pause for cooldown between head and base runs.
--force-base Rerun benchmarks on the base reference even if the output already exists.
--git-cmd="git" The executable to use for git commands.
--json Format output as JSON.
--on-degrade=0 Exit code when there is a statistically significant degradation in the
results.
--tolerance=10.0 The minimum percent change before a result is considered degraded.

benchmark command line
--bench="." Run only those benchmarks matching a regular expression. To run all
benchmarks, use '--bench .'.
--benchmark-args=args Override the default args to the go command. This may be a template. See
https://github.com/willabides/benchdiff for details."
--benchmark-cmd="go" The command to use for benchmarks.
--benchmem Memory allocation statistics for benchmarks.
--benchtime=STRING Run enough iterations of each benchmark to take t, specified as a
time.Duration (for example, --benchtime 1h30s). The default is 1 second
(1s). The special syntax Nx means to run the benchmark N times (for
example, -benchtime 100x).
--count=10 Run each benchmark n times. If --cpu is set, run n times for each
GOMAXPROCS value.'
--cpu=GOMAXPROCS,... Specify a list of GOMAXPROCS values for which the benchmarks should be
executed. The default is the current value of GOMAXPROCS.
--packages="./..." Run benchmarks in these packages.
--show-bench-cmdline Instead of running benchmarks, output the command that would be used and
exit.
--tags=STRING Set the -tags flag on the go test command
--warmup-count=INT Run benchmarks with -count=n as a warmup
--warmup-time=STRING When warmups are run, set -benchtime=n

benchstat options
--alpha=0.05 consider change significant if p < α
--benchstat-output="text" format for benchstat output (csv,html,markdown or text)
--delta-test="utest" significance test to apply to delta: utest, ttest, or none
--geomean print the geometric mean of each file
--norange suppress range columns (CSV and markdown only)
--reverse-sort reverse sort order
--sort="none" sort by order: delta, name, none
--split="pkg,goos,goarch" split benchmarks by labels

benchmark result cache
--cache-dir=STRING Override the default directory where benchmark output is kept.
--clear-cache Remove benchdiff files from the cache dir.
--show-cache-dir Output the cache dir and exit.
```
<!--- end usage output --->

Expand Down
8 changes: 6 additions & 2 deletions cmd/benchdiff/benchdiff.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ import (

"github.com/alecthomas/kong"
"github.com/willabides/benchdiff/cmd/benchdiff/internal"
"github.com/willabides/benchdiff/cmd/benchdiff/internal/helpprinter"
"github.com/willabides/benchdiff/pkg/benchstatter"
"golang.org/x/perf/benchstat"
)
Expand Down Expand Up @@ -243,8 +242,13 @@ func main() {
benchVars["CacheDirDefault"] = filepath.Join(userCacheDir, "benchdiff")

kctx := kong.Parse(&cli, benchstatVars, benchVars, groupHelp,
kong.Help(helpprinter.NewHelpPrinter(nil)),
kong.Description(strings.TrimSpace(description)),
kong.ExplicitGroups([]kong.Group{
{Key: "benchstat", Title: "benchstat options"},
{Key: "cache", Title: "benchmark result cache"},
{Key: "gotest", Title: "benchmark command line"},
{Key: "x"},
}),
)

benchArgs, err := getBenchArgs()
Expand Down
9 changes: 0 additions & 9 deletions cmd/benchdiff/internal/helpprinter/guesswidth.go

This file was deleted.

41 changes: 0 additions & 41 deletions cmd/benchdiff/internal/helpprinter/guesswidth_unix.go

This file was deleted.

156 changes: 0 additions & 156 deletions cmd/benchdiff/internal/helpprinter/helpprinter.go

This file was deleted.

Loading