Add bz1 size benchmarks

[benbjohnson]

Overview

This pull request add benchmarks to show the size difference between different block sizes.

Results

$ go test -run=XXX -bench=. ./tsdb/engine/bz1
PASS
BenchmarkEngine_WriteIndex_512b-4    1000000          2812 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_512b-4
    bz1_test.go:357: pts=  1000000  bytes/pt=13.5  leaf-util= 96%
BenchmarkEngine_WriteIndex_1KB-4     1000000          3102 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_1KB-4
    bz1_test.go:357: pts=  1000000  bytes/pt=12.4  leaf-util= 92%
BenchmarkEngine_WriteIndex_4KB-4     1000000          2362 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_4KB-4
    bz1_test.go:357: pts=  1000000  bytes/pt=11.2  leaf-util= 82%
BenchmarkEngine_WriteIndex_16KB-4    1000000          3006 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_16KB-4
    bz1_test.go:357: pts=  1000000  bytes/pt=10.6  leaf-util= 84%
BenchmarkEngine_WriteIndex_32KB-4    1000000          3386 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_32KB-4
    bz1_test.go:357: pts=  1000000  bytes/pt= 9.9  leaf-util= 90%
ok      github.com/influxdb/influxdb/tsdb/engine/bz1    20.413s

/cc @pauldix

[benbjohnson]

I added this to retrieve per-series page allocation stats.

[pauldix]

Can you go bigger than 32k and see what it does? Curious what it looks like all the way up to like 256k or something.

[pauldix]

Oh, also, will the leaf utilization have a big impact on the size of the overall BoltDB?

[benbjohnson]

@pauldix New results:

BenchmarkEngine_WriteIndex_512b-4    1000000          2935 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_512b-4
    bz1_test.go:360: pts=  1000000  bytes/pt=13.5  leaf-util= 96%
BenchmarkEngine_WriteIndex_1KB-4     1000000          2676 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_1KB-4
    bz1_test.go:360: pts=  1000000  bytes/pt=12.4  leaf-util= 92%
BenchmarkEngine_WriteIndex_4KB-4     1000000          2437 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_4KB-4
    bz1_test.go:360: pts=  1000000  bytes/pt=11.2  leaf-util= 82%
BenchmarkEngine_WriteIndex_16KB-4    1000000          2193 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_16KB-4
    bz1_test.go:360: pts=  1000000  bytes/pt=10.6  leaf-util= 84%
BenchmarkEngine_WriteIndex_32KB-4    1000000          2213 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_32KB-4
    bz1_test.go:360: pts=  1000000  bytes/pt= 9.9  leaf-util= 90%
BenchmarkEngine_WriteIndex_64KB-4    1000000          2207 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_64KB-4
    bz1_test.go:360: pts=  1000000  bytes/pt= 9.2  leaf-util= 96%
BenchmarkEngine_WriteIndex_128KB-4   1000000          2187 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_128KB-4
    bz1_test.go:360: pts=  1000000  bytes/pt= 9.1  leaf-util= 98%
BenchmarkEngine_WriteIndex_256KB-4   1000000          2405 ns/op
--- BENCH: BenchmarkEngine_WriteIndex_256KB-4
    bz1_test.go:360: pts=  1000000  bytes/pt= 9.0  leaf-util= 99%

The leaf utilization is the most important because there's a lot more leaf pages than branch pages. It gives a good indication of overall file utilization. However, the bytes/pt is really the more important metric.

There's a point of diminishing returns with large blocks because it's harder for Bolt to reallocate contiguous pages and you can get fragmentation. I think 32KB or 64KB is probably a good block size.

[pauldix]

Sure, 64k sounds good