Skip to content
This repository has been archived by the owner on Jun 20, 2023. It is now read-only.

Commit

Permalink
Improve performance of buzhash
Browse files Browse the repository at this point in the history
```
name             old time/op    new time/op    delta
Buzhash2/1K-4       610ns ± 4%     643ns ±16%    ~     (p=0.421 n=8+10)
Buzhash2/1M-4      1.25ms ± 5%    1.16ms ± 4%  -7.31%  (p=0.000 n=10+10)
Buzhash2/16M-4     19.2ms ± 2%    17.5ms ± 2%  -8.73%  (p=0.000 n=9+9)
Buzhash2/100M-4     117ms ± 1%     107ms ± 3%  -8.26%  (p=0.000 n=10+10)

name             old speed      new speed      delta
Buzhash2/1K-4    1.68GB/s ± 4%  1.60GB/s ±14%    ~     (p=0.408 n=8+10)
Buzhash2/1M-4     842MB/s ± 5%   908MB/s ± 3%  +7.86%  (p=0.000 n=10+10)
Buzhash2/16M-4    875MB/s ± 2%   959MB/s ± 2%  +9.57%  (p=0.000 n=9+9)
Buzhash2/100M-4   897MB/s ± 1%   977MB/s ± 3%  +9.02%  (p=0.000 n=10+10)

name             old alloc/op   new alloc/op   delta
Buzhash2/1K-4      1.17kB ± 1%    1.17kB ± 0%  -0.50%  (p=0.006 n=10+10)
Buzhash2/1M-4      1.08MB ± 1%    1.07MB ± 0%    ~     (p=0.739 n=10+10)
Buzhash2/16M-4     17.1MB ± 0%    17.1MB ± 0%    ~     (p=0.579 n=10+10)
Buzhash2/100M-4     106MB ± 0%     106MB ± 0%  -0.01%  (p=0.000 n=9+7)

name             old allocs/op  new allocs/op  delta
Buzhash2/1K-4        3.00 ± 0%      3.00 ± 0%    ~     (all equal)
Buzhash2/1M-4        8.00 ± 0%      8.00 ± 0%    ~     (all equal)
Buzhash2/16M-4       72.0 ± 0%      72.0 ± 0%    ~     (all equal)
Buzhash2/100M-4       406 ± 0%       406 ± 0%    ~     (all equal)
```

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
  • Loading branch information
Jakub Sztandera committed Nov 6, 2019
1 parent 21b0c06 commit 79bdab2
Showing 1 changed file with 21 additions and 5 deletions.
26 changes: 21 additions & 5 deletions buzhash.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,17 +61,33 @@ func (b *Buzhash) NextBytes() ([]byte, error) {

var state uint32 = 0

if buzMin > len(b.buf) {
panic("this is impossible")
}

for ; i < buzMin; i++ {
state = bits.RotateLeft32(state, 1)
state = state ^ bytehash[b.buf[i]]
}

if b.n+n > len(b.buf) {
panic("this is impossible, but gives +9 to performance")
}
{
max := b.n + n - 32 - 1

for ; state&buzMask != 0 && i < b.n+n; i++ {
state = bits.RotateLeft32(state, 1) ^ bytehash[b.buf[i-32]] ^ bytehash[b.buf[i]]
buf := b.buf
bufshf := b.buf[32:]
i = buzMin - 32
_ = buf[max]
_ = bufshf[max]

This comment has been minimized.

Copy link
@ribasushi

ribasushi Apr 6, 2020

Contributor

@Kubuxu I am not familiar with this idiom and googling does not turn up anything. What does looking at a value at an index and throwing it away achieve?

This comment has been minimized.

Copy link
@Kubuxu

Kubuxu Apr 6, 2020

Member

It is a hint for bounds checker that buf[x] and bufshf[x] with x in range of 0 to max are safe.
See: https://golang.org/src/encoding/binary/binary.go#L54
This allows for removal of runtime bounds checking in the loop (only the one before the loop is performed)

This comment has been minimized.

Copy link
@ribasushi

ribasushi Apr 7, 2020

Contributor

Knowing the name for this unlocks stuff like https://go101.org/article/bounds-check-elimination.html
All clear now, thanks!


for ; i <= max; i++ {
if state&buzMask == 0 {
break
}
state = bits.RotateLeft32(state, 1) ^
bytehash[buf[i]] ^
bytehash[bufshf[i]]
}
i += 32
}

res := make([]byte, i)
Expand Down

0 comments on commit 79bdab2

Please sign in to comment.