Skip to content

Commit

Permalink
Provide Benchmark.quick_compare to quickly compare methods on an obje…
Browse files Browse the repository at this point in the history
…ct (#134)

* Provide Benchmark.quick_compare to quickly compare methods on an object

* Move quick_compare to Kernel for more flexible usage

* New approach

* Splat methods, send all options to Job

* Tests

* Fix variable shadow

---------

Co-authored-by: Nate Berkopec <nate.berkopec@gmail.com>
  • Loading branch information
evanphx and nateberkopec authored Sep 8, 2024
1 parent a2c863d commit f66045d
Show file tree
Hide file tree
Showing 4 changed files with 76 additions and 6 deletions.
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,18 @@ One benefit to using this method is benchmark-ips automatically determines the
data points for testing our code, so we can focus on the results instead of
guessing iteration counts as we do with the traditional Benchmark library.

You can also use `ips_quick` to save a few lines of code:

```ruby
Benchmark.ips_quick(:upcase, :downcase, on: "hello") # runs a suite comparing "hello".upcase and "hello".downcase

def first; MyJob.perform(1); end
def second; MyJobOptimized.perform(1); end
Benchmark.ips_quick(:first, :second) # compares :first and :second
```

This adds a very small amount of overhead, which may be significant (i.e. ips_quick will understate the difference) if you're microbenchmarking things that can do over 1 million iterations per second. In that case, you're better off using the full format.

### Custom Suite

Pass a custom suite to disable garbage collection during benchmark:
Expand Down
17 changes: 17 additions & 0 deletions examples/quick.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env ruby

require 'benchmark/ips'

def add
1 + 1
end

def sub
2 - 1
end

Benchmark.ips_quick(:add, :sub, warmup: 1, time: 1)

h = {}

Benchmark.ips_quick(:size, :empty?, on: h)
41 changes: 35 additions & 6 deletions lib/benchmark/ips.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,14 @@
# Performance benchmarking library
module Benchmark
# Benchmark in iterations per second, no more guessing!
# @see https://github.com/evanphx/benchmark-ips
#
# See Benchmark.ips for documentation on using this gem~
#
# @see {https://github.com/evanphx/benchmark-ips}
module IPS

# Benchmark-ips Gem version.
VERSION = "2.13.0"
VERSION = "2.13.1"

# CODENAME of current version.
CODENAME = "Long Awaited"
Expand Down Expand Up @@ -73,6 +76,32 @@ def ips(*args)
report
end

# Quickly compare multiple methods on the same object.
# @param methods [Symbol...] A list of method names (as symbols) to compare.
# @param receiver [Object] The object on which to call the methods. Defaults to Kernel.
# @param opts [Hash] Additional options for customizing the benchmark.
# @option opts [Integer] :warmup The number of seconds to warm up the benchmark.
# @option opts [Integer] :time The number of seconds to run the benchmark.
#
# @example Compare String#upcase and String#downcase
# ips_quick(:upcase, :downcase, on: "hello")
#
# @example Compare two methods you just defined, with a custom warmup.
# def add; 1+1; end
# def sub; 2-1; end
# ips_quick(:add, :sub, warmup: 10)
def ips_quick(*methods, on: Kernel, **opts)
ips(opts) do |x|
x.compare!

methods.each do |name|
x.report(name) do |iter|
iter.times { on.__send__ name }
end
end
end
end

# Set options for running the benchmarks.
# :format => [:human, :raw]
# :human format narrows precision and scales results for readability
Expand All @@ -83,13 +112,13 @@ def self.options

module Helpers
SUFFIXES = ['', 'k', 'M', 'B', 'T', 'Q'].freeze

def scale(value)
scale = (Math.log10(value) / 3).to_i
scale = (Math.log10(value) / 3).to_i
scale = 0 if scale < 0 || scale >= SUFFIXES.size
suffix = SUFFIXES[scale]
scaled_value = value.to_f / (1000 ** scale)

"%10.3f#{suffix}" % scaled_value
end
module_function :scale
Expand All @@ -109,7 +138,7 @@ def humanize_duration(duration_ns)
end
end

extend Benchmark::IPS # make ips available as module-level method
extend Benchmark::IPS # make ips/ips_quick available as module-level method

##
# :singleton-method: ips
Expand Down
12 changes: 12 additions & 0 deletions test/test_benchmark_ips.rb
Original file line number Diff line number Diff line change
Expand Up @@ -276,4 +276,16 @@ def test_humanize_duration
assert_equal Benchmark::IPS::Helpers.humanize_duration(123456789.0123456789), "123.46 ms"
assert_equal Benchmark::IPS::Helpers.humanize_duration(123456789012.3456789012), "123.46 s"
end

def test_quick
Benchmark.ips_quick(:upcase, :downcase, on: "Hello World!", warmup: 0.001, time: 0.001)

assert $stdout.string.size > 0
end

def test_quick_on_kernel
Benchmark.ips_quick(:srand, :rand, warmup: 0.001, time: 0.001)

assert $stdout.string.size > 0
end
end

0 comments on commit f66045d

Please sign in to comment.