Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide Benchmark.quick_compare to quickly compare methods on an object #134

Merged
merged 6 commits into from
Sep 8, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,18 @@ One benefit to using this method is benchmark-ips automatically determines the
data points for testing our code, so we can focus on the results instead of
guessing iteration counts as we do with the traditional Benchmark library.

You can also use `ips_quick` to save a few lines of code:

```ruby
Benchmark.ips_quick([:upcase, :downcase], "hello") # runs a suite comparing "hello".upcase and "hello".downcase

def first; MyJob.perform(1); end
def second; MyJobOptimized.perform(1); end
Benchmark.ips_quick([:first, second]) # compares :first and :second
nateberkopec marked this conversation as resolved.
Show resolved Hide resolved
```

This adds a very small amount of overhead, which may be significant (i.e. ips_quick will understate the difference) if you're microbenchmarking things that can do over 1 million iterations per second. In that case, you're better off using the full format.

### Custom Suite

Pass a custom suite to disable garbage collection during benchmark:
Expand Down
4 changes: 2 additions & 2 deletions examples/quick.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ def sub
2 - 1
end

quick_compare(:add, :sub, warmup: 1, time: 1)
Benchmark.ips_quick([:add, :sub], warmup: 1, time: 1)

h = {}

h.quick_compare(:size, :empty?)
Benchmark.ips_quick([:size, :empty?], h)
37 changes: 32 additions & 5 deletions lib/benchmark/ips.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
require 'benchmark/ips/job/stream_report'
require 'benchmark/ips/job/multi_report'
require 'benchmark/ips/job'
require 'benchmark/ips/quick'

# Performance benchmarking library
module Benchmark
Expand Down Expand Up @@ -77,6 +76,34 @@ def ips(*args)
report
end

# Quickly compare multiple methods on the same object.
# @param methods [Array<Symbol>] An array of method names (as symbols) to compare.
# @param receiver [Object] The object on which to call the methods. Defaults to Kernel.
# @param opts [Hash] Additional options for customizing the benchmark.
# @option opts [Integer] :warmup The number of seconds to warm up the benchmark. (optional)
# @option opts [Integer] :time The number of seconds to run the benchmark. (optional)
nateberkopec marked this conversation as resolved.
Show resolved Hide resolved
#
# @example Compare String#upcase and String#downcase
# ips_quick([:upcase, :downcase], "hello")
#
# @example Compare two methods you just defined, with a custom warmup.
# def add; 1+1; end
# def sub; 2-1; end
# ips_quick([:add, :sub], warmup: 10)
def ips_quick(methods, receiver = Kernel, **opts)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if having quick in the name would make people think that it runs faster than the normal version

nateberkopec marked this conversation as resolved.
Show resolved Hide resolved
ips do |x|
x.compare!
x.warmup = opts[:warmup] if opts[:warmup]
x.time = opts[:time] if opts[:time]

methods.each do |name|
x.report(name) do |x|
x.times { receiver.__send__ name }
end
end
end
end

# Set options for running the benchmarks.
# :format => [:human, :raw]
# :human format narrows precision and scales results for readability
Expand All @@ -87,13 +114,13 @@ def self.options

module Helpers
SUFFIXES = ['', 'k', 'M', 'B', 'T', 'Q'].freeze

def scale(value)
scale = (Math.log10(value) / 3).to_i
scale = (Math.log10(value) / 3).to_i
scale = 0 if scale < 0 || scale >= SUFFIXES.size
suffix = SUFFIXES[scale]
scaled_value = value.to_f / (1000 ** scale)

"%10.3f#{suffix}" % scaled_value
end
module_function :scale
Expand All @@ -113,7 +140,7 @@ def humanize_duration(duration_ns)
end
end

extend Benchmark::IPS # make ips available as module-level method
extend Benchmark::IPS # make ips/ips_quick available as module-level method

##
# :singleton-method: ips
Expand Down
30 changes: 0 additions & 30 deletions lib/benchmark/ips/quick.rb

This file was deleted.

Loading