Skip to content

Latest commit

 

History

History
115 lines (71 loc) · 3.12 KB

ChangeLog.rst

File metadata and controls

115 lines (71 loc) · 3.12 KB
Raw

2.2.0

Key changes

  • Added simplified fixture SLTBENCH_FUNCTION_WITH_FIXTURE_BUILDER. It allows simple fixtures without implementing custom Fixture class. Variants with arguments and arguments generators are also provided.
  • Added DoNotOptimize utility function to prevent the code under testing being ompimized out by compiler.
  • Startup heatup time decreased to 3 seconds (was 7 seconds).

Other

  • Added runtime warning to console about DEBUG build configuration.
  • --name_prev and --name_next command line arguments of performance results comparator script allows to customize columns names with previous and next results.

2.1.0

  • Reorganized documentation.
  • Added "why sltbench" section to documentation to mension key features.
  • Added comparison with googlebench by key metrics.

2.0.0

Key changes

  • Speeded up benchmarking of fast functions (less than 0.00001 sec) in about 3x - 100x times.
  • Removed 20-nanoseconds (approx.) overhead on function call, which is critical for functions with execution time less then 0.0000005 sec (500 nanoseconds). For other functions 20 nanoseconds fits into timing error and does not influe on performance results.
  • Significantly improved reproducibility, quality and benchmarking time for functions with execution time about tens of nanoseconds.
  • Now sltbench is about 4x times faster than googlebench on training dataset and produce more stable results (see metrics section below).
  • Added tool to compare performance results. Usable to check was functions dataset speeded up or not. See tools/compare/main.py
  • Added tools to compare key project metrics for sltbench and competitors like googlebench (see tools/metrics): benchmark time, performance results reproducibilty (rel. error) and compilation time.
  • Added IMPRECISE status for datasets when sltbench cannot guarantee reproducible results. If you see IMPRECISE status, the problem is in a test in 99% cases. See the corresponding documentation notes about the most frequent cases of imprecise behaviour.

Other

  • Added --heatup off command line option to skip heatup stage.
  • Added --reporter command line option to choose report format.
  • Added warning log about the same test suite registered twice.

1.1.0

Key changes

  • Added lazy generators to save RAM for large input datasets.
  • Speeded up benchmarking of fast functions (less than 1 ms) in about 20%.

Other

  • Added tarvis and appveyor CI.
  • Added build instructions to README.

1.0.1

  • Fixed --filter option when positional command line argument is provided.

1.0.0

  • Initial release.
  • Added benchmarks for procedures, fixtures, arguments and generators.
  • Added bencmharks filtering by command line --filter option with regexp.
  • Added instruction HOWTO implement and run benchmarks.
  • Measure algorithm is learned on training dataset.
  • The first sltbench release test functions about 3x times faster than googlebench. Looks reasonable.