v3.2.0

What's Changed

This release improves the reliability of the measurement, primarily focusing on the wall-time instrument.

🚀 Features

• Increase the min round time to a bigger value (+/- 1ms) by @art049

• Fix the random seed while measuring with instruments by @art049 in #48

🐛 Bug Fixes

• Use time per iteration instead of total round time in stats by @art049

🏗️ Refactor

• Replace hardcoded outlier factor for improved readability by @art049 in #67

⚙️ Internals

• Fix self-dependency by @adriencaccia in #66
• Fix uv version in CI by @adriencaccia
• Add benchmarks-walltime job to run additional performance benchmarks by @art049 in #65

⚠️ Since the wall-time instrument is still in beta, this minor release might create unexpected performance changes for this new instrument.

Full Changelog: v3.1.2...v3.2.0

v3.1.2

What's Changed

🐛 Bug Fixes

• fix: update package_data to include header and source files for valgrind wrapper by @art049 in #64

Full Changelog: v3.1.1...v3.1.2

v3.1.1

What's Changed

⚙️ Internals

• Add a py3-none-any fallback wheel by @art049 in #61

Full Changelog: v3.1.0...v3.1.1

v3.1.0

What's changed

🚀 Features

• Build the native extension during packaging to avoid runtime build issues by @art049 in #57

Full Changelog: v3.0.0...v3.1.0

v3.0.0

What's Changed

This new release includes a new wall time instrument that allows measurement of real execution timings. This will be very useful for macro benchmarks running on properly isolated machines. You can now use our CodSpeed Macro runners without changing anything to your existing CI workflows.

Find more info on our Walltime instrument here.

Breaking changes

• Dropped support for Python 3.8 since it reached its EOL

Features

• feat: add walltime clock benchmarking by @art049 in #41
• feat/add max time and quartiles by @art049 in #45
• feat: also save the lower and upper fences in the json data by @art049 in #46

Fixes

• fix: pytest-benchmark 5.0.0 compatibility by @art049 in #54
• fix: don't override pytest's default protocol by @kenodegard in #32

Internals

• chore: Add TCH, FA, and UP lints by @kenodegard in #29
• Expose type information by @Dreamsorcerer in #53
• ci: bump actions/checkout to v4 by @fargito in #47
• chore: support python 3.13 and drop 3.7 by @art049 in #40
• feat: use a true semver version by @art049 in #44
• test: add benches for TheAlgorithms/backtracking by @art049 in #43
• test: add benches for TheAlgorithms/audio_filters by @art049 in #42
• docs(readme): update action version in the CI workflow configuration by @frgfm in #39

New Contributors

• @kenodegard made their first contribution in #29
• @frgfm made their first contribution in #39
• @fargito made their first contribution in #47
• @Dreamsorcerer made their first contribution in #53

Full Changelog: v2.2.1...v3.0.0

v2.2.1

What's Changed

• fix: Loosen runtime requirements by @edgarrmondragon in #21
• fix: support pytest 8.1.1 by @art049 in #28

New Contributors

• @edgarrmondragon made their first contribution in #21

Full Changelog: v2.2.0...v2.2.1

v2.2.0

What's Changed

pytest-xdist is now fully supported. Thus if you have a lot of benches, you can totally leverage xdist test distribution (on the same CPU) to speed up benchmark execution.

Details

• feat: fix pytest-xdist-compatibility by @art049 in #18

Full Changelog: v2.1.0...v2.2.0

v2.1.0

What's Changed

Relative paths are now properly handled when using the working-directory action parameter.

Details

• fix: git relative path with working-directory by @art049 in #15

Internals

• fix: fix typo in release.yml by @art049 in #14

Full Changelog: v2.0.1...v2.1.0

v2.0.1

What's Changed

Added

• Added support for returning values by @patrick91 in #12
• Add a return type to the benchmark fixture by @art049 in #13

New Contributors

• @patrick91 made their first contribution in #12

Full Changelog: v2.0.0...v2.0.1

v2.0.0

🎉 What's Changed

• Flame graph generation is now supported with Python 3.12+, allowing more granularity in the performance reports. More details can be found in the documentation.

• Each benchmark is now warmed up a single time before each run to reduce the flakiness and the side effects related to the order in which the benchmarks are run.

🚀 Migration Path

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🎉!

We recommend you create a standalone PR to update the pytest-codspeed package, acknowledge the potential regressions, and merge it to update the baseline on your default branch.

📚 Details

Fixes

• support benchmark.extra_info from pytest-benchmark by @art049 in #10

Internals

• add introspection benchmarks by @art049 in #9
• bump isort to 5.12 by @art049 in #11

Full Changelog: v1.2.2...v2.0.0