Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add AsyncBencher support #7

Merged
merged 1 commit into from
Jul 25, 2023
Merged

Conversation

art049
Copy link
Member

@art049 art049 commented Jul 25, 2023

Fixes #6

@art049 art049 force-pushed the feat/support-async-bencher branch from be5bcc9 to bb2f08b Compare July 25, 2023 12:50
@codspeed-hq
Copy link

codspeed-hq bot commented Jul 25, 2023

CodSpeed Performance Report

Merging #7 will degrade performances by 16.6%

⚠️ No base runs were found

Falling back to comparing feat/support-async-bencher (79ecdb7) with main (9c6a5a8)

Summary

❌ 4 regressions
✅ 37 untouched benchmarks

🆕 10 new benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main feat/support-async-bencher Change
from_elem_decimal[2048] 4.2 µs 5 µs -16.6%
Auto 223.3 ns 251.1 ns -11.06%
Linear 223.3 ns 251.1 ns -11.06%
Flat 672.8 ns 756.1 ns -11.02%
🆕 iter N/A 29.4 ns N/A
🆕 iter_with_setup N/A 29.4 ns N/A
🆕 iter_with_large_setup N/A 29.4 ns N/A
🆕 iter_with_large_drop N/A 29.4 ns N/A
🆕 iter_batched_small_input N/A 29.4 ns N/A
🆕 iter_batched_large_input N/A 29.4 ns N/A
🆕 iter_batched_per_iteration N/A 29.4 ns N/A
🆕 iter_batched_ref_small_input N/A 29.4 ns N/A
🆕 iter_batched_ref_large_input N/A 29.4 ns N/A
🆕 iter_batched_ref_per_iteration N/A 29.4 ns N/A

@art049 art049 changed the base branch from main to feat/add-warmup July 25, 2023 13:03
@art049 art049 force-pushed the feat/support-async-bencher branch from bb2f08b to 9d0a91f Compare July 25, 2023 13:04
@art049 art049 changed the base branch from feat/add-warmup to main July 25, 2023 13:05
@art049 art049 force-pushed the feat/support-async-bencher branch 5 times, most recently from 1bd65f8 to 3cc6bb8 Compare July 25, 2023 13:30
@art049 art049 force-pushed the feat/support-async-bencher branch from 7af5b43 to 79ecdb7 Compare July 25, 2023 15:10
@art049 art049 merged commit 79ecdb7 into main Jul 25, 2023
bors bot added a commit to evenfurther/pathfinding that referenced this pull request Sep 22, 2023
440: chore(deps): update pre-commit/action action to v3 r=samueltardieu a=renovate[bot]

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [pre-commit/action](https://togithub.com/pre-commit/action) | action | major | `v2.0.3` -> `v3.0.0` |

---

### Release Notes

<details>
<summary>pre-commit/action (pre-commit/action)</summary>

### [`v3.0.0`](https://togithub.com/pre-commit/action/releases/tag/v3.0.0): pre-commit/action@v3.0.0

[Compare Source](https://togithub.com/pre-commit/action/compare/v2.0.3...v3.0.0)

##### Breaking

-   remove pushing behaviour.
    -   PR [#&#8203;164](https://togithub.com/pre-commit/action/issues/164) by [`@&#8203;asottile](https://togithub.com/asottile).`

see [README] for alternatives

[README]: https://togithub.com/pre-commit/action#using-this-action-in-private-repositories

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/samueltardieu/pathfinding).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi45Ny4xIiwidXBkYXRlZEluVmVyIjoiMzYuOTcuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->


443: chore(deps): update rust crate codspeed-criterion-compat to v2 r=samueltardieu a=renovate[bot]

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [codspeed-criterion-compat](https://codspeed.io) ([source](https://togithub.com/CodSpeedHQ/codspeed-rust)) | dev-dependencies | major | `1.1.0` -> `2.2.0` |

---

### Release Notes

<details>
<summary>CodSpeedHQ/codspeed-rust (codspeed-criterion-compat)</summary>

### [`v2.2.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.2.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/v2.1.0...v2.2.0)

#### What's Changed

It's now possible to disable `codspeed_criterion_compat` default features.

##### Details

-   fix: allow disabling criterion default features by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#12

**Full Changelog**: CodSpeedHQ/codspeed-rust@v2.1.0...v2.2.0

### [`v2.1.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.1.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/v2.0.0...v2.1.0)

#### 🎉 What's Changed

-   Relative paths are now properly handled when using the `working-directory` action parameter.

#### 📚 Details

-   fix: fix relative path when using working-directory by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#9

**Full Changelog**: CodSpeedHQ/codspeed-rust@v2.0.0...v2.1.0

### [`v2.0.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.0.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/codspeed-criterion-compat@1.1.0...v2.0.0)

#### 🎉 What's Changed

-   Each benchmark is warmed up before each run to reduce the flakiness and the side effects related to the order in which the benchmarks are run.
-   `AsyncBencher` is now supported with the `criterion` compatibility layer.
-   Debug symbols are now generated during the build of the benchmarks, allowing more granularity in the performance reports and detailed profiling. More details can be found in [the documentation](https://docs.codspeed.io/features/trace-generation).

#### 🚀 Migration Path

This is a **breaking change** and might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🎉!

In some cases, the warmup might optimize out the benchmark. In that case, adding `black_box` should help, also don't hesitate to open an issue so we can have a look and document the behavior.

We recommend you create a standalone PR to update the package, acknowledge the potential regressions, and merge it to update the baseline on your default branch.

#### 📚 Details

##### Features

-   feat: add debug symbols in the benchmark build config by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#4
-   feat: add AsyncBencher support by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#7
-   feat!: add warmup runs  by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#5

##### Internals

-   chore: bump criterion by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#8

**Full Changelog**: CodSpeedHQ/codspeed-rust@v1.1.0...v2.0.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/samueltardieu/pathfinding).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi45Ny4xIiwidXBkYXRlZEluVmVyIjoiMzYuOTcuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->


Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
bors bot added a commit to evenfurther/pathfinding that referenced this pull request Sep 22, 2023
443: chore(deps): update rust crate codspeed-criterion-compat to v2 r=samueltardieu a=renovate[bot]

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [codspeed-criterion-compat](https://codspeed.io) ([source](https://togithub.com/CodSpeedHQ/codspeed-rust)) | dev-dependencies | major | `1.1.0` -> `2.2.0` |

---

### Release Notes

<details>
<summary>CodSpeedHQ/codspeed-rust (codspeed-criterion-compat)</summary>

### [`v2.2.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.2.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/v2.1.0...v2.2.0)

#### What's Changed

It's now possible to disable `codspeed_criterion_compat` default features.

##### Details

-   fix: allow disabling criterion default features by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#12

**Full Changelog**: CodSpeedHQ/codspeed-rust@v2.1.0...v2.2.0

### [`v2.1.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.1.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/v2.0.0...v2.1.0)

#### 🎉 What's Changed

-   Relative paths are now properly handled when using the `working-directory` action parameter.

#### 📚 Details

-   fix: fix relative path when using working-directory by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#9

**Full Changelog**: CodSpeedHQ/codspeed-rust@v2.0.0...v2.1.0

### [`v2.0.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.0.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/codspeed-criterion-compat@1.1.0...v2.0.0)

#### 🎉 What's Changed

-   Each benchmark is warmed up before each run to reduce the flakiness and the side effects related to the order in which the benchmarks are run.
-   `AsyncBencher` is now supported with the `criterion` compatibility layer.
-   Debug symbols are now generated during the build of the benchmarks, allowing more granularity in the performance reports and detailed profiling. More details can be found in [the documentation](https://docs.codspeed.io/features/trace-generation).

#### 🚀 Migration Path

This is a **breaking change** and might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🎉!

In some cases, the warmup might optimize out the benchmark. In that case, adding `black_box` should help, also don't hesitate to open an issue so we can have a look and document the behavior.

We recommend you create a standalone PR to update the package, acknowledge the potential regressions, and merge it to update the baseline on your default branch.

#### 📚 Details

##### Features

-   feat: add debug symbols in the benchmark build config by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#4
-   feat: add AsyncBencher support by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#7
-   feat!: add warmup runs  by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#5

##### Internals

-   chore: bump criterion by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#8

**Full Changelog**: CodSpeedHQ/codspeed-rust@v1.1.0...v2.0.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/samueltardieu/pathfinding).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi45Ny4xIiwidXBkYXRlZEluVmVyIjoiMzYuOTcuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->


Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
bors bot added a commit to evenfurther/pathfinding that referenced this pull request Sep 22, 2023
443: chore(deps): update rust crate codspeed-criterion-compat to v2 r=samueltardieu a=renovate[bot]

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [codspeed-criterion-compat](https://codspeed.io) ([source](https://togithub.com/CodSpeedHQ/codspeed-rust)) | dev-dependencies | major | `1.1.0` -> `2.2.0` |

---

### Release Notes

<details>
<summary>CodSpeedHQ/codspeed-rust (codspeed-criterion-compat)</summary>

### [`v2.2.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.2.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/v2.1.0...v2.2.0)

#### What's Changed

It's now possible to disable `codspeed_criterion_compat` default features.

##### Details

-   fix: allow disabling criterion default features by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#12

**Full Changelog**: CodSpeedHQ/codspeed-rust@v2.1.0...v2.2.0

### [`v2.1.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.1.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/v2.0.0...v2.1.0)

#### 🎉 What's Changed

-   Relative paths are now properly handled when using the `working-directory` action parameter.

#### 📚 Details

-   fix: fix relative path when using working-directory by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#9

**Full Changelog**: CodSpeedHQ/codspeed-rust@v2.0.0...v2.1.0

### [`v2.0.0`](https://togithub.com/CodSpeedHQ/codspeed-rust/releases/tag/v2.0.0)

[Compare Source](https://togithub.com/CodSpeedHQ/codspeed-rust/compare/codspeed-criterion-compat@1.1.0...v2.0.0)

#### 🎉 What's Changed

-   Each benchmark is warmed up before each run to reduce the flakiness and the side effects related to the order in which the benchmarks are run.
-   `AsyncBencher` is now supported with the `criterion` compatibility layer.
-   Debug symbols are now generated during the build of the benchmarks, allowing more granularity in the performance reports and detailed profiling. More details can be found in [the documentation](https://docs.codspeed.io/features/trace-generation).

#### 🚀 Migration Path

This is a **breaking change** and might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🎉!

In some cases, the warmup might optimize out the benchmark. In that case, adding `black_box` should help, also don't hesitate to open an issue so we can have a look and document the behavior.

We recommend you create a standalone PR to update the package, acknowledge the potential regressions, and merge it to update the baseline on your default branch.

#### 📚 Details

##### Features

-   feat: add debug symbols in the benchmark build config by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#4
-   feat: add AsyncBencher support by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#7
-   feat!: add warmup runs  by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#5

##### Internals

-   chore: bump criterion by [`@&#8203;art049](https://togithub.com/art049)` in [CodSpeedHQ/codspeed-rust#8

**Full Changelog**: CodSpeedHQ/codspeed-rust@v1.1.0...v2.0.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/samueltardieu/pathfinding).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi45Ny4xIiwidXBkYXRlZEluVmVyIjoiMzYuOTcuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->


Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

re: timeline on async benchmarks
1 participant