Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade maturin to 0.14 #336

Merged
merged 1 commit into from
Nov 22, 2022
Merged

Conversation

messense
Copy link
Contributor

Cargo.lock is now included in sdist by default.

@codspeed-hq
Copy link

codspeed-hq bot commented Nov 22, 2022

CodSpeed Performance Report

Merging #336 maturin-0.14 (1e8c478) will not alter performances.

Summary

🔥 2 improvements
❌ 0 regressions
✅ 61 untouched benchmarks

🆕 0 new benchmarks
⁉️ 0 dropped benchmarks

Benchmarks breakdown

Benchmark main maturin-0.14 Change
🔥 test_many_models_core_model 100.4 ms 15.9 ms 84.11%
🔥 test_list_of_nullable_core 1,019 µs 905 µs 11.19%

@art049
Copy link
Contributor

art049 commented Nov 22, 2022

Do you think this performance change is accurate? @messense @samuelcolvin
This seems a bit extreme

@messense
Copy link
Contributor Author

maturin upgrade shouldn't affect performance in general.

@samuelcolvin
Copy link
Member

test_many_models_core_model matches the change on #327, so I would guess the strange result is on main.

But generally I do see these wide variations from time to time, @art049 any idea why?

@art049
Copy link
Contributor

art049 commented Nov 22, 2022

There are clearly 2 results clusters:
image

Between the 2, there are 5 times more instructions executed in the slowest one and 4~ times more memory writes.
I think it's probably on the Python side, maybe the garbage collection. I'll try to run some tests this afternoon.

Do you think it could also be something coming from the Rust part? This seems unlikely to me but I'm not familiar yet with the internals of Rust.
As you mentioned @samuelcolvin, I'll try to make the hashmap use a fixed seed to see if this is the root cause but with such a big difference I don't think it's the root cause of the flakiness.

Did you see something like this for this benchmark in the results from pytest-benchmark?

@art049
Copy link
Contributor

art049 commented Nov 22, 2022

I managed to reproduce:
image

In orange with garbage collection not disabled and in yellowish green with GC disabled. I'll disable it by default when running tests the pytest plugin.

@samuelcolvin
Copy link
Member

thanks so much @messense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants