Exponential Performance Regression in Rust 1.73 Read API from BorrowedCursor::ensure_init

[PSeitz]

Reported by a lz4_flex user PSeitz/lz4_flex#147

Code

I created a small repo to demonstrate on a minimal example.
https://github.com/PSeitz/copy_regression_1.73

BorrowedCursor::ensure_init from this PR #97015 seems exhibit exponential cost.

The program creates a Vec and reads it in chunks of 32kb. Size is specified as a command line argument (100MB-400MB).

time cargo run --release 100000000 (If your default is 1.73)

Executed in    3.84 secs    fish           external
   usr time    3.80 secs  268.00 micros    3.80 secs
   sys time    0.03 secs  262.00 micros    0.03 secs

time cargo run --release 200000000

Executed in   16.90 secs    fish           external
   usr time   16.82 secs  626.00 micros   16.82 secs
   sys time    0.04 secs    0.00 micros    0.04 secs

time cargo run --release 400000000

Executed in   72.84 secs    fish           external
   usr time   72.65 secs  586.00 micros   72.65 secs
   sys time    0.06 secs  138.00 micros    0.06 secs

time cargo +1.72 run --release 100000000

Executed in   57.23 millis    fish           external
   usr time   16.83 millis  404.00 micros   16.43 millis
   sys time   37.02 millis    0.00 micros   37.02 millis

time cargo +1.72 run --release 400000000

Executed in  257.30 millis    fish           external
   usr time  150.46 millis  477.00 micros  149.98 millis
   sys time  106.64 millis  112.00 micros  106.53 millis

[lqd]

Maybe try cargo-bisect-rustc (tutorial: https://rust-lang.github.io/cargo-bisect-rustc/) to find which PR caused this?

[PSeitz]

********************************************************************************
Regression in nightly-2023-07-10
********************************************************************************

fetching https://static.rust-lang.org/dist/2023-07-09/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2023-07-09: 40 B / 40 B [==============================================================================================================================================] 100.00 % 807.24 KB/s 
converted 2023-07-09 to 83964c156db1f444050a38b2498dbd0da6d5d503
fetching https://static.rust-lang.org/dist/2023-07-10/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2023-07-10: 40 B / 40 B [==============================================================================================================================================] 100.00 % 571.61 KB/s 
converted 2023-07-10 to 1065d876cdbc34a872b9e17c78caaa59ea0c94d4
looking for regression commit between 2023-07-09 and 2023-07-10
cloning rust repository
...
searched nightlies: from nightly-2022-11-01 to nightly-2023-11-01
regressed nightly: nightly-2023-07-10
searched commit range: https://github.com/rust-lang/rust/compare/83964c156db1f444050a38b2498dbd0da6d5d503...1065d876cdbc34a872b9e17c78caaa59ea0c94d4
regressed commit: https://github.com/rust-lang/rust/commit/a9eba8d793696d75c51a87ca731983bf5967028f

Really nice bisect tool.
From the commit list in that Auto merge it seems to be #113493

[lqd]

cc author @the8472

[the8472]

The loop creates a new BorrowedBuf on each iteration, which is fine if the reader can fill the provided buffer in one go but does unnecessary work on short reads. Should be easy to fix.

@rustbot claim

[bonsairobo]

Any chance of getting this fix in a backport to 1.73?