Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize str::repeat #48657

Merged
merged 3 commits into from
Mar 6, 2018
Merged

Optimize str::repeat #48657

merged 3 commits into from
Mar 6, 2018

Conversation

sinkuu
Copy link
Contributor

@sinkuu sinkuu commented Mar 2, 2018

Improves the performance of str::repeat by bulk copying. Here is the benchmarks of "abcde".repeat(n):

n old [ns/iter] new [ns/iter] diff [%]
1 27.205 27.421 +0.794
2 27.500 27.516 +0.0581
3 27.923 27.648 -0.985
4 31.206 30.145 -3.40
5 35.144 31.861 -9.34
7 43.131 34.621 -19.7
10 54.945 36.203 -34.1
100 428.31 52.895 -87.7

@rust-highfive
Copy link
Collaborator

r? @dtolnay

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 2, 2018
hanna-kruppe
hanna-kruppe reviewed Mar 2, 2018
View reviewed changes
src/liballoc/str.rs Outdated
return String::new();
}

// n = 2^k + l (2^k > l)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: l looks like 1. Better use another letter.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better not use single letter identifiers at all...

@Centril Centril added the T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. label Mar 3, 2018
@Centril
Copy link
Contributor

Centril commented Mar 3, 2018

Since this introduces new unsafe bits, and introduces non-trivial logic, I think more comments about why this is safe would be good as well as an overall explanation of the logic.

@dtolnay
Copy link
Member

dtolnay commented Mar 4, 2018

Nice work! I tried some tests like the following on my computer and see numbers similar to yours.

#[bench]
fn repeat100(b: &mut test::Bencher) {
    b.iter(|| { S.repeat(100); });
}
n old [ns/iter] new [ns/iter] diff [%]
1 28 28 0
2 28 28 0
3 29 29 0
4 33 31 −6.1
5 37 34 −8.1
7 46 35 −24.
10 59 37 −37.
100 444 65 −85.

@dtolnay
Copy link
Member

dtolnay commented Mar 4, 2018

@bors r+

@bors
Copy link
Contributor

bors commented Mar 4, 2018

📌 Commit 3d58543 has been approved by dtolnay

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 4, 2018
Manishearth added a commit to Manishearth/rust that referenced this pull request Mar 5, 2018
@Manishearth
Rollup merge of rust-lang#48657 - sinkuu:opt_str_repeat, r=dtolnay
aff626a 
Optimize str::repeat

Improves the performance of `str::repeat` by bulk copying. Here is the benchmarks of `"abcde".repeat(n)`:

|`n`|old [ns/iter]|new [ns/iter]|diff [%]|
---|---|---|---
|1|27.205|27.421|+0.794|
|2|27.500|27.516|+0.0581|
|3|27.923|27.648|-0.985|
|4|31.206|30.145|-3.40|
|5|35.144|31.861|-9.34|
|7|43.131|34.621|-19.7|
|10|54.945|36.203|-34.1|
|100|428.31|52.895|-87.7|
@Manishearth Manishearth mentioned this pull request Mar 5, 2018
Closed
@kennytm kennytm mentioned this pull request Mar 6, 2018
Merged
kennytm added a commit to kennytm/rust that referenced this pull request Mar 6, 2018
@kennytm
Rollup merge of rust-lang#48657 - sinkuu:opt_str_repeat, r=dtolnay
46d629a 
Optimize str::repeat

Improves the performance of `str::repeat` by bulk copying. Here is the benchmarks of `"abcde".repeat(n)`:

|`n`|old [ns/iter]|new [ns/iter]|diff [%]|
---|---|---|---
|1|27.205|27.421|+0.794|
|2|27.500|27.516|+0.0581|
|3|27.923|27.648|-0.985|
|4|31.206|30.145|-3.40|
|5|35.144|31.861|-9.34|
|7|43.131|34.621|-19.7|
|10|54.945|36.203|-34.1|
|100|428.31|52.895|-87.7|
bors added a commit that referenced this pull request Mar 6, 2018
@bors
Auto merge of #48768 - kennytm:rollup, r=kennytm
2789b06 
Rollup of 14 pull requests

- Successful merges: #48403, #48432, #48546, #48573, #48590, #48657, #48727, #48732, #48753, #48754, #48761, #48474, #48507, #47463
- Failed merges:
@alexcrichton alexcrichton merged commit 3d58543 into rust-lang:master Mar 6, 2018
@sinkuu sinkuu deleted the opt_str_repeat branch March 6, 2018 22:14
@scottmcm scottmcm mentioned this pull request Mar 7, 2018
Closed
alexcrichton added a commit to alexcrichton/rust that referenced this pull request Sep 20, 2018
@alexcrichton
std: Check for overflow in str::repeat
4d1d1be 
This commit fixes a buffer overflow issue in the standard library
discovered by Scott McMurray where if a large number was passed to
`str::repeat` it may cause and out of bounds write to the buffer of a `Vec`.
This bug was accidentally introduced in rust-lang#48657 when optimizing the
`str::repeat` function. The bug affects stable Rust releases 1.26.0 to
1.29.0. We plan on backporting this fix to create a 1.29.1 release, and
the 1.30.0 release onwards will include this fix.

The fix in this commit is to introduce a deterministic panic in the case of
capacity overflow. When repeating a slice where the resulting length is larger
than the address space, there’s no way it can succeed anyway!

The standard library and surrounding libraries were briefly checked to see if
there were othere instances of preallocating a vector with a calculation that
may overflow. No instances of this bug (out of bounds write due to a calculation
overflow) were found at this time.

Note that this commit is the first steps towards fixing this issue,
we'll be making a formal post to the Rust security list once these
commits have been merged.
alexcrichton added a commit to alexcrichton/rust that referenced this pull request Sep 20, 2018
@alexcrichton
std: Check for overflow in str::repeat
8ac88d3 
This commit fixes a buffer overflow issue in the standard library
discovered by Scott McMurray where if a large number was passed to
`str::repeat` it may cause and out of bounds write to the buffer of a `Vec`.
This bug was accidentally introduced in rust-lang#48657 when optimizing the
`str::repeat` function. The bug affects stable Rust releases 1.26.0 to
1.29.0. We plan on backporting this fix to create a 1.29.1 release, and
the 1.30.0 release onwards will include this fix.

The fix in this commit is to introduce a deterministic panic in the case of
capacity overflow. When repeating a slice where the resulting length is larger
than the address space, there’s no way it can succeed anyway!

The standard library and surrounding libraries were briefly checked to see if
there were othere instances of preallocating a vector with a calculation that
may overflow. No instances of this bug (out of bounds write due to a calculation
overflow) were found at this time.

Note that this commit is the first steps towards fixing this issue,
we'll be making a formal post to the Rust security list once these
commits have been merged.
alexcrichton added a commit to alexcrichton/rust that referenced this pull request Sep 20, 2018
@alexcrichton
std: Check for overflow in str::repeat
1b94b84 
This commit fixes a buffer overflow issue in the standard library
discovered by Scott McMurray where if a large number was passed to
`str::repeat` it may cause and out of bounds write to the buffer of a `Vec`.
This bug was accidentally introduced in rust-lang#48657 when optimizing the
`str::repeat` function. The bug affects stable Rust releases 1.26.0 to
1.29.0. We plan on backporting this fix to create a 1.29.1 release, and
the 1.30.0 release onwards will include this fix.

The fix in this commit is to introduce a deterministic panic in the case of
capacity overflow. When repeating a slice where the resulting length is larger
than the address space, there’s no way it can succeed anyway!

The standard library and surrounding libraries were briefly checked to see if
there were othere instances of preallocating a vector with a calculation that
may overflow. No instances of this bug (out of bounds write due to a calculation
overflow) were found at this time.

Note that this commit is the first steps towards fixing this issue,
we'll be making a formal post to the Rust security list once these
commits have been merged.
This was referenced Sep 20, 2018
Merged
Merged
Merged
bors added a commit that referenced this pull request Sep 20, 2018
@bors
Auto merge of #54397 - alexcrichton:fix-bug-stable, r=steveklabnik
b801ae6 
[stable] std: Check for overflow in `str::repeat`

This commit fixes a buffer overflow issue in the standard library
discovered by Scott McMurray where if a large number was passed to
`str::repeat` it may cause and out of bounds write to the buffer of a `Vec`.
This bug was accidentally introduced in #48657 when optimizing the
`str::repeat` function. The bug affects stable Rust releases 1.26.0 to
1.29.0. We plan on backporting this fix to create a 1.29.1 release, and
the 1.30.0 release onwards will include this fix.

The fix in this commit is to introduce a deterministic panic in the case of
capacity overflow. When repeating a slice where the resulting length is larger
than the address space, there’s no way it can succeed anyway!

The standard library and surrounding libraries were briefly checked to see if
there were othere instances of preallocating a vector with a calculation that
may overflow. No instances of this bug (out of bounds write due to a calculation
overflow) were found at this time.

Note that this commit is the first steps towards fixing this issue,
we'll be making a formal post to the Rust security list once these
commits have been merged.
bors added a commit that referenced this pull request Sep 21, 2018
@bors
Auto merge of #54398 - alexcrichton:fix-bug-beta, r=steveklabnik
1703b80 
[beta] std: Check for overflow in `str::repeat`

This commit fixes a buffer overflow issue in the standard library
discovered by Scott McMurray where if a large number was passed to
`str::repeat` it may cause and out of bounds write to the buffer of a `Vec`.
This bug was accidentally introduced in #48657 when optimizing the
`str::repeat` function. The bug affects stable Rust releases 1.26.0 to
1.29.0. We plan on backporting this fix to create a 1.29.1 release, and
the 1.30.0 release onwards will include this fix.

The fix in this commit is to introduce a deterministic panic in the case of
capacity overflow. When repeating a slice where the resulting length is larger
than the address space, there’s no way it can succeed anyway!

The standard library and surrounding libraries were briefly checked to see if
there were othere instances of preallocating a vector with a calculation that
may overflow. No instances of this bug (out of bounds write due to a calculation
overflow) were found at this time.

Note that this commit is the first steps towards fixing this issue,
we'll be making a formal post to the Rust security list once these
commits have been merged.
bors added a commit that referenced this pull request Sep 21, 2018
@bors
Auto merge of #54399 - alexcrichton:fix-bug, r=steveklabnik
1002e40 
std: Check for overflow in `str::repeat`

This commit fixes a buffer overflow issue in the standard library
discovered by Scott McMurray where if a large number was passed to
`str::repeat` it may cause and out of bounds write to the buffer of a `Vec`.
This bug was accidentally introduced in #48657 when optimizing the
`str::repeat` function. The bug affects stable Rust releases 1.26.0 to
1.29.0. We plan on backporting this fix to create a 1.29.1 release, and
the 1.30.0 release onwards will include this fix.

The fix in this commit is to introduce a deterministic panic in the case of
capacity overflow. When repeating a slice where the resulting length is larger
than the address space, there’s no way it can succeed anyway!

The standard library and surrounding libraries were briefly checked to see if
there were othere instances of preallocating a vector with a calculation that
may overflow. No instances of this bug (out of bounds write due to a calculation
overflow) were found at this time.

Note that this commit is the first steps towards fixing this issue,
we'll be making a formal post to the Rust security list once these
commits have been merged.
@scottmcm scottmcm mentioned this pull request Sep 21, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hanna-kruppe hanna-kruppe hanna-kruppe left review comments

@sanmai-NL sanmai-NL sanmai-NL left review comments

Assignees

@dtolnay dtolnay

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@sinkuu @rust-highfive @Centril @dtolnay @bors @hanna-kruppe @sanmai-NL @alexcrichton