Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggest Option<&T> instead of &Option<T> #13336

Merged
merged 3 commits into from
Sep 28, 2024
Merged

Conversation

nyurik
Copy link
Contributor

@nyurik nyurik commented Sep 3, 2024

closes #13054

// bad code
fn foo(a: &Option<T>) {}
fn bar(&self) -> &Option<T> {}

// Use instead
fn foo(a: Option<&T>) {}
fn bar(&self) -> Option<&T> {}

Handles argument types and return types in functions, methods, and closures with explicit types. Honors avoid_breaking_exported_api parameter.

See this great YouTube video with the in-depth explanation.

Open Questions

These are not blocking, and could be done in separate PRs if needed.

  • Should &Option<Box<T>> be suggested as Option<&T> -- without the box? Handled by clippy::borrowed_box
  • Should &Option<String> be suggested as Option<&str> -- using de-refed type?

Possible Future Improvements

These cases might also be good to handle, probably in a separate PR.

fn lambdas() {
  let x = |a: &Option<String>| {};
  let x = |a: &Option<String>| -> &Option<String> { todo!() };
}

fn mut_ref_to_ref(a: &mut &Option<u8>) {}

changelog: [ref_option]: Suggest Option<&T> instead of &Option<T>

@rustbot
Copy link
Collaborator

rustbot commented Sep 3, 2024

r? @llogiq

rustbot has assigned @llogiq.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties label Sep 3, 2024
@llogiq
Copy link
Contributor

llogiq commented Sep 3, 2024

I'd shorten the name to ref_option. Also are you going to work on the TODOs within this PR or do you want me to review it as is and follow up with another PR later?

@nyurik
Copy link
Contributor Author

nyurik commented Sep 3, 2024

@llogiq I would rather finish at least the first two TODOs before merging, but I can already use your help because there are a lot of in-code TODOs that I could use some help with. Thx!

@nyurik nyurik force-pushed the ref-option-sig branch 10 times, most recently from 6b40c38 to 16a588a Compare September 4, 2024 00:09
@nyurik
Copy link
Contributor Author

nyurik commented Sep 4, 2024

@llogiq I think I'm done with the changes I wanted to make - now it is all open questions and review :)

@PatchMixolydic
Copy link
Contributor

PatchMixolydic commented Sep 4, 2024

Should &mut Option<T> parameter be included? (currently yes)

This could break code that mutates the Option directly (eg. by setting it to None, or by overwriting a None with Some(concrete_value)).

@nyurik
Copy link
Contributor Author

nyurik commented Sep 4, 2024

@PatchMixolydic could you give a full example to make sure we evaluate it properly? I am suspecting mut might need to be excluded, but just to ensure we are talking about the same thing

@llogiq
Copy link
Contributor

llogiq commented Sep 4, 2024

Example:

let x = &mut Some(42);
*x = None;

@nyurik
Copy link
Contributor Author

nyurik commented Sep 4, 2024

@llogiq thx, but we should discuss it as a full "semi-working" example - i.e. as a function param. Although if you are also certain this should not be checked, i will just exclude it from the lint

@llogiq
Copy link
Contributor

llogiq commented Sep 4, 2024

Yes, I am certain this should not be linted. Please exclude it by only linting immutable RPtrs.

@nyurik
Copy link
Contributor Author

nyurik commented Sep 4, 2024

Should both of these cases be ignored?

fn mut_1(a: &mut Option<String>) {}
fn mut_2(a: &mut &Option<String>) {}

@llogiq
Copy link
Contributor

llogiq commented Sep 5, 2024

No, only the direct &mut Option. A &mut &T only means that the reference to T might be exchanged, but the object itself cannot be modified through the immutable reference.

Copy link
Contributor

@llogiq llogiq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some feedback re your TODOs. Hopefully this should give you a clear path forward.

/// Warns when a function signature uses `&Option<T>` instead of `Option<&T>`.
/// ### Why is this bad?
/// More flexibility, better memory optimization, and more idiomatic Rust code.
/// See [YouTube video](https://www.youtube.com/watch?v=6c7pZYP_iIE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// See [YouTube video](https://www.youtube.com/watch?v=6c7pZYP_iIE)
/// See this [YouTube video](https://www.youtube.com/watch?v=6c7pZYP_iIE)

Copy link
Contributor

@Rudxain Rudxain Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the video title and author/channel would also be nice, as it would serve as alt text

https://en.wikipedia.org/wiki/Wikipedia:Bare_URLs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Rudxain how can you add an alt text to a non-image/video link? I thought the ![Alt text](...) is only for images and such. Please give an example of what you think this markdown should be like. I couldn't find any other code in the rust repo that had alt text like this. Thx!

Copy link
Member

@jieyouxu jieyouxu Sep 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, you could try to summarize succintly the key reasoning from the YouTube video for why this might enable better optimizations (i.e. it's unlikely people are going to want to watch a youtube video linked from doc comments).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, updated.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also copy-pasta your summary to your PR description, makes it easier to understand why your PR included these changes 😸

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add an alt text to a non-image/video link?

I'm sorry for not being clear. I meant it as "equivalent to alt" not literally an alt. It doesn't matter how you add the extra information, what matters is the information itself.

I was thinking of a good ol' link: the "name" of the link can be used as an alt:

[This link points to an example web page](https://example.org)

clippy_lints/src/functions/ref_option.rs Outdated Show resolved Hide resolved
clippy_lints/src/functions/ref_option.rs Outdated Show resolved Hide resolved
clippy_lints/src/functions/ref_option.rs Outdated Show resolved Hide resolved
clippy_lints/src/functions/ref_option.rs Outdated Show resolved Hide resolved
clippy_lints/src/functions/ref_option.rs Outdated Show resolved Hide resolved
@nyurik nyurik force-pushed the ref-option-sig branch 4 times, most recently from b0f0e62 to 87e6df6 Compare September 28, 2024 02:19
@nyurik
Copy link
Contributor Author

nyurik commented Sep 28, 2024

@flip1995 thx for all the help with this PR on Zullip! @llogiq same goes to you for the feedback here. I think this is ready for another round of reviews. Thx!

@nyurik nyurik force-pushed the ref-option-sig branch 2 times, most recently from 2eb85d2 to 7abb99a Compare September 28, 2024 03:25
bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 28, 2024
Migrate compiler's `&Option<T>` into `Option<&T>`

Similar to rust-lang#130962 but for the compiler.

Trying out my new lint rust-lang/rust-clippy#13336 - according to the [video](https://www.youtube.com/watch?v=6c7pZYP_iIE), this could lead to some performance and memory optimizations.
Comment on lines 410 to 414
/// `&Option<T>` in a function signature breaks encapsulation because the caller must own T
/// and move it into an Option to call with it. When returned, the owner must internally store
/// it as `Option<T>` in order to return it.
/// At a lower level `&Option<T>` points to memory that has `presence` bit flag + value,
/// whereas `Option<&T>` is always optimized to a single pointer.
Copy link
Member

@jieyouxu jieyouxu Sep 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remark: it would be good if we can reference any guarantees that these optimizations are performed for justification of this lint. We should double-check Option<T> representation guarantees by std docs: https://doc.rust-lang.org/std/option/index.html#representation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx, i updated the text with the link

Comment on lines 413 to 414
/// At a lower level `&Option<T>` points to memory that has `presence` bit flag + value,
/// whereas `Option<&T>` is always optimized to a single pointer.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: is this always guaranteed, or is it only guaranteed if T: Sized (I haven't looked very hard myself)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I haven't looked into this too far; please take everything I say with a grain of salt)

Option's documentation in std only guarantees this for &{,mut} impl Sized, but it seems like the Unsafe Code Guidelines Working Group tenatively posit that similar optimizations occur for all values with a niche, which would include &{,mut} impl !Sized (as all references must be nonnull, even if their pointee is !Sized).

Obviously, though, Option<&impl!Thin> cannot be optimized to a thin pointer, and there does not seem to be any guarantee about what the metadata component would be in None::<&impl !Thin>.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, which is why I asked, because I wasn't sure if this is always the case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real question is if we want to have all this complexity explained in a lint? I agree we should link to the options doc, and perhaps talk about possible performance improvement. Any suggestions on the wording?

An even more important question is: how common is a case where API is deliberately, as oppose to accidentally, designed to take or return &Option<T>? If this is extremely rare, I think it makes sense to promote this to style, at least in the longer term.

nyurik added a commit to nyurik/cargo that referenced this pull request Sep 28, 2024
As part of this, also modify `Option<&String>` to `Option<&str>`, same for `PathBuf`

Trying out my new lint rust-lang/rust-clippy#13336 - according to the [video](https://www.youtube.com/watch?v=6c7pZYP_iIE), this could lead to some performance and memory optimizations.

Basic thoughts expressed in the video that seem to make sense:
* `&Option<T>` in an API breaks encapsulation:
  * caller must own T and move it into an Option to call with it
  * if returned, the owner must store it as Option<T> internally in order to return it
* Performance is subject to compiler optimization, but at the basics, `&Option<T>` points to memory that has `presence` flag + value, whereas `Option<&T>` by specification is always optimized to a single pointer.
nyurik added a commit to nyurik/cargo that referenced this pull request Sep 28, 2024
As part of this, also modify `Option<&String>` to `Option<&str>`, same for `PathBuf`

Trying out my new lint rust-lang/rust-clippy#13336 - according to the [video](https://www.youtube.com/watch?v=6c7pZYP_iIE), this could lead to some performance and memory optimizations.

Basic thoughts expressed in the video that seem to make sense:
* `&Option<T>` in an API breaks encapsulation:
  * caller must own T and move it into an Option to call with it
  * if returned, the owner must store it as Option<T> internally in order to return it
* Performance is subject to compiler optimization, but at the basics, `&Option<T>` points to memory that has `presence` flag + value, whereas `Option<&T>` by specification is always optimized to a single pointer.
@llogiq
Copy link
Contributor

llogiq commented Sep 28, 2024

Looks ok to me. Thank you!

@bors r+

@bors
Copy link
Collaborator

bors commented Sep 28, 2024

📌 Commit 10e02cf has been approved by llogiq

It is now in the queue for this repository.

@bors
Copy link
Collaborator

bors commented Sep 28, 2024

⌛ Testing commit 10e02cf with merge 7b566c2...

@bors
Copy link
Collaborator

bors commented Sep 28, 2024

☀️ Test successful - checks-action_dev_test, checks-action_remark_test, checks-action_test
Approved by: llogiq
Pushing 7b566c2 to master...

@bors bors merged commit 7b566c2 into rust-lang:master Sep 28, 2024
11 checks passed
nyurik added a commit to nyurik/cargo that referenced this pull request Sep 28, 2024
As part of this, also modify `Option<&String>` to `Option<&str>`, same for `PathBuf`

Trying out my new lint rust-lang/rust-clippy#13336 - according to the [video](https://www.youtube.com/watch?v=6c7pZYP_iIE), this could lead to some performance and memory optimizations.

Basic thoughts expressed in the video that seem to make sense:
* `&Option<T>` in an API breaks encapsulation:
  * caller must own T and move it into an Option to call with it
  * if returned, the owner must store it as Option<T> internally in order to return it
* Performance is subject to compiler optimization, but at the basics, `&Option<T>` points to memory that has `presence` flag + value, whereas `Option<&T>` by specification is always optimized to a single pointer.
nyurik added a commit to nyurik/cargo that referenced this pull request Sep 28, 2024
As part of this, also modify `Option<&String>` to `Option<&str>`, same for `PathBuf`

Trying out my new lint rust-lang/rust-clippy#13336 - according to the [video](https://www.youtube.com/watch?v=6c7pZYP_iIE), this could lead to some performance and memory optimizations.

Basic thoughts expressed in the video that seem to make sense:
* `&Option<T>` in an API breaks encapsulation:
  * caller must own T and move it into an Option to call with it
  * if returned, the owner must store it as Option<T> internally in order to return it
* Performance is subject to compiler optimization, but at the basics, `&Option<T>` points to memory that has `presence` flag + value, whereas `Option<&T>` by specification is always optimized to a single pointer.
@nyurik nyurik deleted the ref-option-sig branch September 28, 2024 23:48
bors added a commit that referenced this pull request Sep 29, 2024
Convert `&Option<T>` to `Option<&T>`

Run `ref_option` (#13336) on the Clippy's own code, quiet a few hits. Per mentioned video, this may actually improve performance as well.  Switch lint to `pedantic`

changelog [`ref_option`]: upgrade lint to `pedantic`
bors added a commit that referenced this pull request Sep 29, 2024
Convert `&Option<T>` to `Option<&T>`

Run `ref_option` (#13336) on the Clippy's own code, quiet a few hits. Per mentioned video, this may actually improve performance as well.  Switch lint to `pedantic`

----

changelog: [`ref_option`]: upgrade lint to `pedantic`
tgross35 added a commit to tgross35/rust that referenced this pull request Oct 12, 2024
Migrate lib's `&Option<T>` into `Option<&T>`

Trying out my new lint rust-lang/rust-clippy#13336 - according to the [video](https://www.youtube.com/watch?v=6c7pZYP_iIE), this could lead to some performance and memory optimizations.

Basic thoughts expressed in the video that seem to make sense:
* `&Option<T>` in an API breaks encapsulation:
  * caller must own T and move it into an Option to call with it
  * if returned, the owner must store it as Option<T> internally in order to return it
* Performance is subject to compiler optimization, but at the basics, `&Option<T>` points to memory that has `presence` flag + value, whereas `Option<&T>` by specification is always optimized to a single pointer.
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Oct 12, 2024
Rollup merge of rust-lang#130962 - nyurik:opts-libs, r=cuviper

Migrate lib's `&Option<T>` into `Option<&T>`

Trying out my new lint rust-lang/rust-clippy#13336 - according to the [video](https://www.youtube.com/watch?v=6c7pZYP_iIE), this could lead to some performance and memory optimizations.

Basic thoughts expressed in the video that seem to make sense:
* `&Option<T>` in an API breaks encapsulation:
  * caller must own T and move it into an Option to call with it
  * if returned, the owner must store it as Option<T> internally in order to return it
* Performance is subject to compiler optimization, but at the basics, `&Option<T>` points to memory that has `presence` flag + value, whereas `Option<&T>` by specification is always optimized to a single pointer.
RalfJung pushed a commit to RalfJung/miri that referenced this pull request Oct 14, 2024
Migrate lib's `&Option<T>` into `Option<&T>`

Trying out my new lint rust-lang/rust-clippy#13336 - according to the [video](https://www.youtube.com/watch?v=6c7pZYP_iIE), this could lead to some performance and memory optimizations.

Basic thoughts expressed in the video that seem to make sense:
* `&Option<T>` in an API breaks encapsulation:
  * caller must own T and move it into an Option to call with it
  * if returned, the owner must store it as Option<T> internally in order to return it
* Performance is subject to compiler optimization, but at the basics, `&Option<T>` points to memory that has `presence` flag + value, whereas `Option<&T>` by specification is always optimized to a single pointer.
bors added a commit that referenced this pull request Oct 20, 2024
Fix version for `ref_option`

Close #13566
As mentioned at #13566 `ref_option` [was merged](#13336) after 1.82.0 so it wasn't include in this version

changelog: none
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties
Projects
None yet
Development

Successfully merging this pull request may close these issues.

✅️Option<&T>; ❌️&Option<T>
7 participants