Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unsafe improvements #6551

Merged
merged 5 commits into from
Oct 21, 2024
Merged

unsafe improvements #6551

merged 5 commits into from
Oct 21, 2024

Conversation

ssbr
Copy link
Contributor

@ssbr ssbr commented Oct 11, 2024

Which issue does this PR close?

N/A

Rationale for this change

I discovered these while reviewing the unsafe code of arrow-rs -- this mostly just removes some instances of potentially-unsafe code, replacing them with safe equivalents. That makes it easier to review, at (so far as I know) no performance cost.

What changes are included in this PR?

  • Reuse unsafe code in the OffsetBufferBuilder.
  • Use a u64 instead of an always-initialized MaybeUninit<u64>.
  • More precisely specify safety for read_bytes_to_u64.

Are there any user-facing changes?

No.

The MaybeUninit doesn't ever hold any uninitialized bytes, but if it _did_,
then this would have been UB! Because the write can write fewer than 8
bytes, it would leave the remaining bytes uninitialized.

Since the `MaybeUninit<u64>` is never uninitialized, we can remove
the use of `MaybeUninit`, and just use `u64`. Then it becomes clear
that the only thing for the unsafe block is the `copy_nonoverlapping`.
@github-actions github-actions bot added the arrow Changes to the arrow crate label Oct 11, 2024
@jp0317
Copy link
Contributor

jp0317 commented Oct 11, 2024

thanks @ssbr !

@findepi
Copy link
Member

findepi commented Oct 12, 2024

@ssbr can you please check the build results?

The alternative would be to actually convert the `debug_assert` to
an `assert`, or clamp to 8, but both of these could conceivably
have a performance impact, so in the interest of being a pure
win, with no downsides, that change is left to a future editor.
@ssbr
Copy link
Contributor Author

ssbr commented Oct 14, 2024

@ssbr can you please check the build results?

I struggled to find them! Sorry, not used to github. The only way I could find was to click into checks, click into the commit history, find the one with the x, click that, and then click to open every check until I found the failures. Is there an easier way? I found two:

  • Tests failing -- those tests are also failing at head, so I ignored this.
  • cargo fmt. I ran cargo fmt. Once again, not used to git or github, so I just made it a commit on top 😛. (At work, this would be cargo fmt; hg absorb.)

I hear git has an `absorb` extension, but I don't have it. :(
@alamb
Copy link
Contributor

alamb commented Oct 15, 2024

https://github.com/apache/arrow-rs/actions/runs/11334312589/job/31523941790?pr=6551 does not appear to be related to this PR (it happens in main as well)

@kazuyukitanimura
Copy link
Contributor

Not sure if the CI failure is real or transitional

@ssbr
Copy link
Contributor Author

ssbr commented Oct 17, 2024

Not sure if the CI failure is real or transitional

It's also present without the change, so real, but unrelated.

Copy link
Contributor

@kazuyukitanimura kazuyukitanimura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
It seems the arrow flight CI has been flaky

@tustvold tustvold merged commit 04d0ebe into apache:master Oct 21, 2024
25 of 26 checks passed
@alamb
Copy link
Contributor

alamb commented Oct 21, 2024

Not sure if the CI failure is real or transitional

It's also present without the change, so real, but unrelated.

@itsjunetime fixed it in #6585

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants