std::fs::rename sometimes fails on Windows due to missing FILE_RENAME_POSIX_SEMANTICS
#123985
Labels
C-discussion
Category: Discussion or questions that doesn't represent real issues.
O-windows
Operating system: Windows
T-libs
Relevant to the library team, which will review and decide on the PR/issue.
Comments
rustbot
added
the
needs-triage
Noratrieb
added
O-windows
|
Makes sense to me. Note though that we generally prefer not to use |
DianQK
added a commit
to DianQK/rust
that referenced
this issue
Dec 21, 2024
…antics, r=ChrisDenton Win: Use POSIX rename semantics for `std::fs::rename` if available Windows 10 1601 introduced `FileRenameInfoEx` as well as `FILE_RENAME_FLAG_POSIX_SEMANTICS`, allowing for atomic renaming and renaming if the target file is has already been opened with `FILE_SHARE_DELETE`, in which case the file gets renamed on disk while the open file handle still refers to the old file, just like in POSIX. This resolves rust-lang#123985, where atomic renaming proved difficult to impossible due to race conditions. If `FileRenameInfoEx` isn't available due to missing support from the underlying filesystem or missing OS support, the renaming is retried with `FileRenameInfo`, which matches the behavior of `MoveFileEx`. This PR also manually replicates parts of `MoveFileEx`'s internal logic, as reverse-engineered from the disassembly: If the source file is a reparse point and said reparse point is a mount point, the mount point itself gets renamed; otherwise the reparse point is resolved and the result renamed. Notes: - Currently, the `win7` target doesn't bother with `FileRenameInfoEx` at all; it's probably desirable to remove that special casing and try `FileRenameInfoEx` anyway if it doesn't exist, in case the binary is run on newer OS versions. Fixes rust-lang#123985
GuillaumeGomez
added a commit
to GuillaumeGomez/rust
that referenced
this issue
Dec 21, 2024
…antics, r=ChrisDenton Win: Use POSIX rename semantics for `std::fs::rename` if available Windows 10 1601 introduced `FileRenameInfoEx` as well as `FILE_RENAME_FLAG_POSIX_SEMANTICS`, allowing for atomic renaming and renaming if the target file is has already been opened with `FILE_SHARE_DELETE`, in which case the file gets renamed on disk while the open file handle still refers to the old file, just like in POSIX. This resolves rust-lang#123985, where atomic renaming proved difficult to impossible due to race conditions. If `FileRenameInfoEx` isn't available due to missing support from the underlying filesystem or missing OS support, the renaming is retried with `FileRenameInfo`, which matches the behavior of `MoveFileEx`. This PR also manually replicates parts of `MoveFileEx`'s internal logic, as reverse-engineered from the disassembly: If the source file is a reparse point and said reparse point is a mount point, the mount point itself gets renamed; otherwise the reparse point is resolved and the result renamed. Notes: - Currently, the `win7` target doesn't bother with `FileRenameInfoEx` at all; it's probably desirable to remove that special casing and try `FileRenameInfoEx` anyway if it doesn't exist, in case the binary is run on newer OS versions. Fixes rust-lang#123985
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this issue
Dec 22, 2024
Rollup merge of rust-lang#131072 - Fulgen301:windows-rename-posix-semantics, r=ChrisDenton Win: Use POSIX rename semantics for `std::fs::rename` if available Windows 10 1601 introduced `FileRenameInfoEx` as well as `FILE_RENAME_FLAG_POSIX_SEMANTICS`, allowing for atomic renaming and renaming if the target file is has already been opened with `FILE_SHARE_DELETE`, in which case the file gets renamed on disk while the open file handle still refers to the old file, just like in POSIX. This resolves rust-lang#123985, where atomic renaming proved difficult to impossible due to race conditions. If `FileRenameInfoEx` isn't available due to missing support from the underlying filesystem or missing OS support, the renaming is retried with `FileRenameInfo`, which matches the behavior of `MoveFileEx`. This PR also manually replicates parts of `MoveFileEx`'s internal logic, as reverse-engineered from the disassembly: If the source file is a reparse point and said reparse point is a mount point, the mount point itself gets renamed; otherwise the reparse point is resolved and the result renamed. Notes: - Currently, the `win7` target doesn't bother with `FileRenameInfoEx` at all; it's probably desirable to remove that special casing and try `FileRenameInfoEx` anyway if it doesn't exist, in case the binary is run on newer OS versions. Fixes rust-lang#123985
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When opening a file on Windows, the default flags indicate sharing for read, write and delete. If you attempt to atomically rename a file over another file while it is being read, however, this results in an access-denied error despite the expectation that those access flags would allow for this to occur.
This makes all attempts at writing atomic-file-replace operations extremely racy and difficult to get right on Windows, while working as expected on other platforms.
By default, Rust's stdlib rename should make use of the new
FILE_RENAME_POSIX_SEMANTICSwhen available. This will allow a file to stay open while the underlying file is atomically replaced with new contents.This surfaced recently in Deno as a failure in the
DiskCacheimplementation on Windows (https://github.com/denoland/deno/blob/main/cli/cache/disk_cache.rs#L123). If a file is being read at the same time an atomic write happens on another thread, the write operation fails with an "access denied" error, but only on Windows.Justification
There is precedent for using POSIX semantics on Windows by default: for example, recursive directory deletes use this where available, and junction points (implemented in #121709) are created with POSIX semantics.
In addition, users can opt out of POSIX rename semantics by opening files without the
DELETEsharing mode.API Background
As of Windows 10 1709 and later, a new
FILE_RENAME_POSIX_SEMANTICSflag is available for theFileRenameInformationExhttps://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-ntsetinformationfile
Related bugs in other frameworks
https://bugs.python.org/msg344575
The text was updated successfully, but these errors were encountered: