the write_at function from std::os::unix::prelude::FileExt doesn't work as described

[nnayo]

I'm pretty new to rust and I'm working on a small project where I need to write into a file some data at various offsets, so the write_at function seems to perfectly fit my need.

So I tried this code (with is the provided example):

use std::fs::File;
use std::io;
use std::os::unix::prelude::FileExt;

fn main() -> io::Result<()> {
    let file = File::open("foo.txt")?;

    // We now write at the offset 10.
    file.write_at(b"sushi", 10)?;
    Ok(())
}

and I do cargo run

I expected to see this happen: a foo.txt in the current directory with a size of 15 bytes

Instead, this happened: Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }

So, I modifed the filename to /tmp/foo.txt (in order to have an absolute path).
I got the same error.

So, I created the file with touch /tmp/foo.txt
Then I got the error: Error: Os { code: 9, kind: Uncategorized, message: "Bad file descriptor" }

So, I add the line println!("{:?}", file); to get some info on the file
Then I got: File { fd: 3, path: "/tmp/foo.txt", read: true, write: false }

So, the file is open read-only, I searched and modified the code to: let file = File::options().append(true).open("/tmp/foo.txt")?;
Then I got: File { fd: 3, path: "/tmp/foo.txt", read: false, write: true } and finally no error (on stdout),

But the file is only 5-byte long whereas I expected it to be 15-byte long.

And checking foo.txt content with od -t x1 -a /tmp/foo.txt gives:

0000000  73  75  73  68  69
          s   u   s   h   i
0000005

The final code is

use std::fs::File;
use std::io;
use std::os::unix::prelude::FileExt;

fn main() -> io::Result<()> {
    let file = File::options().append(true).open("/tmp/foo.txt")?;
    println!("{:?}", file);
              
    // We now write at the offset 10.
    file.write_at(b"sushi", 10)?;
    Ok(())
}

and you need to manually create the file before running the binary.

Meta

rustc --version --verbose:

rustc 1.70.0 (90c541806 2023-05-31)
binary: rustc
commit-hash: 90c541806f23a127002de5b4038be731ba1458ca
commit-date: 2023-05-31
host: x86_64-unknown-linux-gnu
release: 1.70.0
LLVM version: 16.0.2

Backtrace

<backtrace>

[workingjubilee]

Wow those docs are terrible, how did that happen?

We can immediately patch the docs to code that actually would work to write to a file, but we need to define what "extended appropriately" means. i.e. Is this intended behavior?

But perhaps more importantly, we need to figure out why the hell we have doctest examples that don't actually work and what can we do to fix that?

[ChrisDenton]

Yeah it's weird that we have a non-working example. That should be fixed asap. It looks like it was copy/pasted from read_at but not updated to use File::create or something like that.

I'll mark this as help wanted. Fixing that should not be controversial.

[ChrisDenton]

The issue with the final code in the OP is that it opens the file in append only mode. The docs do not cover this case at all so I can't say what the intended behaviour is. I guess write_at is really incompatible with append only because it needs to have some knowledge of the real file position in order to work as expected.

[workingjubilee]

( for the doc fix part, anyways. )

[workingjubilee]

It does explain that returning short writes is valid, so I guess the behavior exhibited in the append-only case presented here is technically acceptable behavior. 🫠

[saethlin]

On Linux, write_at just calls pwrite64 which documents this:

BUGS
       POSIX requires that opening a file with the O_APPEND flag should have no effect on the location at which pwrite()
       writes data. However, on Linux, if a file is opened with O_APPEND, pwrite() appends data to the end of the file, re‐
       gardless of the value of offset.

[joboet]

On Linux, write_at just calls pwrite64 which documents this:

BUGS
       POSIX requires that opening a file with the O_APPEND flag should have no effect on the location at which pwrite()
       writes data. However, on Linux, if a file is opened with O_APPEND, pwrite() appends data to the end of the file, re‐
       gardless of the value of offset.

There was a kernel proposal to fix this via a flag on pwritev2, but that didn't get merged...

[Amanieu]

We discussed this in the libs-api meeting. It seems that there are 2 actionable items here, both in the documentation:

• The example in write_at needs to be fixed to use File::create (write mode) instead of File::open (read mode).
• The documentation for write_at should link to https://man7.org/linux/man-pages/man2/pwrite.2.html#BUGS to indicate that, on some platforms, files opened in append mode ignore the position passed in and always append to the end of the file.

[darklyspaced]

@rustbot claim