Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an append_link() method to handle long link targets #273

Merged
merged 1 commit into from
Nov 19, 2021
Merged

Add an append_link() method to handle long link targets #273

merged 1 commit into from
Nov 19, 2021

Conversation

cgwalters
Copy link
Collaborator

We should support appending long symlink targets, because this
occurs in real world filesystems. In my case, RPM set up
.build-id symlinks which can get long.

Add an append_link() method following the precendent of
append_path() - we're just supporting two potentially
long filenames.

As a side benefit, we can just do the std::io::empty() dance
internally and not require the caller to specify it.

The addition of special case append() methods is unfortunate,
because the header API methods are then really an attractive nuisance.
We should potentially consider deprecating them.

Closes: #192

@cgwalters
Copy link
Collaborator Author

This passes my unit test, but I'm still doing some more real world testing and comparing this (and my higher level code) with what GNU tar actually does.

@cgwalters cgwalters mentioned this pull request Nov 18, 2021
Closed
@cgwalters
Add an append_link() method to handle long link targets
02a9426 
We should support appending long symlink targets, because this
occurs in real world filesystems.  In my case, RPM set up
`.build-id` symlinks which can get long.

Add an `append_link()` method following the precendent of
`append_path()` - we're just supporting *two* potentially
long filenames.

As a side benefit, we can just do the `std::io::empty()` dance
internally and not require the caller to specify it.

The addition of special case `append()` methods is unfortunate,
because the header API methods are then really an attractive nuisance.
We should potentially consider deprecating them.

Closes: alexcrichton#192
@cgwalters cgwalters marked this pull request as ready for review November 18, 2021 21:52
cgwalters added a commit to cgwalters/ostree-rs-ext that referenced this pull request Nov 18, 2021
@cgwalters
Use new append_link() API to handle long symlinks
c077548 
I hit this when exporting Fedora Silverblue, there are some
long symlinks in there.

Depends: alexcrichton/tar-rs#273
Closes: ostreedev#162
@cgwalters cgwalters mentioned this pull request Nov 18, 2021
Merged
@cgwalters
Copy link
Collaborator Author

OK this works, I tested in concert with ostreedev/ostree-rs-ext#165
What delayed me a bit was debugging ostreedev/ostree-rs-ext#164 because in this code it appears as object (SHA-256) corruption, and it wasn't immediately obvious to me that the link name being truncated was the source.

cgwalters added a commit to cgwalters/ostree-rs-ext that referenced this pull request Nov 18, 2021
@cgwalters
Use new append_link() API to handle long symlinks
c666d1d 
I hit this when exporting Fedora Silverblue, there are some
long symlinks in there.

Depends: alexcrichton/tar-rs#273
Closes: ostreedev#162
@alexcrichton
Copy link
Owner

Thanks!

@alexcrichton alexcrichton merged commit ec5edf1 into alexcrichton:master Nov 19, 2021
@cgwalters cgwalters mentioned this pull request Dec 10, 2021
Merged
cgwalters added a commit to cgwalters/ostree-rs-ext that referenced this pull request Dec 14, 2021
@cgwalters
Use new append_link() API to handle long symlinks
39d4b93 
I hit this when exporting Fedora Silverblue, there are some
long symlinks in there.

Depends: alexcrichton/tar-rs#273
Closes: ostreedev#162
@jonhoo jonhoo mentioned this pull request Mar 24, 2023
Merged
jonhoo pushed a commit to jonhoo/rust that referenced this pull request Mar 25, 2023
Allow long link names in tar files
4d55010 
Without this, users trying to run `x.py dist` under a sufficiently long
path run into problems when we build the resulting tarballs due to
length limits in the original tar spec. The error looks like:

        Finished release [optimized + debuginfo] target(s) in 0.34s
    Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-musl)
    Building stage0 tool rust-installer (x86_64-unknown-linux-gnu)
        Finished release [optimized] target(s) in 0.35s
    Dist rust-std-1.67.1-x86_64-unknown-linux-musl
    Error: failed to generate installer

    Caused by:
        0: failed to tar file '/home/AAAAAAAAAAAAAA/BBBBBB/CCCC/DDD/EEEEE/FFFFFFFFFFFF/GGGGGGGGGGGGGGGG/HHHHHHHHHH/IIIIIIIIIIIIIII/JJJJJ/KKKKKKK/src/build/tmp/tarball/rust-std/x86_64-unknown-linux-musl/rust-std-1.67.1-x86_64-unknown-linux-musl/rust-std-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/libc.a'
        1: provided value is too long when setting link name for
    Build completed unsuccessfully in 0:00:03

The fix is to make use of the widely-supported GNU tar extensions which
lift this restriction. Switching to [`tar::Builder::append_link`] takes
care of that for us. See also alexcrichton/tar-rs#273.

[`tar::Builder::append_link`]: https://docs.rs/tar/0.4.38/tar/struct.Builder.html#method.append_link
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 28, 2023
@bors
Auto merge of rust-lang#109577 - jonhoo:long-tar-names, r=Mark-Simula…
cbc064b 
…crum

[rust-installer] Allow long link names in tar files

Without this, users trying to run `x.py dist` under a sufficiently long path run into problems when we build the resulting tarballs due to length limits in the original tar spec. The error looks like:

        Finished release [optimized + debuginfo] target(s) in 0.34s
    Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-musl)
    Building stage0 tool rust-installer (x86_64-unknown-linux-gnu)
        Finished release [optimized] target(s) in 0.35s
    Dist rust-std-1.67.1-x86_64-unknown-linux-musl
    Error: failed to generate installer

    Caused by:
        0: failed to tar file '/home/AAAAAAAAAAAAAA/BBBBBB/CCCC/DDD/EEEEE/FFFFFFFFFFFF/GGGGGGGGGGGGGGGG/HHHHHHHHHH/IIIIIIIIIIIIIII/JJJJJ/KKKKKKK/src/build/tmp/tarball/rust-std/x86_64-unknown-linux-musl/rust-std-1.67.1-x86_64-unknown-linux-musl/rust-std-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/libc.a'
        1: provided value is too long when setting link name for
    Build completed unsuccessfully in 0:00:03

The fix is to make use of the widely-supported GNU tar extensions which lift this restriction. Switching to [`tar::Builder::append_link`] takes care of that for us. See also alexcrichton/tar-rs#273.

[`tar::Builder::append_link`]: https://docs.rs/tar/0.4.38/tar/struct.Builder.html#method.append_link
oli-obk pushed a commit to oli-obk/miri that referenced this pull request Apr 4, 2023
@bors
Auto merge of #109577 - jonhoo:long-tar-names, r=Mark-Simulacrum
8a2eba1 
[rust-installer] Allow long link names in tar files

Without this, users trying to run `x.py dist` under a sufficiently long path run into problems when we build the resulting tarballs due to length limits in the original tar spec. The error looks like:

        Finished release [optimized + debuginfo] target(s) in 0.34s
    Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-musl)
    Building stage0 tool rust-installer (x86_64-unknown-linux-gnu)
        Finished release [optimized] target(s) in 0.35s
    Dist rust-std-1.67.1-x86_64-unknown-linux-musl
    Error: failed to generate installer

    Caused by:
        0: failed to tar file '/home/AAAAAAAAAAAAAA/BBBBBB/CCCC/DDD/EEEEE/FFFFFFFFFFFF/GGGGGGGGGGGGGGGG/HHHHHHHHHH/IIIIIIIIIIIIIII/JJJJJ/KKKKKKK/src/build/tmp/tarball/rust-std/x86_64-unknown-linux-musl/rust-std-1.67.1-x86_64-unknown-linux-musl/rust-std-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/libc.a'
        1: provided value is too long when setting link name for
    Build completed unsuccessfully in 0:00:03

The fix is to make use of the widely-supported GNU tar extensions which lift this restriction. Switching to [`tar::Builder::append_link`] takes care of that for us. See also alexcrichton/tar-rs#273.

[`tar::Builder::append_link`]: https://docs.rs/tar/0.4.38/tar/struct.Builder.html#method.append_link
@cgwalters cgwalters mentioned this pull request Sep 25, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

adding a long hardlink without _fs APIs
2 participants
@cgwalters @alexcrichton