tar with symbolic links

[tinysun212]

I could create a symbolic links by ln -s.

root@localhost:~# ls -l kkk
total 0
-rw-rw-rw- 1 root root 14 Apr  7 04:53 atest
lrwxrwxrwx 1 root root  5 Apr  7 04:54 btest -> atest

But I can't archive with the tar.

root@localhost:~# tar cvf kkk.tar kkk
kkk/
kkk/atest
tar: kkk/btest: Cannot readlink: Invalid argument
tar: Exiting with failure status due to previous errors

In reverse, the symbolic links in a tar file which built on Linux can't be extracted.

[jokeyrhyme]

Related to #3 and/or #6 ?

[tinysun212]

The error messages are different.

In my case, ln -s is working, but not compatible with tar.

Here is the reverse operation output. ( extracting symlinks-containing tar whilch is built on real Linux )

root@localhost:~# tar tvf /mnt/c/temp/kkk.tar
drwxr-xr-x root/root         0 2016-04-07 05:25 kkk/
lrwxrwxrwx root/root         0 2016-04-07 05:25 kkk/btest -> atest
-rw-r--r-- root/root        12 2016-04-07 05:24 kkk/atest
root@localhost:~# tar xvf /mnt/c/temp/kkk.tar
kkk/
kkk/btest
tar: kkk/btest: Cannot create symlink to ‘atest’: Invalid argument
kkk/atest
tar: Exiting with failure status due to previous errors

[hfaran]

I can echo this: I tried to install rvm from get.rvm.io and the script crashes and burns because of the tar symlink incompatibility.

[mphaney]

I also can confirm this. (Running on 64-bit, on the off chance that matters). Can create symlinks just fine, cannot tar them or untar packages which contain them. As an interesting consequence, apt packages which contain symlinks (Java) also cannot be installed.

[christoh]

I was trying to copy files from a real linux box using tar. Also running into the symlink issue. As a workaround I was able to copy the files using bsdtar which creates symlinks correctly.

[ljharb]

This is affecting http://nvm.sh installs via windows as well. nvm-sh/nvm#1054

[jbaribeault]

This might relate to is #44 as well...for me anyway, the move won't work on tarballs/zips from CPAN which contain soft links.

[tomgrek]

+1.
Can't install Meteor due to this issue.
Copying the files from a Linux VM sort of works if you re-create the tarball with the h option (i.e. tar cvfh).

[filipeaguiar]

As @ljharb said: nvm doesn't install properly. And none of the tools that uses symlinks with tar works right now.

[ljharb]

To clarify, this is affecting nvm's ability to install node. nvm itself installs fine.

[julihermes]

because of this error i am not able to install rbenv

[SvenGroot]

The tar extraction problem with symlinks is caused by the missing symlinkat syscall (symlink was supported but not the at variant). This bug has been fixed internally.

Similarly, I believe creating a tar file with symlinks fails because readlinkat is missing.

[boulund]

Just as a note if someone else stumbles upon this: this also affects installation of Miniconda using Continuum Analytics Miniconda installation shell script that extracts a tar archive containing symlinks.

You get errors like this:

tar: bin/2to3: Cannot create symlink to ‘2to3-3.5’: Invalid argument

[davidkrider]

Having the same trouble installing RVM. I can download the zip version and install manually, but I get a warning at the end of the install:

* WARNING: you are missing '/dev/fd' it is important for shell scripting used by RVM,
You can fix it on most linux systems with:
ln -sf /proc/self/fd /dev/fd

/proc/self/fd does exist, but following those instructions gives me:

root@localhost:~# ln -s /proc/self/fd /dev/fd
ln: failed to create symbolic link ‘/dev/fd’: Operation not permitted

This is a different error. I make a note here wondering if the (internal) fix would address this, or if we're into some unsupported behavior in the /proc filesystem now.

I got RVM installed manually, but, of course, it wants to install rubies from tar-ed sources, which loops me back around to the original problem, and I'm tired of fighting it.

[SvenGroot]

DunKirk, that error is due to some limitations of our /dev filesystem that prevent you from creating a link there. We're working to address this. This is separate from the symlinkat error.

[MadcowD]

@SvenGroot thank you much!

[jessicah]

I have an example where openat() on a symlink returns no such file or directory despite the symlink existing but not the target. Does the symlinkat fix also apply here?

# e.g. openat("libsolv.so") fails with no such file or directory
root@localhost:# ls -l
total 0
lwrxwrxwrx 1 root root 12 Apr 10 23:33 libsolv.so -> libsolv.so.1
root@localhost:#

Example: https://github.com/haiku/haiku/blob/master/src/bin/package/command_extract.cpp#L552

[sunilmut]

Thanks for the feedback. Support for both symlinkat and readlinkat has been added to the code, but the build has not made it to flighting yet.

[EdoaLive]

Is there some kind of workaround for this?

[perlun]

Thanks for the feedback. Support for both symlinkat and readlinkat has been added to the code, but the build has not made it to flighting yet.

@sunilmut - any idea about when the new build will be released? Is there a release plan hidden somewhere? 😉

[sunilmut]

@perlun - Sorry, that information is not something we can provide reliably at this time. Though, we understand the need for better tracking and availability of information, and are looking into how we can improve that on our side. So, please bear with us until then.

[tinysun212]

It works finally!

root@DESKTOP-V9QH45Q:~# ls -lR kkk
kkk:
total 0
-rw-rw-r-- 1 1000 1000 11 Apr 23 22:40 atest
lrwxrwxrwx 1 1000 1000  5 Apr 23 22:40 btest -> atest
root@DESKTOP-V9QH45Q:~# tar cvf kkk.tar kkk
kkk/
kkk/atest
kkk/btest
root@DESKTOP-V9QH45Q:~# tar tvf kkk.tar
drwxrwxr-x 1000/1000         0 2016-04-23 22:40 kkk/
-rw-rw-r-- 1000/1000        11 2016-04-23 22:40 kkk/atest
lrwxrwxrwx 1000/1000         0 2016-04-23 22:40 kkk/btest -> atest

It seems that the problem I presented in beginning is solved. If there is no objection, I will close this issue in this weekend.

Thanks you WSL team, that's totally awesome!

[julihermes]

its fixed in the new build?

[mrdrmmn]

This doesn't appear to be completely fixed or has been reverted. I am running build 14342.rs1_release.160506-1708.

This can be tested in the following way:

JUPITER:/tmp$ ls
JUPITER:/tmp$ echo FOO > tmp1
JUPITER:/tmp$ ln -s tmp1 tmp2
JUPITER:/tmp$ grep -r FOO ./
./tmp1:FOO
grep: ./tmp2: Invalid argument

[tinysun212]

Good detection!

This thread is for the combination of symlink and 'tar', not for all things related to symlink or tar.
Please create new issue with title like 'grep with symbolic links'.

[benhillis]

@therevmj @tinysun212 I tried this repro internally and verified that this is now working. The fix will be making its way to a windows insider build soon.

[perlun]

Sorry to be a bit OT, but how do you push your insider machine to download the latest build? Mine is still running the "original" 160402 (or so) release, meaning I don't have the symlink fixes etc...

[hfaran]

@perlun http://www.cnet.com/how-to/change-to-the-fast-ring-for-more-frequent-windows-10-preview-updates/

[perlun]

@hfaran - thanks. The problem is I am already on the fast ring, but the updates don't seem to be coming. And I cannot find a way to manually force them to come. Any suggestions?