WSL 2's drvfs can't list directories on many third-party filesystems

[mrnerdhair]

Environment

Windows build number: 10.0.19041.508
Your Distribution version: Ubuntu 20.04
Whether the issue is on WSL 2 and/or WSL 1: WSL 2

Steps to reproduce

Using a development version of Dokan:

mkdir C:\test
echo bar>C:\test\foo.txt
mirror.exe /r C:\test /l X

Then, on WSL 2:

WSL logs:

Expected behavior

ls -al /mnt/x lists bar.txt

Actual behavior

ls -al /mnt/x returns ls: reading directory '/mnt/x': Function not implemented

The Actual Problem

Many third-party filesystems (Dokan and StableBit DrivePool, for example) don't work with WSL 2 because they don't return the (technically) correct error code when asked for an unsupported FileInformationClass in IRP_MN_QUERY_DIRECTORY requests.

Why

WSL2's implementation of drvfs connects to a 9P2000.L (Plan 9) file server over a Hyper-V socket. The Plan 9 server code is in vp9fs.dll, and is hosted in a dllhost.exe process which runs with the token of the user that launched the WSL distro. It invokes invokes ZwQueryDirectoryFile, asking for the FileIdExtdDirectoryInformation information class first. A fallback mechanism exists which will retry the request with increasingly general information classes (namely FileIdFullDirectoryInformation, FileIdBothDirectoryInformation, FileFullDirectoryInformation, and FileDirectoryInformation, in that order). However, this fallback only occurs if the request completes with STATUS_SUCCESS and sets the IO_STATUS_BLOCK Status parameter to STATUS_INVALID_INFO_CLASS, STATUS_INVALID_PARAMETER, or STATUS_NOT_SUPPORTED.

As far as standards go, [MS-FSCC] sections 2.4 and 2.5 seem to indicate that STATUS_INVALID_PARAMETER is the correct return code for functions which are implemented but which are asked to query or set an unimplemented information class. However, this is quite an esoteric requirement, and many third-party filesystems return other codes; Dokan returns STATUS_NOT_IMPLEMENTED, and DrivePool returns STATUS_INTERNAL_ERROR, but other filesystems may use other values.

The Solution

The relevant fallback code in the LxUtilpDirectoryEnumeratorFillBuffer() function in vp9fs.dll should be updated to attempt fallback in response to a more general set of error codes; also consider matching against a set of explicitly excluded codes instead of a set of included ones.

Third-party filesystem drivers should be updated to return the canonical STATUS_INVALID_PARAMETER value in response to requests for unsupported file information classes.

The Terrible Example

A quick-n-dirty patch I wrote in Cheat Engine will edit the code in the copy of vp9fs.dll loaded by the dllhost.exe instance serving as the WSL 2 Plan 9 file server so that it also works with STATUS_NOT_IMPLEMENTED and STATUS_INTERNAL_ERROR. It may be instructive to engineers trying to fix the issue, but no one should actually use this unless you understand A) what it does and B) how little sympathy you'll get if you screw up something trying to hack Windows with what's essentially a game trainer.

References

• StableBit DrivePool thread and tracking issue
• More in-depth analysis here and here
• Dokan issue: Can't list directories in WSL2 dokan-dev/dokany#932
• Cryptomator issue: Unable to access Cryptomator-Container from Windows10/WSL  cryptomator/cryptomator#1033
• Related WSL issues: Mount Google Drive File System to WSL #2999, Still Unable to Use NFS/Fuse? #1962, mount.drvfs fails to mount virtualbox shares #2896, probably some more too

[mrnerdhair]

This isn't technically the same issue -- but as long as we're collecting changes that should go into vp9fs.dll's LxUtilNtStatusToLxError(), let me recommend translating STATUS_EAS_NOT_SUPPORTED into EOPNOTSUPP, and perhaps STATUS_INVALID_DEVICE_REQUEST as well. This is because while Linux doesn't require a filesystem to support extended attributes, it does always check to see if security.capabilities is set whenever running execve() so that it can set or clear any appropriate capability flags the new process should or shouldn't have. It's fine with an EOPNOTSUPP response (or with ENODATA, indicating that security.capabilities is missing or blank) but not many of the NTSTATUSes Plan 9 might encounter will translate to those codes. The generic response is EINVAL, which fails out of the security check and prevents anything from ever running at all off the affected volume.

EOPNOTSUPP is reported automatically, and not as a result of a translated NTSTATUS, if an FS reports FILE_SUPPORTS_EXTENDED_ATTRIBUTES -- but I've seen a filesystem driver that always reports that, even if it only actually supports EAs in a certain set of circumstances determined later at runtime. Even if they were returning the correct STATUS_EAS_NOT_SUPPORTED (which they're not at the moment, they're returning STATUS_INVALID_DEVICE_REQUEST) it wouldn't work -- because the "default" translation from LxUtilNtStatusToLxError() is EINVAL not EOPNOTSUPP.

Here's a dump I've created of every possible NTSTATUS that doesn't map to one of the "fallback" codes. If it's not in this list, it maps all values less than 0x80000000 to 0 and everything else to EINVAL.

[mrnerdhair]

Proposed vp9fs!LxUtilNtStatusToLxError edits:

STATUS_INVALID_INFO_CLASS       0xC0000003  EINVAL      22
STATUS_INFO_LENGTH_MISMATCH     0xC0000004  EOVERFLOW   75
STATUS_ACCESS_VIOLATION         0xC0000005  EFAULT      14
STATUS_PAGEFILE_QUOTA           0xC0000007  ENOMEM      12
STATUS_NO_SUCH_DEVICE           0xC000000E  ENODEV      19
STATUS_NO_SUCH_FILE             0xC000000F  ENOENT      2
STATUS_INVALID_DEVICE_REQUEST   0xC0000010  EOPNOTSUPP  95
STATUS_NOT_SUPPORTED            0xC00000BB  EOPNOTSUPP  95
STATUS_EAS_NOT_SUPPORTED        0xC000004F  EOPNOTSUPP  95

[benhillis]

@reidrankin - Thank you for filing this excellent issue. It does look like there are some deficiencies in our current error mapping, I'll take a look at this.

[esumii]

Not sure if this is the same problem, but I am experiencing the following similar issue with WSL2 on Windows 10 version 20H2 (OS build 19042.630):

sumii@LAPTOP-xxxxxxxx:~$ export LANG=C
sumii@LAPTOP-xxxxxxxx:~$ touch /mnt/c/Users/eijir/Desktop/Dropbox/zzz
sumii@LAPTOP-xxxxxxxx:~$ ls -l /mnt/c/Users/eijir/Desktop/Dropbox/zzz
-rwxrwxrwx 1 sumii sumii 0 Nov 23 10:28 /mnt/c/Users/eijir/Desktop/Dropbox/zzz
sumii@LAPTOP-xxxxxxxx:~$ ls -l /mnt/c/Users/eijir/Desktop/Dropbox/zz*
ls: cannot access '/mnt/c/Users/eijir/Desktop/Dropbox/zz*': No such file or dire
ctory
sumii@LAPTOP-xxxxxxxx:~$ ls -l /mnt/c/Users/eijir/Desktop/Dropbox/ | grep '^zz'
ls: reading directory '/mnt/c/Users/eijir/Desktop/Dropbox/': Input/output error
sumii@LAPTOP-xxxxxxxx:~$ ls -al /mnt/c/Users/eijir/Desktop/Dropbox/ | grep '^zz'
ls: reading directory '/mnt/c/Users/eijir/Desktop/Dropbox/': Input/output error
sumii@LAPTOP-xxxxxxxx:~$ uname -a
Linux LAPTOP-xxxxxxxx 4.19.128-microsoft-standard #1 SMP Tue Jun 23 12:58:10 UTC
 2020 x86_64 GNU/Linux
sumii@LAPTOP-xxxxxxxx:~$ cat /etc/debian_version
10.5

[microsoft-github-policy-service]

This issue has been automatically closed since it has not had any activity for the past year. If you're still experiencing this issue please re-file this as a new issue or feature request.

Thank you!