path/filepath: filepath.Walk does not work on mounted volumes in Windows containers

[buddhike]

What version of Go are you using (go version)?

1.10

Does this issue reproduce with the latest release?

Not sure

What operating system and processor architecture are you using (go env)?

windows 386

What did you do?

• Cross compile the following program from osx GOOS=windows GOARCH=386 go build -o walk.exe main.go (cross compilation does not really matter, the behaviour outlined below is also consistent even if the binary is produced in a windows)

• Created a docker container from microsoft/windowsservercore

docker run --rm -it -v c:\files:c:\files microsoft/windowsservercore

• Ran the program on the mounted volume

walk.exe -path c:\files

package main

import (
	"flag"
	"fmt"
	"os"
	"path/filepath"
)

func main() {
	path := flag.String("path", "", "path to walk")
	flag.Parse()
	fmt.Printf("walking path %v\n", *path)

	err := filepath.Walk(*path, func(path string, info os.FileInfo, err error) error {
		if err != nil {
			panic(err)
		}
		fmt.Printf("visiting %v\n", path)
		return nil
	})

	if err != nil {
		panic(err)
	}
}

What did you expect to see?

List of files and directories in c:\files.

What did you see instead?

Just the root directory

[slrz]

You run the walk program outside of the started container, just on the same directory that is also mounted into the container?

The directory search works ok when using other (non-Go) programs? Can you check what this C program would output?

[buddhike]

Sorry about the lag in my response. Program is run from the default CWD inside the container. However it's in a mounted directory. To be precise:

Step 1 - Launch the container. c:\files has the items we want to traverse. c:\bin has walk.exe.

docker run --rm -it -v c:\files:c:\files -v c:\bin:c:\bin microsoft/windowsservercore

Step 2 - From cwd, run walk.exe

c:\bin\walk.exe -path c:\files

Using other programs such as dir works just fine on mounted directories.
I will update this thread the with the result of the above program over the next couple of days.

Hopefully this clarifies the situation a bit further.

[igorrendulic]

same issue with mounted docker volume on linux. If i run it from VSCode/command line it works as expected (on the same folder which is not mounted). As long as it's mounted through docker volume, it doesn't work. Logged into docker container to check if folder with files exists inside of it and all is there. Might be permission issues of some sorts?

[TBBle]

I've confirmed the same behaviour with go1.15.5 for junction points outside a container. My use-case is for volume mounts that happen to be a WCOW container image.

I mounted a WCOW container image using wclayer, and used the source code in the initial report to test. Walking inside the mount point is fine, but walking the mount point directly or walking a directory above the mount point, the walk stops at the mount.

I suspect the core problem is that filepath.Walk does not follow symbolic links, and to Go, a volume mount point appears to be a symlink back to itself due to calling normaliseLinkPath (leading to GetFinalPathNameByHandle) on it to try and turn it into a DOS-style path; see readlink in https://golang.org/src/os/file_windows.go and isSymlink in https://golang.org/src/os/types_windows.go.

The DOS-style path for a volume mount point is of course the volume mount point. The rest of Go handles this as a symlink to itself, rather than like a bind-mount in Linux, for example. The difference is that if you proceed down a symlink to itself, you get an error. If you follow a volume mount point or bind-mount, you see the contents.

The question of "is this a directory or a symlink" was asked and answered well before this code is executed though, so it's not easily solved without changing the definition of isSymlink and isDir to take this into account.

Unlike a real Symlink, it's not necessarily feasible to call filepath.Walk on the target of a mount point, as it may be a volume reference. So there's no real workaround to using filepath.Walk with the current semantics.

PoC, with openSymlink and readlink copied out of file_windows.go.

package main

import (
	"flag"
	"fmt"
	"os"
	"path/filepath"
	"syscall"

	"github.com/Microsoft/go-winio"
)

func openSymlink(path string) (syscall.Handle, error) {
	p, err := syscall.UTF16PtrFromString(path)
	if err != nil {
		return 0, err
	}
	attrs := uint32(syscall.FILE_FLAG_BACKUP_SEMANTICS)
	// Use FILE_FLAG_OPEN_REPARSE_POINT, otherwise CreateFile will follow symlink.
	// See https://docs.microsoft.com/en-us/windows/desktop/FileIO/symbolic-link-effects-on-file-systems-functions#createfile-and-createfiletransacted
	attrs |= syscall.FILE_FLAG_OPEN_REPARSE_POINT
	h, err := syscall.CreateFile(p, 0, 0, nil, syscall.OPEN_EXISTING, attrs, 0)
	if err != nil {
		return 0, err
	}
	return h, nil
}

func readlink(path string) ([]byte, error) {
	h, err := openSymlink(path)
	if err != nil {
		return nil, err
	}
	defer syscall.CloseHandle(h)

	rdbbuf := make([]byte, syscall.MAXIMUM_REPARSE_DATA_BUFFER_SIZE)
	var bytesReturned uint32
	err = syscall.DeviceIoControl(h, syscall.FSCTL_GET_REPARSE_POINT, nil, 0, &rdbbuf[0], uint32(len(rdbbuf)), &bytesReturned, nil)
	if err != nil {
		return nil, err
	}
	return rdbbuf, nil
}

func main() {
	path := flag.String("path", "", "path to walk")
	flag.Parse()
	fmt.Printf("walking path %v\n", *path)

	err := filepath.Walk(filepath.Clean(*path), func(path string, info os.FileInfo, err error) error {
		if err != nil {
			panic(err)
		}
		fmt.Printf("visiting %v\n", path)
		if info.Mode()&os.ModeSymlink == 0 {
			return nil
		}

		name, err := os.Readlink(path)
		if err != nil {
			panic(err)
		}
		fmt.Printf("\tpoints to %v\n", name)

		buff, err := readlink(name)
		if err != nil {
			panic(err)
		}
		rp, err := winio.DecodeReparsePoint(buff)
		if err != nil {
			panic(err)
		}
		fmt.Printf("\treally points to %v\n", rp.Target)
		return nil
	})

	if err != nil {
		panic(err)
	}
}

Produces:

> walk.exe -path C:\Users\paulh\Documents\BuildKit\hct\
walking path C:\Users\paulh\Documents\BuildKit\test\
visiting C:\Users\paulh\Documents\BuildKit\test
visiting C:\Users\paulh\Documents\BuildKit\test\mount
        points to C:\Users\paulh\Documents\BuildKit\test\mount\
        really points to \??\Volume{98a20d7d-b960-4880-9452-2563efe0c400}\

Edit: Thinking about this a little, I don't understand why Go treats Volume Mount Points as symlinks, whether they are mounts of \\?\{VolumeID} or mounts of paths from another place. Those are more-similar to a normal mount and bind-mount respectively in Linux, and filepath.Walk (and os.Lstat) don't see those as symlinks either.

So this could be resolved by dropping all the support for IO_REPARSE_TAG_MOUNT_POINT in Go.

Or teaching os.Lstat on Linux to recognise mount points as symlinks, so that everyone has an equal share of the pain...

Looks like this was briefly touched-upon in discussion of the change that introduced it but the discussion wasn't followed up on, except to note that tests that expected to treat mount points as symlinks fail if you don't treat mount points as symlinks...

Edit: I realised much later that readLink + winio.DecodeReparsePoint in my test code above is probably just reimplementing golang.org/x/sys/windows.Readlink. Although that ReadLink doesn't tell you what kind of reparse point it was, so it's not quite a general solution as for the use-cases discussed here, you want to distinguish symlinks from mount points. winio.DecodeReparsePoint does tell you if it's a mount point or not.

[TBBle]

Thinking about this, one rationale for treating an NTFS Mount Point like a Symlink is that, unlike mount --rbind in Linux, subsequently-created Volume Mounts are visible through that reparse point, so it's possible to introduce an infinite recursion.

I assume that avoiding infinite recursion is why filepath.Walk doesn't follow symlinks in the first place.

[bcmills]

After https://go.dev/cl/463177, the fix here is probably to add a trailing separator to the path passed to Walk. @buddhike (if this issue still matters to you), does that solve the problem?

[TBBle]

For the record, the above fix was in Go 1.21.

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)