Allow Walk to visit volume mounts on Windows

[gabriel-samfira]

This change adds a fork of the filepath.Walk function from the standard library, with changes that allows it to visit reparse points that are in fact volume mounts.

Currently go makes no distinction between a symlink, directory junction, volume mount or Unix domain socket. All reparse points are regarded as symlinks. This means that FileMode does not have ModeDir set for any path that also has ModeSymlink. This includes volume mounts. See: https://go-review.googlesource.com/c/go/+/41830/

This implementation also attempts to avoid recursions. I am on the fence about the added ErrPathRecursion. The idea is to raise an error if a recursion is detected and allow the walkFn function to handle that specific error. But I am not sure what the idiomatic way to handle this, is. Any suggestions are welcome.

The current test is a copy of the test from the standard library. I am looking into adding an additional test to check the volume mount specific code and recursion prevention, but that requires mounting a volume somewhere and attempting to walk it. I am unsure how this scenario would easily be set up. Locally, I created a VHDx, attached it to the system, created a partition, formatted it and then mounted it. But this requires some powershell commandlets to be available, or at least the windows provider to handle VHDx disks.

This change should remove the need to resort to workarounds like these: containerd/containerd@100624b on Windows. See containerd/containerd#4419 for details regarding the need for this.

Signed-off-by: Gabriel Adrian Samfira gsamfira@cloudbasesolutions.com

[gabriel-samfira]

A sample application to test this:

package main

import (
	"errors"
	"flag"
	"fmt"
	"log"
	"os"
	"path/filepath"
	"syscall"

	"github.com/containerd/continuity/pathdriver"
	"golang.org/x/sys/windows"
)

func isSystemOrHidden(path string) (bool, error) {
	attr, err := windows.GetFileAttributes(syscall.StringToUTF16Ptr(path))
	if err != nil {
		return false, err
	}
	if attr&syscall.FILE_ATTRIBUTE_HIDDEN != 0 && attr&syscall.FILE_ATTRIBUTE_SYSTEM != 0 {
		return true, nil
	}

	return false, nil
}

func main() {
	pth := flag.String("target", "", "path to walk")
	flag.Parse()

	final, err := filepath.Abs(*pth)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(final)
	err = pathdriver.LocalPathDriver.Walk(*pth, func(path string, info os.FileInfo, err error) error {
		if err != nil {
			isSystemOrHidden, errHidden := isSystemOrHidden(path)
			if errHidden != nil {
				return errHidden
			}
			if isSystemOrHidden {
				log.Printf("Skipping hidden system file/folder: %s", path)
				return nil
			}
			if errors.Is(err, pathdriver.ErrPathRecursion) {
				log.Printf("Not visiting previously seen path %s", path)
				return nil
			}
			log.Fatalf("%+v", err)
		}

		fmt.Printf("visiting %v\n", path)
		return nil
	})

	if err != nil {
		log.Fatalf("error: %+v", err)
	}
}

The folder structure and volume mount set up for testing. Volume is mounted twice. Once in C:\var\vhd and in C:\var\vhd\recurse:

Output of that sample app:

[TBBle]

I had written some tests that create and mount VHDs, but they end up pulling in hcsshim... (for access to computestorage API) and that seems to have pulled in a lot of stuff as a consequence.

It's possible that hcsshim has been fixed to not end up pulling in containerd when used as a library (it was only needed in the test code) but I realise now I had forgotten about that PR for more than a year. >_<

[cpuguy83]

This should be path_driver_unix.go.

[gabriel-samfira]

I renamed the file.

[kzys]

Isn't it for Windows?

[gabriel-samfira]

The current test is a copy of the test from the standard library. I am looking into adding an additional test to check the volume mount specific code and recursion prevention, but that requires mounting a volume somewhere and attempting to walk it. I am unsure how this scenario would easily be set up. Locally, I created a VHDx, attached it to the system, created a partition, formatted it and then mounted it. But this requires some powershell commandlets to be available, or at least the windows provider to handle VHDx disks.

It is. The tests are copied from the standard library. The idea was to make sure we maintain compatibility. But this PR was useful when we thought we needed to walk junctions (windows symlinks). Since then, we've replaced junctions with the bind filter, so this is no longer needed.

[gabriel-samfira]

Closing this PR. We've replaced the need for junctions with the bind filter (containerd/containerd#8043 (comment)), which does not use reparse points. If anyone still needs this work, feel free to grab the branch and open a new PR.