debug: automatically Infer Path Mapping for Remote Debugging

[quoctruong]

Background

Existing Implementation

Currently, we are using 2 attributes in the launch configuration to configure path mapping: cwd and remotePath.

cwd - Workspace relative or absolute path to the working directory of the program being debugged. Default is the current workspace.
remotePath - Absolute path to the file being debugged on the remote machine in case of remote debugging.

Let’s say the local path is C:\Users\quoct\Github\project\hello-world and the remote path is /app/project/hello-world.

Processing and Storing the local and remote path

Using the path in cwd and remotePath, we detect the local path separator (LPS) and remote path separator (RPS) respectively.

With these separators, we attempt to find the longest common suffix between the 2 paths. For the example above, that would be project/hello-world.

We truncate the longest common suffix from the local and remote path and store them internally. For the example above, that would be C:\Users\quoct\Github and /app. If there are no common suffixes, we just trim the last path separator of the remote path. Let’s call these processed local root (PLR) and processed remote root (PRR) respectively.

Setting a breakpoint

This happens when the user sets a breakpoint in a file, for example: C:\Users\quoct\Github\project\hello-world\testing.go.

We use the remote path separator in the file name. So it will become C:/Users/quoct/Github/project/hello-world/testing.go. We then substitute the PLR (C:\Users\quoct\Github) with the PRR (/app). So now the breakpoint file will become /app/project/hello-world/testing.go. We call Delve to set a breakpoint in this file.

Breakpoint hit

This happens if a breakpoint is hit on the remote Delve instance. In this case, Delve will send us back the name of the file that contains the breakpoint and we will have to map this file to a file on the local machine. Let’s say the name of the file is /app/project/hello-world/testing.go.

If the path to convert starts with the PRR (/app), we then substitute the PRR with the PLR (C:\Users\quoct\Github) in the path. So it will become C:\Users\quoct\Github\project\hello-world\testing.go. There seems to be a BUG here where we are not taking into account different path separators for remote and local roots.

If it doesn’t start with the PRR, then we assume that it can be mapped to a file in the GOROOT directory. We do this by assuming that the remote file’s name starts with /app (which is another BUG?).

Problems

A lot of times users don’t know that they have to provide path mapping for the breakpoints to hit. Even if they do, they don’t provide the correct path mappings. So what happens is that the debugger appears to be running but no breakpoints are hit. This can cause a lot of confusion to the users.
Users currently cannot set breakpoints for packages in GOPATH. There is a proposal to add another field to map this. Even the experience for setting breakpoints for files in GOROOT is not great as we are blindly assuming that the GOROOT starts with /src.
The current implementation expects the paths to be absolute but they can be relative. There are PRs out there trying to address this by introducing more fields

Proposal

Instead of trying to add multiple fields to address the relative path mapping and GOPATH problems, we can fix these problems and reduce the user’s friction points by attempting to automatically infer the path mapping for the user.

Delve has an API to list all the files used by the program. So we can simply get the list of files on the remote machine and map it to the files on the local machine. These include files in the GOROOT and GOPATH on the remote machine as well so we can try to map them to the files in the local machine GOROOT and GOPATH.

Mapping local path to remote path

Since we get all the remote sources, we can retrieve all the remote source files that share the same file name with the local file. We can then do a suffix match to get the best matching remote path.
Mapping remote path to local path
This is more complicated because there are multiple places where we can try to find a match on the local machine.

However, we can use the ListPackagesBuildInfo to help us retrieve all the different Go packages used. Each package item returned comes with a Directory Path and an Import Path. Using Directory Path, we can determine if the remote path is in a particular package or not:

• If the remote path is in a package, we can try to find whether the package exists on the local machine. We will be searching for it in 3 places:

  • The current user’s workspace folder.
  • The current user’s GOPATH.
  • The current user’s GOMOD folder (%GOMOD%/pkg/mod) if the package name contains pkg/mod.

• If the remote path is not in any package, then we retrieve the name of the remote path and search for files with that name in the current workspace folder. We find the best matching one using a suffix match.

@polinasok @hyangah @stamblerre

[gopherbot]

Change https://golang.org/cl/234020 mentions this issue: Add logic to infer remote path automatically

[sbalabanov]

Does running ListPackagesBuildInfo involve discovering go packages, and thus running 'go list' on remote side? If so, getting this information can be pretty expensive.

In loaded IDE, we can know folders paths from opened project metadata, along with GOPATH and GOROOT. In 99.(9)% of the cases this is enough to match the remote file with the local one.

One concern that I have is that getting a list of all source paths from remote may be expensive.

[gopherbot]

Change https://golang.org/cl/236017 mentions this issue: src/debugAdapter/goDebug: add extra check for remote path inference

[hyangah]

BTW, is it possible for users to offer a delve configuration with explicit path mapping info currently?

[polinasok]

Yes, for now, but there is talk of taking out that option once this one is stable. The problem is that a single path option can only point to either user's source code or 3rd party library in a different directory, but not both.

[hyangah]

@polinasok Maybe my question was not specific. I am asking if users can supply config.yml that supports multiple path substitution.

[koxu1996]

This is side comment that might help someone. Documentation regarding path mapping is not explaining clearly how to set it up and how it works. I had to investigate code and that allowed me to understand two things:

• remotePath keyword is misleading, because it's path to sources during executable build. Even if you move sources, then you must provide original one (I think they are compiled in), so for me name like originalBuildPath would be more clear.
• local path setup - this is actually path obtained from cwd, but I would prefer setting something like localPath explicitly.

[shaunco]

@koxu1996 - Thank you! I spent far too long trying to get breakpoints to work during a remote debug session between vscode on windows and delv in an ubuntu docker. Setting remotePath to the go workspace directory the binary was built from (even though it had no relation to the windows machine) and then setting cwd to ${workspaceFolder}, and everything worked.

It would be awesome if the vscode-go extension could just try remapping whatever paths are in the binary on the remote machine to the currently opened ${workspaceFolder}, as it seems like the most likely case.

https://github.com/golang/vscode-go/blob/master/docs/debugging.md#remote-debugging is also out of date and still mentions using program instead of cwd, among other things.

[gopherbot]

Change https://golang.org/cl/253741 mentions this issue: docs/debugging.md: update remote debugging documentation

[hyangah]

We updated the section for remote debugging. Please review and if you find it's still unclear and needs improvement, please file a new issue or submit your PRs.