runtime/pprof: takes zero samples in BashOnWindows (WSL)

[bradleyjkemp]

This is probably a BashOnWindows issue but just wanted to raise here to check.

What version of Go are you using (go version)?

go version go1.9.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/mnt/c/Users/Bradley/GoglandProjects/workflow"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build102190546=/tmp/go-
build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

Using dummy benchmark in https://play.golang.org/p/orUkRfuAl1:
When run in windows this produces a cpu.pprof file of 4.5Kb and opening with pprof gives the following information:

Type: cpu
Time: Oct 20, 2017 at 8:12pm (DST)
Duration: 12.25s, Total samples = 3.83s (31.28%)

When the same program is run in BashOnWindows this produces a cpu.pprof file of 239Bytes and opening with pprof gives:

Type: cpu
Time: Oct 20, 2017 at 8:11pm (DST)
Duration: 12.22s, Total samples = 0

What did you expect to see?

Both environments to produce a cpu.pprof file containing a similar number of samples.

What did you see instead?

When run in BashOnWindows pprof never takes a single sample.

[bradfitz]

Yes, this is probably because WSL doesn't implement profiling signals.

/cc @jessfraz to escalate within Microsoft.

[jessfraz]

will see what I can do

…

On Fri, Oct 20, 2017 at 3:33 PM, Brad Fitzpatrick ***@***.***> wrote: Yes, this is probably because WSL doesn't implement profiling signals. /cc @jessfraz <https://github.com/jessfraz> to escalate within Microsoft. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22366 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYNbHTH3fiG3_EUJGGiO5IMPlCw-xC5ks5suPVpgaJpZM4QBHV-> .

-- Jessie Frazelle 4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3 pgp.mit.edu <http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3>

[rsc]

If someone wants to try again with WSL2, this may work now that there's a full Linux kernel inside.

[adam000]

It would appear there is no change from WSL1 to WSL2:

C:\>wsl -l -v
  NAME            STATE           VERSION
* Ubuntu-18.04    Stopped         2
  Legacy          Stopped         1

C:\>wsl

[12:05:23] (wsl2test) /mnt/c
Z cd

[12:05:40] (wsl2test) ~
Z cd src/pprof-test

[12:05:47] (wsl2test) ~/src/pprof-test
Z vim main.go

[12:07:24] (wsl2test) ~/src/pprof-test
Z go run main.go

[12:09:10] (wsl2test) ~/src/pprof-test
Z pprof cpu.pprof
File: main
Type: cpu
Time: Sep 18, 2019 at 12:07pm (PDT)
Duration: 1.67mins, Total samples = 0
No samples were found with the default sample value type.
Try "sample_index" command to analyze different sample values.
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) ^C
pprof: Interrupt

[12:16:13] (wsl2test) ~/src/pprof-test
Z go version
go version go1.12.9 linux/amd64

Disclaimer: I don't know much about pprof, just happened to have a WSL2 instance and wanted to help. If I'm doing something wrong or if there are more tests for me to run, I'm happy to help.

[tobiaskohlbau]

Does the kernel need any specific support to enable profiling? I can confirm that WSL2 does not support this at the moment, but they accept suggestions for additional build flags. If someone can point me on resources or information about which flags/features the kernel needs to support pprof I would build a custom kernel.

Edit:

I've got profiling working with a custom kernel. The missing piece is the CONFIG_HIGH_RES_TIMERS=y kernel option. I got this hint through the notice on the pprof documentation which links to #13841 where the option is referenced. For anyone who is curios how to get this working on WSL2:

• clone https://github.com/microsoft/WSL2-Linux-Kernel
• add option CONFIG_HIGH_RES_TIMERS=y to Microsoft/config-wsl
• follow README-Microsoft.WSL2 (notice a missing apt within the dependency installation)
• copy vmlinux within the root folder of the repo over to windows
• add %USERPROFILE%/.wslconfig which sets kernel to vmlinux copied earlier
  • see Release Notes for file format
• reboot computer or restart wsl with wsl --shutdown

I will contact the WSL team to see if it's possible to get the flag into the official kernel.

[3AceShowHand]

Does the kernel need any specific support to enable profiling? I can confirm that WSL2 does not support this at the moment, but they accept suggestions for additional build flags. If someone can point me on resources or information about which flags/features the kernel needs to support pprof I would build a custom kernel.

Edit:

I've got profiling working with a custom kernel. The missing piece is the CONFIG_HIGH_RES_TIMERS=y kernel option. I got this hint through the notice on the pprof documentation which links to #13841 where the option is referenced. For anyone who is curios how to get this working on WSL2:

• clone https://github.com/microsoft/WSL2-Linux-Kernel

• add option CONFIG_HIGH_RES_TIMERS=y to Microsoft/config-wsl

• follow README-Microsoft.WSL2 (notice a missing apt within the dependency installation)

• copy vmlinux within the root folder of the repo over to windows

• add %USERPROFILE%/.wslconfig which sets kernel to vmlinux copied earlier

  • see Release Notes for file format

• reboot computer or restart wsl with wsl --shutdown

I will contact the WSL team to see if it's possible to get the flag into the official kernel.

I try to compile the kernel, but a lot of error message, failed...

please send them a message, this feature should be added asap.

[tobiaskohlbau]

I'm unable to reproduce your error messages, I built it on two machines and everything worked fine. I got the response from one of the maintainers that they would like to add the kernel flag. But I'm very sure this takes a few weeks to settle in the latest preview. So until it's in the official kernel the only option is to build a custom kernel.

Notice that for building you need a recent ubuntu distribution like 19.04, as stated in the documentation from the kernel:

Install a recent Ubuntu distribution

[3AceShowHand]

I see, thank you for your kindness.

[tobiaskohlbau]

Quick followup:

The release notes from insider build 19013 state:

Update kernel version to 4.19.79. (add CONFIG_HIGH_RES_TIMERS, CONFIG_TASK_XACCT, CONFIG_TASK_IO_ACCOUNTING, CONFIG_SCHED_HRTICK, and CONFIG_BRIDGE_VLAN_FILTERING).

I've tested and can confirm that the default kernel on build 19013 and later produces the right pprof profile which has samples in it. I don't know if this resolves this issue or it should be left open cause wsl2 is in insiders only and wsl1 still has the issue.

[bradfitz]

Thanks! I'll close this.