Builds on Amazon EKS fail with error: failed to solve: read /proc/pressure/cpu: operation not supported

[t089]

When using docker buildx build with a buildkit builder on an AWS EKS (v1.23.17-eks) node, the build runs successfully, but the process fails with an error:

error: failed to solve: read /proc/pressure/cpu: operation not supported

When using buildkit:latest this error does not occur.

Example

docker buildx create --name test2 --driver docker-container --driver-opt image=moby/buildkit:master,network=host --use
docker buildx build . -t test
[+] Building 0.9s (5/5) FINISHED                                                                                                                                                                  
 => [internal] load build definition from Dockerfile                                                                                                                                         0.0s
 => => transferring dockerfile: 167B                                                                                                                                                         0.0s
 => [internal] load metadata for public.ecr.aws/amazoncorretto/amazoncorretto:11                                                                                                             0.7s
 => [internal] load .dockerignore                                                                                                                                                            0.0s
 => => transferring context: 2B                                                                                                                                                              0.0s
 => CACHED [1/2] FROM public.ecr.aws/amazoncorretto/amazoncorretto:11@sha256:a7e405bc6bf0aa173e5d00d77117e37d9ca26ccea2d00230b4f08b69c9530c35                                                0.0s
 => => resolve public.ecr.aws/amazoncorretto/amazoncorretto:11@sha256:a7e405bc6bf0aa173e5d00d77117e37d9ca26ccea2d00230b4f08b69c9530c35                                                       0.0s
 => [2/2] RUN touch hello_world                                                                                                                                                          0.1s
error: failed to solve: read /proc/pressure/cpu: operation not supported

bash-4.2# docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2)

Server:
 Containers: 31
  Running: 30
  Paused: 0
  Stopped: 1
 Images: 30
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: 5fd4c4d144137e991c4acebb2146ab1483a97925
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.4.219-126.411.amzn2.x86_64
 Operating System: Amazon Linux 2
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.27GiB
 Name: ***.compute.internal
 ID: CNT7:6GF2:MICG:XLLS:NKGT:SARS:LT7C:64XR:SBWI:CHIK:7SE5:YSKI
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: true

[crazy-max]

Seems linked to #3860

Kernel Version: 5.4.219-126.411.amzn2.x86_64

Didn't try BuildKit master on Amazon Linux 2 myself so might be this kernel does not have CONFIG_PSI enabled but build should not fail.
What's the output of cat /boot/config-$(uname -r) | grep "CONFIG_PSI" ?

[crazy-max]

@tonistiigi Could we have an opt-out param in buildkit config for proc pressure? I recall Netdata has some opt-outs for os metrics: https://learn.netdata.cloud/docs/data-collection/linux-systems/os-provided-metrics

[tonistiigi]

PSI is already optional. We let the "file not exist" error fail cleanly. But I'm not sure why for this system, the error would be "not supported". Does the file exist, but you are just not allowed to read it? Do you know what component is adding that restriction?

[crazy-max]

@tonistiigi I was looking at another machine and it seems CONFIG_PSI_DEFAULT_DISABLED=y is the culprit:

$ cat /boot/config-$(uname -r) | grep "CONFIG_PSI"
CONFIG_PSI=y
CONFIG_PSI_DEFAULT_DISABLED=y

I will make further tests and let you know

[crazy-max]

awslabs/amazon-eks-ami#660 (comment)

[t089]

The files in /proc/pressure exist, but eg cat /proc/pressure/cpu gives this operation not supported error.