[batch] OSError: no such device

[danking]

What happened?

This input container is about to get cleaned up. Maybe we have a race where we measure resource even though the job is complete?

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/batch/resource_usage.py", line 250, in periodically_measure
    await self.measure()
  File "/usr/local/lib/python3.9/dist-packages/batch/resource_usage.py", line 210, in measure
    percent_cpu_usage = self.percent_cpu_usage()
  File "/usr/local/lib/python3.9/dist-packages/batch/resource_usage.py", line 122, in percent_cpu_usage
    now_cpu_ns = self.cpu_ns()
  File "/usr/local/lib/python3.9/dist-packages/batch/resource_usage.py", line 114, in cpu_ns
    for line in f.readlines():
OSError: [Errno 19] No such device

https://cloudlogging.app.goo.gl/tte29H271hPvp4tn9

Version

0.2.124

Relevant log output

No response

[daniel-goldstein]

There is always a race because crun will delete the cgroup once the container completes. Looks like the memory tracking checks for this exact error but not cpu, it would assume it should be both.