Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Podman cannot load stats from rootless containers #137

Closed
marusak opened this issue Oct 15, 2019 · 1 comment · Fixed by #271
Closed

Podman cannot load stats from rootless containers #137

marusak opened this issue Oct 15, 2019 · 1 comment · Fixed by #271
Labels
knownissue

Comments

@marusak
Copy link
Member

marusak commented Oct 15, 2019

Reported: containers/podman#4268
Logs in console: > warning: Failed to update container stats: {"error":"io.podman.ErrorOccurred","parameters":{"reason":"Link not found"}}
Fails with:

Traceback (most recent call last):
  File "test/check-application", line 441, in testLifecycleOperationsUser
    self._testLifecycleOperations(False)
  File "test/check-application", line 477, in _testLifecycleOperations
    self.assertIn('%', cpu)
AssertionError: '%' not found in ''
@marusak
Copy link
Member Author

marusak commented Oct 15, 2019 •
edited by cockpituous
Loading

verify/fedora-31
Ooops, it happened again

# testLifecycleOperationsUser (__main__.TestApplication)
#
Created symlink /home/admin/.config/systemd/user/sockets.target.wants/io.podman.socket -> /usr/lib/systemd/user/io.podman.socket.
Warning: Stopping cockpit.service, but it can still be activated by:
  cockpit.socket

DevTools listening on ws://127.0.0.1:9519/devtools/browser/5de626f8-aa3e-4cc0-b02f-871f2524f5e3
[1015/112020.458633:ERROR:egl_util.cc(60)] Failed to load GLES library: /usr/lib64/chromium-browser/swiftshader/libGLESv2.so: /usr/lib64/chromium-browser/swiftshader/libGLESv2.so: cannot open shared object file: No such file or directory
[1015/112020.460946:ERROR:viz_main_impl.cc(170)] Exiting GPU process due to errors during initialization
[1015/112020.464204:WARNING:gpu_process_host.cc(1188)] The GPU process has crashed 1 time(s)
> warning: grep: /sys/class/dmi/id/product_serial: Permission denied
grep: /sys/class/dmi/id/power/autosuspend_delay_ms: Input/output error
grep: /sys/class/dmi/id/chassis_serial: Permission denied
grep: /sys/class/dmi/id/product_uuid: Permission denied

> warning: Failed to update container stats: {"error":"io.podman.ErrorOccurred","parameters":{"reason":"Link not found"}}
> warning: Unhandled event type  system
Wrote screenshot to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2501-FAIL.png
Wrote HTML dump to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2501-FAIL.html
Wrote JS log to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2501-FAIL.js.log
Journal extracted to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2501-FAIL.log
Connection to 127.0.0.2 closed by remote host.
Traceback (most recent call last):
  File "check-application", line 441, in testLifecycleOperationsUser
    self._testLifecycleOperations(False)
  File "check-application", line 477, in _testLifecycleOperations
    self.assertIn('%', cpu)
AssertionError: '%' not found in ''

not ok 5 testLifecycleOperationsUser (__main__.TestApplication) # duration: 56s

First occurrence: 2019-10-15T14:05:50.867173
Times recorded: 2
Latest occurrences:

  • 2019-10-15T14:05:50.867173
  • 2019-10-15T12:50:48.300309 | revision 366c5d2305b6d1530d023b199158221b04f6bb89
# ----------------------------------------------------------------------
# testLifecycleOperationsUser (__main__.TestApplication)
#
Created symlink /home/admin/.config/systemd/user/sockets.target.wants/io.podman.socket -> /usr/lib/systemd/user/io.podman.socket.
Warning: Stopping cockpit.service, but it can still be activated by:
  cockpit.socket

DevTools listening on ws://127.0.0.1:9266/devtools/browser/49d6c75f-8eca-4bd3-9810-b9879f1e270b
> warning: grep: /sys/class/dmi/id/product_serial: Permission denied
grep: /sys/class/dmi/id/power/autosuspend_delay_ms: Input/output error
grep: /sys/class/dmi/id/chassis_serial: Permission denied
grep: /sys/class/dmi/id/product_uuid: Permission denied

> warning: Failed to update container stats: {"error":"io.podman.ErrorOccurred","parameters":{"reason":"Link not found"}}
Wrote screenshot to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2201-FAIL.png
Wrote HTML dump to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2201-FAIL.html
Wrote JS log to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2201-FAIL.js.log
> warning: Unhandled event type  system
Journal extracted to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2201-FAIL.log
Connection to 127.0.0.2 closed by remote host.
Traceback (most recent call last):
  File "./test/check-application", line 441, in testLifecycleOperationsUser
    self._testLifecycleOperations(False)
  File "./test/check-application", line 479, in _testLifecycleOperations
    self.assertIn('%', cpu)
AssertionError: '%' not found in ''

not ok 1 testLifecycleOperationsUser (__main__.TestApplication) # duration: 38s

First occurrence: 2019-10-17T12:09:55.659431
Times recorded: 1
Latest occurrences:

  • 2019-10-17T12:09:55.659431
# testLifecycleOperationsUser (__main__.TestApplication)
#
Created symlink /home/admin/.config/systemd/user/sockets.target.wants/io.podman.socket -> /usr/lib/systemd/user/io.podman.socket.
Warning: Stopping cockpit.service, but it can still be activated by:
  cockpit.socket

DevTools listening on ws://127.0.0.1:9237/devtools/browser/81fe9d5f-3df5-4fab-8fd6-347302e01ce2
[1017/102921.463312:ERROR:egl_util.cc(60)] Failed to load GLES library: /usr/lib64/chromium-browser/swiftshader/libGLESv2.so: /usr/lib64/chromium-browser/swiftshader/libGLESv2.so: cannot open shared object file: No such file or directory
[1017/102921.465763:ERROR:viz_main_impl.cc(170)] Exiting GPU process due to errors during initialization
[1017/102921.470677:WARNING:gpu_process_host.cc(1188)] The GPU process has crashed 1 time(s)
> warning: grep: /sys/class/dmi/id/product_serial: Permission denied
grep: /sys/class/dmi/id/power/autosuspend_delay_ms: Input/output error
grep: /sys/class/dmi/id/chassis_serial: Permission denied
grep: /sys/class/dmi/id/product_uuid: Permission denied

> warning: Failed to update container stats: {"error":"io.podman.ErrorOccurred","parameters":{"reason":"Link not found"}}
> warning: Unhandled event type  system
Wrote screenshot to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2401-FAIL.png
Wrote HTML dump to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2401-FAIL.html
Wrote JS log to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2401-FAIL.js.log
Journal extracted to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2401-FAIL.log
Connection to 127.0.0.2 closed by remote host.
Traceback (most recent call last):
  File "check-application", line 441, in testLifecycleOperationsUser
    self._testLifecycleOperations(False)
  File "check-application", line 474, in _testLifecycleOperations
    b.wait(lambda: b.text("#containers-containers tbody tr:contains('busybox:latest') > td:nth-child(5)") != "")
  File "testlib.py", line 324, in wait
    raise Error('timed out waiting for predicate to become true')
Error: timed out waiting for predicate to become true

not ok 5 testLifecycleOperationsUser (__main__.TestApplication) # duration: 130s

# 1 TESTS FAILED [131s on 1-ci-srv-06]
make: *** [Makefile:136: check] Error 1

First occurrence: 2019-10-17T12:36:31.289185
Times recorded: 4
Latest occurrences:

  • 2019-10-17T12:36:31.289185
  • 2019-10-17T12:39:24.097796
  • 2019-10-17T13:18:17.283704 | revision d1018a26ddda65e9963e1edfe22714f888919f64
  • 2019-10-17T13:53:40.460216 | revision 3aa8368599647306ed8e95cdd421556fd003db1d
``` # ---------------------------------------------------------------------- # testLifecycleOperationsUser (__main__.TestApplication) # Created symlink /home/admin/.config/systemd/user/sockets.target.wants/io.podman.socket -> /usr/lib/systemd/user/io.podman.socket. Warning: Stopping cockpit.service, but it can still be activated by: cockpit.socket

DevTools listening on ws://127.0.0.1:9810/devtools/browser/8a2d1815-7f29-4cbc-a011-19cf8fe7dffd
[1018/052415.663224:ERROR:egl_util.cc(60)] Failed to load GLES library: /usr/lib64/chromium-browser/swiftshader/libGLESv2.so: /usr/lib64/chromium-browser/swiftshader/libGLESv2.so: cannot open shared object file: No such file or directory
[1018/052415.665391:ERROR:viz_main_impl.cc(170)] Exiting GPU process due to errors during initialization
[1018/052415.668650:WARNING:gpu_process_host.cc(1188)] The GPU process has crashed 1 time(s)

warning: grep: /sys/class/dmi/id/product_serial: Permission denied
grep: /sys/class/dmi/id/power/autosuspend_delay_ms: Input/output error
grep: /sys/class/dmi/id/chassis_serial: Permission denied
grep: /sys/class/dmi/id/product_uuid: Permission denied

warning: Failed to update container stats: {"error":"io.podman.ErrorOccurred","parameters":{"reason":"Link not found"}}
warning: Unhandled event type system
Wrote screenshot to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2501-FAIL.png
Wrote HTML dump to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2501-FAIL.html
Wrote JS log to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2501-FAIL.js.log
Journal extracted to TestApplication-testLifecycleOperationsUser-fedora-31-127.0.0.2-2501-FAIL.log
Connection to 127.0.0.2 closed by remote host.
Traceback (most recent call last):
File "check-application", line 441, in testLifecycleOperationsUser
self._testLifecycleOperations(False)
File "check-application", line 474, in _testLifecycleOperations
b.wait(lambda: b.text("#containers-containers tbody tr:contains('busybox:latest') > td:nth-child(5)") != "")
File "testlib.py", line 335, in wait
raise Error('timed out waiting for predicate to become true')
Error: timed out waiting for predicate to become true

not ok 5 testLifecycleOperationsUser (main.TestApplication) # duration: 119s

First occurrence: 2019-10-18T05:25:32.122394 | revision 7197e090eb5419fcde9988b313fcb4cadd8570bb
Times recorded: 19
Latest occurrences:

- 2019-10-23T14:41:21.976187 | revision d2a25792c063fdcf88eefe55710474fe11410136
- 2019-10-24T13:27:29.277483 | revision f5e4b462b650cbe58744aacc513b0bbbb49e4560
- 2019-10-25T13:33:55.832537 | revision a110a65749c77483b03738334647ae86c188064f
- 2019-10-25T14:00:56.790991 | revision ef42986a1dfc99e62647dbf5e87710a3903cea7b
- 2019-10-25T14:30:02.526414 | revision 531ef46135970e66ca4d81a7cdeb53efece52f7d
- 2019-10-25T14:49:47.230856 | revision b3a1b37022b541f455a6225697828df9955df343
- 2019-10-25T15:53:12.711147 | revision 8f3c04eb0144d0fbd1d79c551e24c8699a70c958
- 2019-10-26T14:05:46.990666 | revision 2011c0da5c074675d85604dbff377da8e7c3c7aa
- 2019-10-29T15:20:49.616795 | revision 1f00e4c557e8498b4779e11ca1e384da7fedb1e1
- 2019-10-30T08:54:07.429308 | revision 2c1a7f5bd04e925b9153b9d21808132b5d91abe2

marusak added a commit to marusak/bots that referenced this issue Oct 15, 2019
@marusak
Add naughty for podman failing to get container stats
4e8bece 
cockpit-project#137
@marusak marusak mentioned this issue Oct 15, 2019
Merged
martinpitt pushed a commit that referenced this issue Oct 15, 2019
@marusak @martinpitt
Add naughty for podman failing to get container stats
f7fa1c4 
#137

Closes #138
@marusak marusak mentioned this issue Oct 17, 2019
Merged
@cockpituous cockpituous mentioned this issue Nov 22, 2019
Merged
marusak pushed a commit that referenced this issue Nov 22, 2019
@marusak
naughty: Close 137: Podman cannot load stats from rootless containers
8ed3bef 
Known issue which has not occurred in 23 days

Podman cannot load stats from rootless containers

Fixes #137

Closes #271
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
knownissue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

naughty: Close 137: Podman cannot load stats from rootless containers
1 participant
@marusak