Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Massive log spam when socket gets stale. #17

Open
bmario opened this issue Jun 20, 2023 · 1 comment
Open

Massive log spam when socket gets stale. #17

bmario opened this issue Jun 20, 2023 · 1 comment

Comments

@bmario
Copy link
Member

bmario commented Jun 20, 2023

The websocket sink can generate up to 600k log entries within 30 seconds.

It starts with a fuckton of these:

 2023-06-13 08:02:05,886 [INFO    ] [metricq_sink_websocket.web_socket] Unsubscribing stale websocket <MetricqWebSocketResponse Switching Protocols GET / > from metric ['taurus.taurusi6355.cpu1.power', 'taurus.taurusi6532.cpu0.power', 'taurus.taurusi6427.cpu0.power', 'taurus.taurusi6576.cpu1.power', 'taurus.taurusi6251.cpu1.power', 'taurus.taurusi7143.power', 'taurus.taurusi6313.cpu0.power', 'taurus.taurusi6113.cpu0.power', 'taurus.taurusi6473.cpu0.power', 'taurus.taurusi6574.cpu1.power', 'taurus.taurusi6304.cpu1.power', 'taurus.taurusi6517.cpu1.power', 'taurus.taurusi6343.cpu1.power', 'taurus.taurusi6100.cpu1.power', 'taurus.taurusi6500.cpu1.power', 'taurus.taurusi6073.cpu0.power', 'taurus.taurusi6548.cpu0.power', 'taurus.taurusi6583.cpu1.power', 'taurus.taurusi6321.cpu1.power', 'taurus.taurusi6384.cpu1.power', 'taurus.taurusi6398.cpu1.power', 'taurus.taurusi6147.cpu1.power', 'taurus.taurusi6215.cpu1.power', 'taurus.taurusi6235.cpu1.power', 'taurus.taurusi6172.cpu0.power', 'taurus.taurusi6290.cpu0.power', 'taurus.taurusi6238.cpu0.power', 'taurus.taurusi6520.cpu1.power', 'taurus.taurusi6234.cpu1.power', 'taurus.taurusi6165.cpu0.power', 'taurus.taurusi6482.cpu0.power', 'taurus.taurusi6018.cpu1.power', 'taurus.taurusi6509.cpu0.power', 'taurus.taurusi6290.cpu1.power', 'taurus.taurusi6056.cpu1.power', 'taurus.taurusi6395.cpu0.power', 'taurus.taurusi6465.cpu1.power', 'taurus.taurusi6103.cpu1.power', 'taurus.taurusi6010.cpu0.power', 'taurus.taurusi6552.cpu0.power', 'taurus.taurusi6566.cpu1.power', 'taurus.taurusi6193.cpu0.power', 'taurus.taurusi6229.cpu1.power', 'taurus.taurusi6530.cpu1.power', 'taurus.taurusi6259.cpu0.power', 'taurus.taurusi6600.cpu0.power', 'taurus.taurusi6344.cpu0.power', 'taurus.taurusi6466.cpu0.power', 'taurus.taurusi6349.cpu0.power', 'taurus.taurusi6552.cpu1.power', 'taurus.taurusi6185.cpu1.power', 'taurus.taurusi6022.cpu1.power', 'taurus.taurusi6114.cpu0.power', 'taurus.taurusi6565.cpu1.power', 'taurus.taurusi6507.cpu0.power', 'taurus.taurusi6393.cpu1.power', 'taurus.taurusi6584.cpu1.power', 'taurus.taurusi6601.cpu0.power', 'taurus.taurusi6316.cpu1.power', 'taurus.taurusi6368.cpu1.power', 'taurus.taurusi6610.cpu1.power', 'taurus.taurusi7136.power', 'taurus.taurusi6334.cpu0.power', 'taurus.taurusi6291.cpu0.power', 'taurus.taurusi6354.cpu1.power', 'taurus.taurusi6201.cpu1.power', 'taurus.taurusi6462.cpu0.power', 'taurus.taurusi6545.cpu1.power', 'taurus.taurusi6606.cpu0.power', 'taurus.taurusi6228.cpu1.power', 'taurus.taurusi6605.cpu1.power', 'taurus.taurusi6596.cpu1.power', 'taurus.taurusi6146.cpu1.power', 'taurus.taurusi6144.cpu1.power', 'taurus.taurusi6386.cpu1.power', 'taurus.taurusi6026.cpu0.power', 'taurus.taurusi6122.cpu1.power', 'taurus.taurusi6271.cpu0.power', 'taurus.taurusi6455.cpu1.power', 'taurus.taurusi6042.cpu0.power', 'taurus.taurusi6415.cpu0.power', 'taurus.taurusi6302.cpu0.power', 'taurus.taurusi6479.cpu0.power', 'taurus.taurusi6104.cpu0.power', 'taurus.taurusi6410.cpu0.power', 'taurus.taurusi6224.cpu1.power', 'taurus.taurusi6292.cpu0.power', 'taurus.taurusi6020.cpu1.power', 'taurus.taurusi6270.cpu0.power', 'taurus.taurusi6584.cpu0.power', 'taurus.taurusi6011.cpu0.power', 'taurus.taurusi6425.cpu0.power', 'taurus.taurusi6216.cpu1.power', 'taurus.taurusi6165.cpu1.power', 'taurus.taurusi6322.cpu0.power', 'taurus.taurusi6392.cpu0.power', 'taurus.taurusi6242.cpu0.power', 'taurus.taurusi6612.cpu0.power', 'taurus.taurusi6515.cpu1.power', 'taurus.taurusi6020.cpu0.power', 'taurus.taurusi6492.cpu1.power', 'taurus.taurusi6136.cpu0.power', 'taurus.taurusi6209.cpu1.power', 'taurus.taurusi6484.cpu0.power', 'taurus.taurusi6379.cpu1.power', 'taurus.taurusi6582.cpu0.power', 'taurus.taurusi6220.cpu0.power', 'taurus.taurusi6134.cpu0.power', 'taurus.taurusi6448.cpu1.power', 'taurus.taurusi6590.cpu1.power', 'taurus.taurusi6366.cpu0.power', 'taurus.taurusi6468.cpu1.power', 'taurus.taurusi6052.cpu0.power', 'taurus.taurusi6322.cpu1.power', 'taurus.taurusi6578.cpu1.power', 'taurus.taurusi6007.cpu1.power', 'taurus.taurusi6197.cpu0.power', 'taurus.taurusi6473.cpu1.power', 'taurus.taurusi7087.power', 'taurus.taurusi6143.cpu0.power', 'taurus.taurusi6548.cpu1.power', 'taurus.taurusi6477.cpu1.power', 'taurus.taurusi6426.cpu1.power', 'taurus.taurusi6072.cpu0.power', 'taurus.taurusi6591.cpu1.power', 'taurus.taurusi6046.cpu0.power', 'taurus.taurusi6504.cpu0.power', 'taurus.taurusi6077.cpu1.power', 'taurus.taurusi6391.cpu0.power', 'taurus.taurusi6074.cpu0.power', 'taurus.taurusi6492.cpu0.power', 'taurus.taurusi6059.cpu1.power', 'taurus.taurusi6360.cpu0.power', 'taurus.taurusi6554.cpu1.power', 'taurus.taurusi6099.cpu1.power', 'taurus.taurusi7084.power', 'taurus.taurusi6063.cpu1.power', 'taurus.taurusi6477.cpu0.power', 'taurus.taurusi6522.cpu1.power', 'taurus.taurusi6112.cpu0.power', 'taurus.taurusi6005.cpu1.power', 'taurus.taurusi6192.cpu1.power', 'taurus.taurusi6496.cpu0.power', 'taurus.taurusi6311.cpu1.power', 'taurus.taurusi6080.cpu1.power', 'taurus.taurusi6126.cpu1.power', 'taurus.taurusi6007.cpu0.power', 'taurus.taurusi6395.cpu1.power', 'taurus.taurusi6034.cpu0.power', 'taurus.taurusi6052.cpu1.power', 'taurus.taurusi6332.cpu0.power', 'taurus.taurusi6024.cpu0.power', 'taurus.taurusi6231.cpu1.power', 'taurus.taurusi6494.cpu1.power', 'taurus.taurusi6154.cpu1.power', 'taurus.taurusi6213.cpu1.power', 'taurus.taurusi6403.cpu0.power', 'taurus.taurusi6138.cpu1.power', 'taurus.taurusi6031.cpu0.power', 'taurus.taurusi6187.cpu1.power', 'taurus.taurusi6318.cpu1.power', 'taurus.taurusi6313.cpu1.power', 'taurus.taurusi6058.cpu1.power', 'taurus.taurusi6345.cpu0.power', 'taurus.taurusi6381.cpu0.power', 'taurus.taurusi6607.cpu1.power', 'taurus.taurusi6314.cpu1.power', 'taurus.taurusi6555.cpu0.power', 'taurus.taurusi6291.cpu1.power', 'taurus.taurusi6099.cpu0.power', 'taurus.taurusi6461.cpu1.power', 'taurus.taurusi6385.cpu1.power', 'taurus.taurusi6098.cpu1.power', 'taurus.taurusi6254.cpu1.power', 'taurus.taurusi6284.cpu1.power', 'taurus.taurusi6274.cpu0.power', 'taurus.taurusi6503.cpu1.power', 'taurus.taurusi6508.cpu1.power', 'taurus.taurusi6575.cpu1.power', 'taurus.taurusi6497.cpu1.power', 'taurus.taurusi6560.cpu0.power', 'taurus.taurusi6355.cpu0.power', 'taurus.taurusi6376.cpu1.power', 'taurus.taurusi6173.cpu1.power', 'taurus.taurusi6327.cpu0.power', 'taurus.taurusi6192.cpu0.power', 'taurus.taurusi6513.cpu0.power', 'taurus.taurusi6252.cpu0.power', 'taurus.taurusi6457.cpu1.power', 'taurus.taurusi6299.cpu0.power', 'taurus.taurusi6305.cpu1.power', 'taurus.taurusi6028.cpu0.power', 'taurus.taurusi6168.cpu1.power', 'taurus.taurusi6347.cpu1.power', 'taurus.taurusi6162.cpu1.power', 'taurus.taurusi6338.cpu0.power', 'taurus.taurusi6087.cpu0.power', 'taurus.taurusi6314.cpu0.power', 'taurus.taurusi6344.cpu1.power', 'taurus.taurusi6466.cpu1.power', 'taurus.taurusi6030.cpu1.power', 'taurus.taurusi6467.cpu1.power', 'taurus.taurusi6022.cpu0.power', 'taurus.taurusi6332.cpu1.power', 'taurus.taurusi6571.cpu1.power', 'taurus.taurusi6059.cpu0.power', 'taurus.taurusi6586.cpu1.power', 'taurus.H02.power', 'taurus.taurusi6183.cpu0.power', 'taurus.taurusi6493.cpu1.power', 'taurus.taurusi6450.cpu0.power', 'taurus.taurusi6005.cpu0.power', 'taurus.taurusi6016.cpu1.power', 'taurus.taurusi7113.power', 'taurus.taurusi6034.cpu1.power', 'taurus.taurusi6469.cpu0.power', 'taurus.taurusi6229.cpu0.power', 'taurus.taurusi6094.cpu1.power', 'taurus.taurusi6469.cpu1.power', 'taurus.taurusi6402.cpu1.power', 'taurus.taurusi6224.cpu0.power', 'taurus.taurusi6268.cpu0.power', 'taurus.taurusi6268.cpu1.power', 'taurus.taurusi6169.cpu1.power', 'taurus.taurusi6288.cpu1.power', 'taurus.taurusi6416.cpu1.power', 'taurus.taurusi6278.cpu0.power', 'taurus.taurusi6529.cpu0.power', 'taurus.taurusi6592.cpu1.power', 'taurus.taurusi6436.cpu1.power', 'taurus.taurusi6559.cpu1.power', 'taurus.taurusi6490.cpu0.power', 'taurus.taurusi6097.cpu0.power', 'taurus.taurusi6069.cpu1.power', 'taurus.taurusi6188.cpu0.power', 'taurus.taurusi6074.cpu1.power', 'taurus.taurusi6125.cpu0.power', 'taurus.taurusi6184.cpu1.power', 'taurus.taurusi6525.cpu0.power', 'ta

Followed by a bunch of those:

2023-06-13 08:02:07,418 [ERROR   ] [metricq_sink_websocket.sink] failed to unsubscribe metric taurus.taurusi6313.cpu0.power: <MetricqWebSocketResponse Switching Protocols GET / >
@tilsche
Copy link
Contributor

tilsche commented Jun 20, 2023

Kinda impressive, that's just 50 us per log entry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants