Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebSocket server load #567

Open
JustAnotherArchivist opened this issue Dec 1, 2023 · 0 comments
Open

WebSocket server load #567

JustAnotherArchivist opened this issue Dec 1, 2023 · 0 comments

Comments

@JustAnotherArchivist
Copy link
Contributor

The WebSocket server causes a lot of load. Before #566, it regularly pinned a core on the AT instance. That change improves the situation, but it's not a real solution, especially not in the longer term if capacity were to grow further.

Some ideas:

  • Batch messages so they can be compressed and sent at once.
  • Reimplement in a different language that's more efficient at the task than Python.
  • Reduce the data sent to the clients. The obvious idea here would be turning it into pub-sub, i.e. making clients subscribe to jobs. For example, the dashboard would subscribe to any job that's currently visible per the filters. This may necessitate buffering messages on the server so they can be sent when the user changes the filter. Stats updates would be sent for any jobs that clients haven't subscribed to.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant