Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restart workers on Memory Limits #1355

Open
withinboredom opened this issue Jan 31, 2025 · 2 comments
Open

Restart workers on Memory Limits #1355

withinboredom opened this issue Jan 31, 2025 · 2 comments
Labels
enhancement New feature or request

Comments

@withinboredom
Copy link
Collaborator

withinboredom commented Jan 31, 2025

Describe you feature request

Currently, in PHP, it allows setting a memory limit per a request. We also have worker scripts that restart based on the number of requests. Inspired by https://externals.io/message/126226, it may make sense to measure memory usage per the limit to handle potential memory leaks.

This needs a few things working together: GOMEMLIMIT for keeping the Go runtime in check, memory_limit ini settings, and the current container/cgroup memory limits (plus current number of threads).

From that, we can derive if a request is going to potentially exceed those limits, we can start a rolling restart of workers before handling the request. If after restarting the workers, and we are still brushing up against the limits, then we can assume there is a memory leak in go and we need to restart the entire process.

Note: this could also be an autoscaling strategy (density vs. speed)

@withinboredom withinboredom added the enhancement New feature or request label Jan 31, 2025
@dunglas
Copy link
Owner

dunglas commented Feb 1, 2025

Probably related: caddyserver/caddy#6809

@AlliBalliBaba
Copy link
Collaborator

It would also be possible to restart single threads, not sure how safe that is though. Some leaking extensions could hang on shutdown (#1296)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants