Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kill Agent child processes #3557

Closed
3 of 4 tasks
mssalvatore opened this issue Aug 4, 2023 · 0 comments · Fixed by #3587
Closed
3 of 4 tasks

Kill Agent child processes #3557

mssalvatore opened this issue Aug 4, 2023 · 0 comments · Fixed by #3587

Comments

@mssalvatore
Copy link
Collaborator

mssalvatore commented Aug 4, 2023

Description

The Agent uses multiprocessing to run plugins. It's possible that a plugin could hang indefinitely, preventing the parent (Agent) process from ever fully shutting down. Add some logic to main.py to attempt to forcefully kill any child processes still running. Watch out for the resource tracker.

Pseudocode

Be sure to add debug logging to the example below.

diff --git a/monkey/infection_monkey/main.py b/monkey/infection_monkey/main.py
index 9cf58b505..b12b84e44 100644
--- a/monkey/infection_monkey/main.py
+++ b/monkey/infection_monkey/main.py
@@ -179,6 +179,20 @@ def _run_agent(
         logger.exception(
             "Exception thrown from monkey's cleanup function: More info: {}".format(err)
         )
+    finally:
+        import psutil
+
+        for p in psutil.Process().children(recursive=True):
+            if "multiprocessing.resource_tracker" in p.cmdline()[2]:
+                # This process will clean itself up, but no other processes should be running at
+                # this time.
+                continue
+
+            p.kill()
 
 
 if "__main__" == __name__:

Tasks

Test Plugin

Mock1-exploiter.tar.gz

@mssalvatore mssalvatore added this to the v2.3.0 milestone Aug 4, 2023
mssalvatore added a commit that referenced this issue Aug 14, 2023
Plugins that hang can prevent the agent from closing. On agent shutdown,
we'll forcefully kill all child processes that have not shutdown on
their own.

Issue #3557
mssalvatore added a commit that referenced this issue Aug 14, 2023
mssalvatore added a commit that referenced this issue Aug 15, 2023
Plugins that hang can prevent the agent from closing. On agent shutdown,
we'll forcefully kill all child processes that have not shutdown on
their own.

Issue #3557
mssalvatore added a commit that referenced this issue Aug 15, 2023
@mssalvatore mssalvatore mentioned this issue Aug 15, 2023
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant