Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process polling #1853

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

Process polling #1853

wants to merge 27 commits into from

Conversation

igankevich
Copy link

@igankevich igankevich commented Dec 28, 2024

This PR adds process polling capabilities via Process type that implements Source.

  • On Linux the polling is done via pidfd.
  • On MacOS/BSD it is done via EVFILT_PROC filter of kqueue.
  • On Windows it is done via job objects.

To implement Process I made the following changes.

  • No changes to epoll-based and poll-based selectors: pidfd is like any other file descriptor.
  • Minor changes to kqueue-based selector and related is_* functions to support new EVFILT_PROC filter and prevent code repetition.
  • Multiple changes to Windows selector. This selector now supports registering arbitrary handles to be polled. Handle's address is used as the token (lpCompletionKey), and each selector maintains a mapping between the handles and the associated data HandleInfo. For processes the associated data consists of user-provided token. Currently only processes are registered as handles, but everything else can be re-implemented using handles to simplify the code (e.g. Arc<CompletionPort> will no longer be needed in NamedPipe and it will be possible to register the same pipe in several selectors). Please let me know if this is something you want to be done in this PR.
  • tests/process.rs test.
  • Changes to tests/util module to support large number of events.

Besides process polling this PR includes fixes suggested by the latest Clippy version.

EDIT

The motivation behind non-blocking process polling is to simplify projects that use mio to collect output from child processes (e.g. test harnesses, virtual networks). Currently such projects either use a separate thread to wait for child processes via wait syscall (that waits for any process to exit) or poll with small timeout and check whether the processes exited after each poll. The former needs mutexes to synchhorinze the waiting thread with the poller, and the latter is as efficient as active polling. With proper process polling such projects would use the same thread to monitor process status, this would greatly simplify their code bases.

EDIT 2

I tested the changes on Linux, MacOS, FreeBSD and Windows using the same commands as in CI and latest Rust version.

@igankevich
Copy link
Author

Fixed (hopefully) the errors reported in CI. Had to use -Zexport-executable-symbols to make leak sanitizer work in CI (rust-lang/rust#111073).

@igankevich
Copy link
Author

igankevich commented Dec 30, 2024

Thanks for approving the pipeline. Hopefully I fixed the flaky test :)

It seems that leak sanitizer doesn't work with panicking tests, so I disabled them when compiled under this sanitizer. The error that we get here might be related to a custom allocator. I'm not sure mio has something to do with this error. I couldn't find any Rust-related issue with the same error.

EDIT

I couldn't reproduce leak sanitizer error in the Docker container with Ubuntu 24.04 and Rust nightly. Not sure why it's happening in the CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant