Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve performance of DaemonPantsRunner exiter #9448

Closed
gshuflin opened this issue Apr 1, 2020 · 0 comments · Fixed by #9722
Closed

Improve performance of DaemonPantsRunner exiter #9448

gshuflin opened this issue Apr 1, 2020 · 0 comments · Fixed by #9722
Assignees
@stuhood

Comments

@gshuflin
Copy link
Contributor

gshuflin commented Apr 1, 2020 •
edited
Loading

Doing profiling on running a pants nailgun client against a warm pantsd, we're spending a lot of time running the DaemonPantsRunner exit method. For instance, see the attached speedscope performance graph taken with sudo py-spy record -p <pantsd_pid> -i -o ./pyspy-output.speedscope --format speedscope, run with the following code:

Effective args: ["--v2", "--no-v1", "--no-v2-ui", "--enable-pantsd", "lint", "src/python/pants/engine::"]
Running with pantsd
Pantsd port: 37963
TIME: 1585782307.393
src/python/pants/engine/objects.py:181:5: F811 redefinition of unused '__getitem__' from line 177

Is v1 set in daemon pants runner: False
All done! ✨ 🍰 ✨
31 files would be left unchanged.

Exiting with: 1

________________________________________________________
Executed in    1.08 secs   fish           external 
   usr time  107.12 millis  716.00 micros  106.40 millis 
   sys time   59.40 millis  197.00 micros   59.20 millis
@gshuflin gshuflin self-assigned this Apr 1, 2020
@stuhood stuhood assigned stuhood and unassigned gshuflin May 7, 2020
@stuhood stuhood mentioned this issue May 10, 2020
Merged
stuhood pushed a commit that referenced this issue May 13, 2020
Port the pantsd nailgun server to rust (#9722)
9506fbf 
### Problem

The setup and teardown of each request made to the nailgun server in `pantsd` had become quite complicated over time... and consequently, slower than it needed to be.

### Solution

Port `pantsd`'s nailgun server to rust using the `nails` crate. Additionally, remove the `Exiter` class, which had accumulated excess responsibilities that can instead be handled by returning `ExitCode` values. Finally, fix a few broken windows including: double logging to pantsd, double help output, closed file errors on pantsd shutdown, and redundant setup codepaths.

### Result

There is less code to maintain, and runs of `./pants --enable-pantsd help` take `~1.7s`, of which `~400ms` are spent in the server. Fixes #9448, fixes #8243, fixes #8206, fixes #8127, fixes #7653, fixes #7613, fixes #7597.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stuhood stuhood

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Port the pantsd nailgun server to rust stuhood/pants
2 participants
@stuhood @gshuflin