Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue where large stdin writes can cause Tornado to hang #189

Merged
merged 6 commits into from
Sep 29, 2022

Conversation

KoopaKing
Copy link
Contributor

A fix for #183

This includes switching to using a dedicated executor for blocking IO, specifically wrapping calls to ptyprocess.write which, can hang when the PTY buffer (the size of which is set by the OS) is exceeded. TermSocket users can provide their own thread pool by using the named blocking_io_executor parameter on initialization, but by default a shared, process-wide single threaded executor will be used.

This also includes a regression test which will fail on Debian and OSX (but may be able to pass on some OSs due to the variance of the PTY buffer's size).

This also switched the default filetype of terminado/tests/basic_test.py from dos to unix to match with the rest of the files in this package and prevent some editors from auto-inserting carriage returns on save.

@KoopaKing
Copy link
Contributor Author

Ah, seems like my assertion about the output size doesn't work on windows. Probably windows doesn't have echo, idk, I'm not a windower. I'll update this PR in a bit to fix this.

@blink1073 blink1073 added the bug label Sep 28, 2022
@blink1073
Copy link
Contributor

I'd say skip that test on Windows.

@KoopaKing
Copy link
Contributor Author

Yeah, I'm doing that real quick, also doing a quick change to the API to move the blocking_io_executor parameter to the TermManager. It seems like TermManager already exposes a shutdown method for controlling lifecycles, so it would be nice to have the lifecycle of autocreated executors managed there. Should have this updated in a few minutes...

@KoopaKing KoopaKing force-pushed the issue183 branch 3 times, most recently from 1c74a8d to 8d24c24 Compare September 28, 2022 23:07
The dedicated executor will be auto created by the manager, or can be
specified at manager creation time if the caller wants to control the
lifecycle. Also adds regression test for large input/output to the
terminal and switches basic_test.py's fileformat to unix from dos so
that it matches with the rest of the source code.
@KoopaKing
Copy link
Contributor Author

Ok, for real though this is my last time trying to make the linter happy...

@KoopaKing KoopaKing force-pushed the issue183 branch 2 times, most recently from b696dd2 to d671779 Compare September 28, 2022 23:36
@blink1073
Copy link
Contributor

I was a bit concerned since technically making on_message a coroutine it is a breaking change. I added a downstream test for jupyter_server_terminals.

@blink1073
Copy link
Contributor

I think we're good, those tests pass. I tried testing spyder-terminal but it has a complicated setup for testing, and it overrides on_message anyway.

Copy link
Contributor

@blink1073 blink1073 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, thank you!

@blink1073 blink1073 merged commit 7210e82 into jupyter:main Sep 29, 2022
@blink1073
Copy link
Contributor

I'll cut a release with this tomorrow.

@blink1073
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants