Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuring stream reader buffer limit. #82

Open
PeterQFR opened this issue Oct 24, 2024 · 2 comments
Open

Configuring stream reader buffer limit. #82

PeterQFR opened this issue Oct 24, 2024 · 2 comments

Comments

@PeterQFR
Copy link
Contributor

Whilst passing images around via PyTAK, I ran into a limitexception on the rxreader generated from the protocol factory.

This generally happens if the asyncio.readuntil doesn't reach the delimiter before the limit is reached.

It doesn't seem PyTak is enabling this to be configured and its set at the asyncio defaults of 2 ** 16 or 64KB. https://github.com/python/cpython/blob/b61fece8523d0fa6d9cc6ad3fd855a136c34f0cd/Lib/asyncio/streams.py#L23

I was able to increase this by increasing the limit directly in pytak/client_functions.py line 220 by specifying the limit. Here I set it to 4GB, I haven't run into the limit error.
eg

reader, writer = await asyncio.open_connection(host, port, limit=(2 ** 32)

Would it be supported to increase this limit or make it configurable?

@ampledata
Copy link
Collaborator

@PeterQFR thanks for this. My usual preference is to set magic numbers as constants, and even then make them configurable, so in this instance lets set a const for 2**32 and come up with a config variable as such (never know when we'll want to tune, fuzz or tweak).

@PeterQFR
Copy link
Contributor Author

PeterQFR commented Feb 6, 2025

No problems, I think the other issue regarding the arbitary sleep probably contributed to this. I'll push some code following the generic config path and go from there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants