Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perform connection check using head request if http_proxy ist set #434

Merged
merged 1 commit into from
Jul 30, 2021
Merged

Perform connection check using head request if http_proxy ist set #434

merged 1 commit into from
Jul 30, 2021

Conversation

LarsHaalck
Copy link
Contributor

I guess this is an edge case, but I was trying to run maestral's cli on a server where connections are only possible through a proxy. The Python SDK from Dropbox uses the requests python package which handles environment variables like http_proxy or similar accordingly.

Maestral's check_connection function uses sockets to establish a connection with dropbox.com over Port 80 which was not possible on my machine. This PR uses a HEAD request (which uses the *_proxy environment variables) if http_proxy is set and uses the existing socket method otherwise.

@samschott
Copy link
Owner

Thanks for catching this, we should definitely respect proxy settings!

Is there any reason not to default to a head request instead of using socket.create_connection()? I'm not that comfortable with the "http_proxy" in os.environ check, it probably misses several cases such as https_proxy or upper-case environment variables.

Finally, could you add a line to CHANGELOG.md?

@LarsHaalck
Copy link
Contributor Author

I think the lower case variant is the most commonly used, but I guess checking for both cases would better. The requests package seems to support both versions as well (see docs).

In my quick tests, the socket version took around 11ms, whereas the head request version was about 26ms in case of a successful and around 1ms and 3ms in case of an unsuccessful connection respectively (both cases without the use of a proxy). When using a proxy the latter case is of course slower and dependents on the proxy.

But because it says "low-latency" check in the comment, I was hesitant to replace the complete method with this head request variant.

I just added a line to the changelog, but I just saw that the CI failed and I'm not sure why.

@samschott
Copy link
Owner

samschott commented Jul 29, 2021

But because it says "low-latency" check in the comment, I was hesitant to replace the complete method with this head request variant.

The "low latency" statement is a bit misleading, it is low latency only because of its default timeout of 2 sec. This is compared to requests from the sync threads which have a timeout of 100 sec. The reason for such a long timeout is to enable uploads / downloads even when the internet connection is flaky.

check_connection on the other hand is called every 10 sec. Depending on its return value:

  1. If it returns True and sync threads were stopped from a previous ConnectionError, they are resumed.
  2. If it returns False, the displayed status is changed to "Connecting...". However, sync threads are not stopped. Instead, they will raise a ConnectionError in their own time, from their own timeout value.

I therefore don't mind the slow overhead of requests vs socket if it results in simpler code.

I just added a line to the changelog, but I just saw that the CI failed and I'm not sure why.

The tests are currently a bit flaky. First, spawning a new process sometimes hangs when run from pytest on macOS (pytest meddles with stdin / stdout). Second, test_unix_permissions sometimes fails due to a bug in the watchdog dependency which results in missing notifications for file metadata changes on macOS. It's already patched upstream and I expect a new release soon. Feel free to ignore those failures...

@samschott
Copy link
Owner

You should however pass linting. We use flake8 and black to enforce a uniform code style and mypy for static type checking.

Connection checking is now done by using head requests to respect proxy
settings set by http_proxy and https_proxy.
@LarsHaalck
Copy link
Contributor Author

Okay I just replaced the socket method completely, squashed the commits and checked for linting.

Copy link
Owner

@samschott samschott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks for the improvements!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants