Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python{,-experimental}] Obey floating point timeouts provided to RESTClientObject.request(...) #7154

Merged
merged 2 commits into from
Aug 10, 2020
Merged

[python{,-experimental}] Obey floating point timeouts provided to RESTClientObject.request(...) #7154

merged 2 commits into from
Aug 10, 2020

Conversation

cjgibson
Copy link
Contributor

@cjgibson cjgibson commented Aug 7, 2020

This PR allows passing of a fractional number of seconds as a timeout when making network requests through the RESTClientObject. Currently timeout durations provided as a float are silently ignored instead of being enforced:

RESTClientObject timeout behavior
>>> # Google uses the OpenAPI generator for their Python client
>>> from kubernetes.client.configuration import Configuration
>>> from kubernetes.client.rest import RESTClientObject
>>> rest_client_object = RESTClientObject(Configuration())
>>>
>>> # unexpectedly returns a response
>>> rest_client_object.request('GET', 'http://neverssl.com', _request_timeout=1e-9)
<kubernetes.client.rest.RESTResponse object at 0x7fd6a968cbd0>
>>>
>>> # fails with a timeout error, as we'd expect
>>> rest_client_object.request('GET', 'http://neverssl.com', _request_timeout=(1e-9, 1e-9))
Traceback (most recent call last):
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connection.py", line 167, in _new_conn
    % (self.host, self.timeout),
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7fd6a96a5050>, 'Connection to neverssl.com timed out. (connect timeout=1e-09)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/kubernetes/client/rest.py", line 214, in request
    headers=headers)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/request.py", line 76, in request
    method, url, fields=fields, headers=headers, **urlopen_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/request.py", line 97, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/poolmanager.py", line 336, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='neverssl.com', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fd6a96a5050>, 'Connection to neverssl.com timed out. (connect timeout=1e-09)'))

The underlying call to urllib3.PoolManager.request will happily accept a float timeout duration, as shown:

urllib3 timeout behavior
>>> # urllib3's PoolManager provides the underlying impelementation for
>>> # OpenAPI's RESTClientObject
>>> from urllib3 import PoolManager, Timeout
>>>
>>> # the following is what we'd expect when passing 1e-9 as our _request_timeout
>>> pool_manager = PoolManager()
>>> pool_manager.request('GET', 'http://neverssl.com', timeout=Timeout(total=1e-9))
Traceback (most recent call last):
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connection.py", line 167, in _new_conn
    % (self.host, self.timeout),
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7fd6a96a5790>, 'Connection to neverssl.com timed out. (connect timeout=1e-09)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/request.py", line 76, in request
    method, url, fields=fields, headers=headers, **urlopen_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/request.py", line 97, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/poolmanager.py", line 336, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/cjgibson/miniconda3/envs/Users.cjgibson.code.rapid/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='neverssl.com', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fd6a96a5790>, 'Connection to neverssl.com timed out. (connect timeout=1e-09)'))

PR checklist

  • Read the contribution guidelines.
  • Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
  • If contributing template-only or documentation-only changes which will change sample output, build the project beforehand.
  • Run the shell script ./bin/generate-samples.shto update all Petstore samples related to your fix. This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master. These must match the expectations made by your contribution. You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*. For Windows users, please run the script in Git BASH.
  • File the PR against the correct branch: master
  • Copy the technical committee to review the pull request if your PR is targeting a particular programming language.

Fixes: kubernetes-client/python#1069

@spacether

@auto-labeler
Copy link

auto-labeler bot commented Aug 7, 2020

👍 Thanks for opening this issue!
🏷 I have applied any labels matching special text in your issue.

The team will review the labels and make any necessary changes.

@cjgibson cjgibson changed the title [python] Obey floating point timeouts provided to RESTClientObject.request(...) [python{,-experimental}] Obey floating point timeouts provided to RESTClientObject.request(...) Aug 7, 2020
@cjgibson cjgibson requested a review from spacether August 7, 2020 19:13
@spacether spacether added this to the 5.0.0 milestone Aug 10, 2020
@spacether
Copy link
Contributor

That error is unrelated to python or python-experimental because it is raised from petstore-okhttp-gson server testing

Copy link
Contributor

@spacether spacether left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR! Looks good to me.

@spacether spacether merged commit d37216a into OpenAPITools:master Aug 10, 2020
@cjgibson cjgibson deleted the cjgibson/allow-float-timeout branch August 11, 2020 17:27
@cjgibson
Copy link
Contributor Author

Awesome - thanks for the quick feedback and review here @spacether! Much appreciated. 😁

@spacether
Copy link
Contributor

You're welcome! Thank you for the fix 👍

jimschubert added a commit that referenced this pull request Aug 12, 2020
* master: (27 commits)
  [WIP][python-exp] Force camelization of imports (#7186)
  Fixes #6942: Added ability to prepend a basePath to typescript-redux-query generators (#6943)
  [Typescript] Import path is invalid in windows. (#7175)
  Fix JaxRS Spec generator additional model types (#7180)
  [python{,-experimental}] Obey floating point timeouts provided to RESTClientObject.request(...) (#7154)
  [C#] Switch the spec to OAS v3 from v2 (#7176)
  [Javascript] Fixing the handling of non primitive types in paramToString (#7171) (#7172)
  [typescript-node] Fix invalid type when using node@10 and ES5 (#7133)
  Minor fix to github workflow badge
  [gradle] Enabling up-to-date checks and gradle caching for openapigenerator tasks (#6716)
  feat(csharp-netcore): Adding response headers to the ApiException (#7169)
  [ci] Verify supported JDK versions on master push (#7085)
  Issue #6830: Java server - Add getter to ApiException templates (#7150)
  update kotlin samples
  [Kotlin] Make ApiClient in jvm-retrofit2 be able to use own OkHttpClient (#6999)
  Sttp - wrap query params (#6884)
  Add a link to https://medium.com/@everisBrasil blog post (#7160)
  [C#][netcore] fix regular expression when it contains double quotes (#7147)
  remove duplicated cancellationToken in comment (#7148)
  update samples
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants