Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

more error handling #26

Closed
donsizemore opened this issue Jun 10, 2019 · 6 comments · Fixed by #34
Closed

more error handling #26

donsizemore opened this issue Jun 10, 2019 · 6 comments · Fixed by #34
Labels
help wanted Extra attention is needed

Comments

@donsizemore
Copy link
Collaborator

would be great to handle these:

  File "/usr/lib64/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib64/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/usr/lib64/python3.6/http/client.py", line 1392, in connect
    super().connect()
  File "/usr/lib64/python3.6/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib64/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/lib64/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
OSError: [Errno 113] No route to host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "metrics.py", line 9, in <module>
    main()
  File "metrics.py", line 5, in main
    download.main()
  File "/var/www/html/dataverse-metrics/download.py", line 32, in main
    process_single_endpoints(installation, single_endpoints, api_response_cache_dir)
  File "/var/www/html/dataverse-metrics/download.py", line 89, in process_single_endpoints
    process_single_endpoint(installation, endpoint, api_response_cache_dir)
  File "/var/www/html/dataverse-metrics/download.py", line 94, in process_single_endpoint
    response = urlrequest.urlopen(url)
  File "/usr/lib64/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib64/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib64/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 113] No route to host>
@pdurbin
Copy link
Member

pdurbin commented Jun 10, 2019

File "/var/www/html/dataverse-metrics/download.py", line 32, in main

Right. Here: https://github.com/IQSS/dataverse-metrics/blob/v0.2.5/download.py#L32

Thanks for the bug report.

@pdurbin pdurbin mentioned this issue Jun 10, 2019
Open
@donsizemore
Copy link
Collaborator Author

and these:

Traceback (most recent call last):
  File "metrics.py", line 9, in <module>
    main()
  File "metrics.py", line 5, in main
    download.main()
  File "/var/www/html/dataverse-metrics/download.py", line 32, in main
    process_single_endpoints(installation, single_endpoints, api_response_cache_dir)
  File "/var/www/html/dataverse-metrics/download.py", line 89, in process_single_endpoints
    process_single_endpoint(installation, endpoint, api_response_cache_dir)
  File "/var/www/html/dataverse-metrics/download.py", line 94, in process_single_endpoint
    response = urlrequest.urlopen(url)
  File "/usr/lib64/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib64/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib64/python3.6/urllib/request.py", line 1321, in do_open
    r = h.getresponse()
  File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib64/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python3.6/http/client.py", line 279, in _read_status
    raise BadStatusLine(line)
http.client.BadStatusLine: <!doctype html>

@pdurbin
Copy link
Member

pdurbin commented Jun 12, 2019

@donsizemore gotcha. Here:

https://github.com/IQSS/dataverse-metrics/blob/v0.2.5/download.py#L94

Thanks for continuing to report these! Sorry for the trouble! Oh, over in #27 I'm suggesting dropping support for Python 2. Of course, this is not at all a requirement for improving the error handling. It's just tricky with all the requirements I've imposed on myself. In https://github.com/IQSS/dataverse-sample-data I'm experimenting with different requirements: Python 3 only, pip install of dependencies. A different and probably more normal way of doing things instead of (old) "batteries included".

@donsizemore
Copy link
Collaborator Author

one more error code (i'm not concerned about these, just dutifully reporting them).

on python2/3 — any OS should be able to easily run 3.

Traceback (most recent call last):
  File "/usr/lib64/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib64/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/usr/lib64/python3.6/http/client.py", line 1392, in connect
    super().connect()
  File "/usr/lib64/python3.6/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib64/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/lib64/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

@donsizemore
Copy link
Collaborator Author

have I exhausted my repertoire? I think not:

Traceback (most recent call last):
  File "metrics.py", line 9, in <module>
    main()
  File "metrics.py", line 5, in main
    download.main()
  File "/var/www/html/dataverse-metrics/download.py", line 32, in main
    process_single_endpoints(installation, single_endpoints, api_response_cache_dir)
  File "/var/www/html/dataverse-metrics/download.py", line 89, in process_single_endpoints
    process_single_endpoint(installation, endpoint, api_response_cache_dir)
  File "/var/www/html/dataverse-metrics/download.py", line 94, in process_single_endpoint
    response = urlrequest.urlopen(url)
  File "/usr/lib64/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib64/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway

@pdurbin pdurbin mentioned this issue Jul 10, 2019
Closed
@pdurbin pdurbin added the help wanted Extra attention is needed label Oct 25, 2019
@donsizemore
Copy link
Collaborator Author

i've tasted blood, and i want more (more! more! more!)

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:877)>

donsizemore added a commit to uncch-rdmc/dataverse-metrics that referenced this issue Oct 28, 2019
@donsizemore
IQSS#26 improve error handling, disable SSL verification
898b739
@donsizemore donsizemore mentioned this issue Oct 28, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

26 more error handling uncch-rdmc/dataverse-metrics
2 participants
@pdurbin @donsizemore