load_krebsregister() fails with [SSL: CERTIFICATE_VERIFY_FAILED] #42

mkdeak · 2017-09-06T09:26:11Z

Hi,

Following the readthedocs documentations, I get the following error when I try to load the krebs register:

>>> krebs_data, krebs_match = load_krebsregister(missing_values=0)
Start downloading the data.
Issue with downloading the data: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/Library/Python/3.6/lib/python/site-packages/recordlinkage/datasets/external.py", line 75, in load_krebsregister
    data = pandas.concat([_krebsregister_block(bl) for bl in block])
  File "/Users/user/Library/Python/3.6/lib/python/site-packages/recordlinkage/datasets/external.py", line 75, in <listcomp>
    data = pandas.concat([_krebsregister_block(bl) for bl in block])
  File "/Users/user/Library/Python/3.6/lib/python/site-packages/recordlinkage/datasets/external.py", line 123, in _krebsregister_block
    compression='zip')
  File "/Users/user/Library/Python/3.6/lib/python/site-packages/pandas/io/parsers.py", line 655, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/Users/user/Library/Python/3.6/lib/python/site-packages/pandas/io/parsers.py", line 405, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/user/Library/Python/3.6/lib/python/site-packages/pandas/io/parsers.py", line 764, in __init__
    self._make_engine(self.engine)
  File "/Users/user/Library/Python/3.6/lib/python/site-packages/pandas/io/parsers.py", line 985, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/Users/user/Library/Python/3.6/lib/python/site-packages/pandas/io/parsers.py", line 1605, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 394, in pandas._libs.parsers.TextReader.__cinit__ (pandas/_libs/parsers.c:4209)
  File "pandas/_libs/parsers.pyx", line 664, in pandas._libs.parsers.TextReader._setup_parser_source (pandas/_libs/parsers.c:8001)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/zipfile.py", line 1082, in __init__
    self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/user/Library/Python/3.6/lib/python/site-packages/recordlinkage/datasets/krebsregister/block_1.zip'

If there's an easy way to fix this, I'd be happy to create PR with the fix.

The text was updated successfully, but these errors were encountered:

J535D165 · 2017-09-06T20:22:23Z

Thanks for reporting this issue. Quite some info on the internet about this problem (https://access.redhat.com/articles/2039753).

I did change https:// to http:// in commit 30312c0. Does this solve your issue?

twilight2001 · 2017-11-06T05:30:31Z

I also have problem with this line, but I get a "ValueError: Unrecognised compression type: zip" error instead ... the error is different from the above: (this is on Win 7 Pro, 64 bit, running Anaconda)

krebs_data, krebs_match = load_krebsregister(missing_values=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\user\AppData\Roaming\Python\Python27\site-packages\recordlinkag
e\datasets\external.py", line 75, in load_krebsregister
    data = pandas.concat([_krebsregister_block(bl) for bl in block])
  File "C:\Users\user\AppData\Roaming\Python\Python27\site-packages\recordlinkag
e\datasets\external.py", line 123, in _krebsregister_block
    compression='zip')
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 529, in parse
r_f
    return _read(filepath_or_buffer, kwds)
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 295, in _read

    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 612, in __ini
t__
    self._make_engine(self.engine)
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 747, in _make
_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 1119, in __in
it__
    self._reader = _parser.TextReader(src, **kwds)
  File "pandas\parser.pyx", line 353, in pandas.parser.TextReader.__cinit__ (pan
das\parser.c:3246)
  File "pandas\parser.pyx", line 567, in pandas.parser.TextReader._setup_parser_
source (pandas\parser.c:5787)
ValueError: Unrecognized compression type: zip

J535D165 · 2017-11-06T09:21:02Z

Can you show the output of

import pandas
pandas.show_versions()

It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")?

twilight2001 · 2017-11-06T10:06:35Z

Hi Jon, Thanks for your follow up again, we tested it on Linux and everything works down to ECM algorithm. My goal was to get some estimates of M and U with frequency dependence to verify our scoring (we have one-to-one matching data, about 1 million records of reference address)... right now our computed numbers of m and u looks crazy... The following is pandas version info from Windows/Anaconda 2 latest version: pandas: 0.18.0 nose: 1.3.7 pip: 9.0.1 setuptools: 20.3 Cython: 0.23.4 numpy: 1.10.4 scipy: 1.0.0 statsmodels: 0.6. xarray: None IPython: 4.1.2 sphinx: 1.3.5 patsy: 0.4.0 dateutil: 2.5.1 pytz: 2016.2 blosc: None bottleneck: 1.0.0 tables: 3.2.2 numexpr: 2.5 matplotlib: 1.5.1 openpyxl: 2.5.0a2 xlrd: 0.9.4 xlwt: 1.0.0 xlsxwriter: 0.8.4 lxml: 3.6.0 bs4: 4.4.1 html5lib: None httplib2: None apiclient: None sqlalchemy: 1.0.1 pymysql: None psycopg2: None jinja2: 2.8 boto: 2.39.0 best regards Geoff

…

On 6 November 2017 at 17:21, Jonathan de Bruin ***@***.***> wrote: Can you show the output of import pandas pandas.show_versions() It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALDDbhMAMk6mkzYTlJzIEbsreaAO2epSks5szs9_gaJpZM4POFjF> .

twilight2001 · 2017-11-06T10:39:52Z

Yes, something is wrong with my winzip (seems overridden by 7z or winrar), need reboot to test again ...

…

On 6 November 2017 at 17:21, Jonathan de Bruin ***@***.***> wrote: Can you show the output of import pandas pandas.show_versions() It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALDDbhMAMk6mkzYTlJzIEbsreaAO2epSks5szs9_gaJpZM4POFjF> .

twilight2001 · 2017-11-06T15:05:36Z

Hi Jonathan, After reinstalling winzip v12, pandas can read csv files in my working directory now. But rerunning krebs_data, krebs_match = load_krebsregister(missing_values=0) I still get ValueError: Unrecognized compression type: zip regards Geoff

…

On 6 November 2017 at 17:21, Jonathan de Bruin ***@***.***> wrote: Can you show the output of import pandas pandas.show_versions() It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALDDbhMAMk6mkzYTlJzIEbsreaAO2epSks5szs9_gaJpZM4POFjF> .

twilight2001 · 2017-11-07T04:27:24Z

Hi Jonathan, Wonder if you consider it feasible to build this hybrid model on top of the toolkit: known as the cocktail approach, it combines probabilistic (EM) and rule based approaches in a flow. First, use EM to figure out what variables are most relevant, then build rules on top of such variables (see figure from Shah) : http://digitalcommons.calpoly.edu/theses/1353/ http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.218.9234&rep=rep1&type=pdf Shah also added relational linkage information to improve accuracy which is very interesting... best regards Geoff

…

On 6 November 2017 at 17:21, Jonathan de Bruin ***@***.***> wrote: Can you show the output of import pandas pandas.show_versions() It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALDDbhMAMk6mkzYTlJzIEbsreaAO2epSks5szs9_gaJpZM4POFjF> .

Michael-E-Rose · 2018-01-29T22:03:30Z

Problem prevails with recordlinkage version 0.11.2

>>> from recordlinkage.datasets import load_krebsregister
>>> krebs_data, krebs_match = load_krebsregister(missing_values=0)
Start downloading the data.
Issue with downloading the data: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/krebsregister/block_10.zip'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/external.py", line 75, in load_krebsregister
    data = pandas.concat([_krebsregister_block(bl) for bl in block])
  File "/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/external.py", line 75, in <listcomp>
    data = pandas.concat([_krebsregister_block(bl) for bl in block])
  File "/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/external.py", line 123, in _krebsregister_block
    compression='zip')
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 709, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 449, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 818, in __init__
    self._make_engine(self.engine)
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 1049, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 1695, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 402, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 664, in pandas._libs.parsers.TextReader._setup_parser_source
  File "/usr/lib/python3.5/zipfile.py", line 1009, in __init__
    self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/krebsregister/block_1.zip'

J535D165 · 2018-01-29T22:14:20Z

Hello, this is a different issue. You do not have permission to write /usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/krebsregister/block_10.zip.

Is it possible to get write access?

Michael-E-Rose · 2018-01-30T22:09:10Z

Yes it is, I invoked python3 with sudo. I wasn't aware of what the problem was in this case.

Thanks for the help!

Dragut · 2019-03-27T04:06:17Z

I encountered the same problem when doing the load:
krebs_X, krebs_true_links = load_krebsregister(missing_values=0)
Error Message:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\s59018\AppData\Roaming\Python\Python36\site-packages\recordlinkage\datasets\krebsregister\block_1.zip'
Please help.

J535D165 · 2019-03-27T07:47:29Z

This is a permission error. Do you have permission to the given folder?

I will work on a permanent solution.

Dragut · 2019-03-27T11:10:19Z

I rerun it successfully from home rather than in the office. Looks like a connection problem which fails to download the data completely.

J535D165 closed this as completed Jan 4, 2018

J535D165 reopened this Mar 27, 2019

J535D165 mentioned this issue Mar 29, 2019

Make use of ~/rl_data for storing large files #92

Merged

J535D165 closed this as completed in a604560 Jul 12, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

load_krebsregister() fails with [SSL: CERTIFICATE_VERIFY_FAILED] #42

load_krebsregister() fails with [SSL: CERTIFICATE_VERIFY_FAILED] #42

mkdeak commented Sep 6, 2017 •

edited

Loading

J535D165 commented Sep 6, 2017

twilight2001 commented Nov 6, 2017 •

edited by J535D165

Loading

J535D165 commented Nov 6, 2017

twilight2001 commented Nov 6, 2017 via email

twilight2001 commented Nov 6, 2017 via email

twilight2001 commented Nov 6, 2017 via email

twilight2001 commented Nov 7, 2017 via email

Michael-E-Rose commented Jan 29, 2018

J535D165 commented Jan 29, 2018

Michael-E-Rose commented Jan 30, 2018

Dragut commented Mar 27, 2019

J535D165 commented Mar 27, 2019

Dragut commented Mar 27, 2019

load_krebsregister() fails with [SSL: CERTIFICATE_VERIFY_FAILED] #42

load_krebsregister() fails with [SSL: CERTIFICATE_VERIFY_FAILED] #42

Comments

mkdeak commented Sep 6, 2017 • edited Loading

J535D165 commented Sep 6, 2017

twilight2001 commented Nov 6, 2017 • edited by J535D165 Loading

J535D165 commented Nov 6, 2017

twilight2001 commented Nov 6, 2017 via email

twilight2001 commented Nov 6, 2017 via email

twilight2001 commented Nov 6, 2017 via email

twilight2001 commented Nov 7, 2017 via email

Michael-E-Rose commented Jan 29, 2018

J535D165 commented Jan 29, 2018

Michael-E-Rose commented Jan 30, 2018

Dragut commented Mar 27, 2019

J535D165 commented Mar 27, 2019

Dragut commented Mar 27, 2019

mkdeak commented Sep 6, 2017 •

edited

Loading

twilight2001 commented Nov 6, 2017 •

edited by J535D165

Loading