Python 3.10/11 SSL: SSLV3_ALERT_HANDSHAKE_FAILURE

[valentinedwv]

Not fatal, but just an issue note:

Seems there is a issue with 3.10/3.11
python/cpython#103142

Mac Intel

and containers using
FROM python:3.11-slim
FROM python:3.10-slim

This url:

https://opentopography.org/sitemap.xml

gets redirected to:

https://portal.opentopography.org/sitemap.xml

If i just use https://portal.opentopography.org/sitemap.xml it works fine.

File "/Users//development/dev_earthcube/earthcube_utilities/venv311/lib/python3.11/site-packages/advertools/sitemaps.py", line 491, in sitemap_to_df
    xml_text = urlopen(Request(sitemap_url, headers=headers))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)>

[eliasdabbas]

Thanks a lot for reporting @valentinedwv !

I can also see the same behavior on a Mac, also on Google Colab (Linux).

Not sure if there is something I can do here?
I'll check.
Thanks again.

[caroheymes]

Hi, it seems that I have the same problem. I am trying to run a scraper on AWS sagemaker studio.

python version : Python 3.10.6
advertools==0.13.4
aiohttp==3.8.5
aiosignal==1.3.1

2023-08-16 16:21:18 [twisted] CRITICAL: Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/opt/conda/lib/python3.10/site-packages/scrapy/crawler.py", line 101, in crawl self.engine = self._create_engine() File "/opt/conda/lib/python3.10/site-packages/scrapy/crawler.py", line 115, in _create_engine return ExecutionEngine(self, lambda _: self.stop()) File "/opt/conda/lib/python3.10/site-packages/scrapy/core/engine.py", line 83, in __init__ self.downloader = downloader_cls(crawler) File "/opt/conda/lib/python3.10/site-packages/scrapy/core/downloader/__init__.py", line 83, in __init__ self.middleware = DownloaderMiddlewareManager.from_crawler(crawler) File "/opt/conda/lib/python3.10/site-packages/scrapy/middleware.py", line 59, in from_crawler return cls.from_settings(crawler.settings, crawler) File "/opt/conda/lib/python3.10/site-packages/scrapy/middleware.py", line 40, in from_settings mwcls = load_object(clspath) File "/opt/conda/lib/python3.10/site-packages/scrapy/utils/misc.py", line 61, in load_object mod = import_module(module) File "/opt/conda/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 688, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/opt/conda/lib/python3.10/site-packages/scrapy/downloadermiddlewares/retry.py", line 27, in <module> from scrapy.core.downloader.handlers.http11 import TunnelError File "/opt/conda/lib/python3.10/site-packages/scrapy/core/downloader/handlers/http11.py", line 23, in <module> from scrapy.core.downloader.contextfactory import load_context_factory_from_settings File "/opt/conda/lib/python3.10/site-packages/scrapy/core/downloader/contextfactory.py", line 11, in <module> from scrapy.core.downloader.tls import DEFAULT_CIPHERS, openssl_methods, ScrapyClientTLSOptions File "/opt/conda/lib/python3.10/site-packages/scrapy/core/downloader/tls.py", line 23, in <module> METHOD_SSLv3: SSL.SSLv3_METHOD, # SSL 3 (NOT recommended) AttributeError: module 'OpenSSL.SSL' has no attribute 'SSLv3_METHOD'

Thanks a lot for your help !

[valentinedwv]

Try running on python 3.09

[caroheymes]

@valentinedwv Thanks ! Finally I did
!pip uninstall -y scrapy !pip install Scrapy==2.6.3
and everything works fine !

[eliasdabbas]

Thanks for sharing the solution @caroheymes !