ResourceWarning: unclosed ssl.SSLSocket

[csvan]

Self-Hosted Version

23.12.1

CPU Architecture

x86_64

Docker Version

24.0.5

Docker Compose Version

2.24.0

Steps to Reproduce

1. Setup a vanilla Sentry instance with s3 backing.
2. Create a React project
3. Enable Replays in the React SDK

Expected Result

Nothing unusual should be logged

Actual Result

Logs are being spammed by the following:

ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=24, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 52310), raddr=('52.218.28.211', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=37, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 59034), raddr=('52.218.90.115', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=24, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 55108), raddr=('52.218.31.3', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=24, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 33574), raddr=('52.218.116.200', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=32, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 38056), raddr=('52.218.57.211', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=36, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 56038), raddr=('52.218.101.155', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=37, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 49380), raddr=('52.92.33.16', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=24, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 49478), raddr=('52.218.100.91', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=24, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 59050), raddr=('52.218.90.115', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=24, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 52316), raddr=('52.218.28.211', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback
ingest-replay-recordings-1                      | /usr/local/lib/python3.10/site-packages/sentry/models/files/abstractfile.py:295: ResourceWarning: unclosed <ssl.SSLSocket fd=38, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.0.38', 55124), raddr=('52.218.31.3', 443)>
ingest-replay-recordings-1                      |   blob = self.FILE_BLOB_MODEL.from_file(blob_fileobj, logger=logger)
ingest-replay-recordings-1                      | ResourceWarning: Enable tracemalloc to get the object allocation traceback

This does NOT happen when using regular file storage, so it seems somehow related to the s3 backing.

Event ID

No response

[aldy505]

Can you share your sentry/sentry.conf.py or sentry/config.yml that contains some settings related to the S3 backend? Please omit the credentials with **** for obvious reasons.

Also are you connecting to AWS's S3? Or some self-hosted instance of S3-compatible API?

If you're connecting to AWS S3, how's the connection configuration? Is it..

1. through IAM roles attached to the EC2 (or Lightsail) instance?
2. through plain access key + secret key + token combo?
3. through credentials placed in ~/.aws directory?

[csvan]

@aldy505 our only s3 config is in sentry/config.yml:

filestore.backend: 's3'
filestore.options:
  bucket_name: 'bucket.name'
  default_acl: 'private'

[aldy505]

@csvan What about the authentication? I haven't touch AWS S3 in a long time, so I can't really reproduce what you're facing. But.. you can find additional configurations (keys on filestore.options) here: https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings

My guess from the error is the connection to S3 is not properly terminated. The solution would be to make sure your EC2 and S3 instance is within the same AZ, or tweak your network settings on sysctl. You can see some here:https://www.ibm.com/docs/en/linux-on-systems?topic=recommendations-network-performance-tuning and https://easyengine.io/tutorials/linux/sysctl-conf/

[csvan]

@aldy505 seems rather to be an issue in Python itself (or rather urllib3)

boto/boto3#454 (comment)

[azaslavsky]

If this is a CPython issue, as indicated by the linked comment, it seems like there's not much for us to do at the moment. It seems like the ideal solution is for this to get fixed upstream in Python itself, and then we'll eventually ingest the fix as we upgrade.

The logic described in the linked comment makes sense to me as well: we shouldn't filter for these errors on our end.