Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not set Content-MD5 in S3. #5226

Merged
merged 3 commits into from
Aug 15, 2024
Merged

Conversation

teo-tsirpanis
Copy link
Member

SC-52300

This PR removes setting the Content-MD5 header on S3 uploads. There are multiple reasons for this:

  • The header is not supported on directory buckets used by S3 Express One Zone.
  • Computing the MD5 hash of the data immediately before uploading them has a performance overhead and does not provide additional security.
  • We don't do a similar thing in the other cloud VFSes.

Validated by:

  • Creating, writing, consolidating and vacuuming an array on a directory bucket.
  • Running quickstart_dense_cpp on a Google Cloud Storage bucket using the S3 compatibility API.

TYPE: BUG
DESC: Fix incompatibilities with S3 Express One Zone, by stopping setting Content-MD5 on all S3 uploads.

@KiterLuc KiterLuc merged commit 5b92634 into TileDB-Inc:dev Aug 15, 2024
59 checks passed
@teo-tsirpanis teo-tsirpanis deleted the s3-no-md5 branch August 19, 2024 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants