Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed #5954 Uploading large files with S3 - Multipart Upload #6341

Merged
merged 4 commits into from
Nov 6, 2019
Merged

Fixed #5954 Uploading large files with S3 - Multipart Upload #6341

merged 4 commits into from
Nov 6, 2019

Conversation

mderuijter
Copy link
Contributor

Related Issues

Pull Request Checklist

@coveralls
Copy link

coveralls commented Nov 5, 2019

Coverage Status

Coverage increased (+0.0003%) to 19.461% when pulling c3fabf8 on mderuijter:5954-multipart-upload-support into b46243d on IQSS:develop.

@pdurbin
Copy link
Member

pdurbin commented Nov 5, 2019

@mderuijter thanks for this pull request!

The code looks fine but where you planning to add anything to the guides? Do you need any help with this?

Also, I noticed a failure at https://jenkins.dataverse.org//job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6341/lastCompletedBuild//testReport/junit/edu.harvard.iq.dataverse.api/AccessIT/AccessIT/ but I'm pretty sure this is because your branch is a little behind the "develop" branch. If you merge the latest from "develop" into your branch, you'll pick up pull request #6334, which should fix that error.

Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a comment I mentioned that some docs might be nice. 😄

…-multipart-upload-support

merging latest develop with branch
@mderuijter
Copy link
Contributor Author

mderuijter commented Nov 5, 2019

@mderuijter thanks for this pull request!

The code looks fine but where you planning to add anything to the guides? Do you need any help with this?

Also, I noticed a failure at https://jenkins.dataverse.org//job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6341/lastCompletedBuild//testReport/junit/edu.harvard.iq.dataverse.api/AccessIT/AccessIT/ but I'm pretty sure this is because your branch is a little behind the "develop" branch. If you merge the latest from "develop" into your branch, you'll pick up pull request #6334, which should fix that error.

Thank you @pdurbin, I was wondering what documentation to add and where to put it. A little help would be nice indeed.

I've followed your instructions regarding the jenkins failure and it seems to have passed all the checks.

edit: I spoke to soon, some checks were not successful

@pdurbin
Copy link
Member

pdurbin commented Nov 5, 2019

@poikilotherm do you have the capacity to push some documentation to this branch or at least stub something out so that @mderuijter can fill in the details?

@mderuijter thanks for merging the latest from develop. A new test is failing on build 2 (SwordIT.testCreateDatasetPublishDestroy) but I've seen this from time to time and I think it's unrelated to your branch: https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6341/2/

@pdurbin pdurbin self-assigned this Nov 5, 2019
@poikilotherm
Copy link
Contributor

@pdurbin: sorry, busy with #5974 right now and profiling #6332 / #6052

@pdurbin
Copy link
Member

pdurbin commented Nov 5, 2019

@mderuijter I added a note in https://github.com/mderuijter/dataverse/pull/1 which you are welcome to merge (or not). 😄

I'm moving this to QA. Thanks again!

@pdurbin pdurbin removed their assignment Nov 5, 2019
@djbrooke djbrooke changed the title Fixed #5954 Uploading large files with S3 Fixed #5954 Uploading large files with S3 - Multipart Upload Nov 5, 2019
@kcondon
Copy link
Contributor

kcondon commented Nov 5, 2019

Hi @mderuijter @pdurbin, to activate this feature one would need s3 storage enabled and upload through either the browser or api or this intended for api use only?

@kcondon kcondon self-assigned this Nov 5, 2019
@pdurbin
Copy link
Member

pdurbin commented Nov 5, 2019

@kcondon the code change is in the S3 driver itself which is used by both the UI and the API.

add note: S3 multipart upload support for files over 4GB #5954
@kcondon kcondon merged commit 99fd872 into IQSS:develop Nov 6, 2019
@djbrooke djbrooke added this to the 4.18 milestone Nov 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Issue uploading large files with an S3 backend
6 participants