Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 0.4.0 #1488

Merged
merged 15 commits into from
Jul 28, 2015
Merged

S3 0.4.0 #1488

merged 15 commits into from
Jul 28, 2015

Conversation

whyrusleeping
Copy link
Member

PR'ing @tv42's s3 branch against dev0.4.0

@whyrusleeping whyrusleeping added the status/in-progress In progress label Jul 16, 2015
@tv42
Copy link
Contributor

tv42 commented Jul 16, 2015

Uhh #1261 much?

@tv42
Copy link
Contributor

tv42 commented Jul 16, 2015

Oh I see that pull request is against master; it's that old.

@tv42 tv42 mentioned this pull request Jul 16, 2015
@whyrusleeping
Copy link
Member Author

@tv42 yeah, no worries! we've been putting off merging this for way too long.

@whyrusleeping whyrusleeping force-pushed the s3-0.4.0 branch 2 times, most recently from f06caf3 to 2dc3ab7 Compare July 17, 2015 23:09
@jbenet
Copy link
Member

jbenet commented Jul 21, 2015

@whyrusleeping not sure why the tests failed. also looks like it needs to be rebased?

otherwise LGTM

tv42 and others added 15 commits July 27, 2015 13:23
The generated file went through some changes because of differing
go-bindata versions.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
This gives us a clean slate for the new code, avoiding leftovers.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
Earlier, it also checked checked the leveldb directory. That part
added no crash safety to the application, and just hardcoded
assumptions about the datastore.

If anything, this should rely on the absolute last item created by
fsrepo.Init, and there should be fsync guarantees about ordering.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
…m for S3

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
…"private"

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
To test it, set up an S3 bucket (in an AWS region that is not US
Standard, for read-after-write consistency), run `ipfs init`, then
edit `~/.ipfs/config` to say

      "Datastore": {
        "Type": "s3",
        "Region": "us-west-1",
        "Bucket": "mahbukkit",
        "ACL": "private"
      },

with the right values. Set `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` in the environment and you should be able to
run `ipfs add` and `ipfs cat` and see the bucket be populated.

No automated tests exist, unfortunately. S3 is thorny to simulate.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
@whyrusleeping
Copy link
Member Author

@jbenet can we merge this?

@jbenet
Copy link
Member

jbenet commented Jul 28, 2015

this is failing go tests on mac pretty reliably. 0/6 (3x2) successes

jbenet added a commit that referenced this pull request Jul 28, 2015
@jbenet jbenet merged commit 25fd6b9 into dev0.4.0 Jul 28, 2015
@jbenet jbenet removed the status/in-progress In progress label Jul 28, 2015
@jbenet jbenet deleted the s3-0.4.0 branch July 28, 2015 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants