Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generalize README.md #1185

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Generalize README.md #1185

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 8 additions & 53 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## S3cmd tool for Amazon Simple Storage Service (S3)
## S3cmd tool for the Amazon Simple Storage Service Protocol (S3)

[![Build Status](https://travis-ci.org/s3tools/s3cmd.svg?branch=master)](https://travis-ci.org/s3tools/s3cmd)

Expand All @@ -21,55 +21,19 @@ See [installation instructions](https://github.com/s3tools/s3cmd/blob/master/INS

### What is S3cmd

S3cmd (`s3cmd`) is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc.
S3cmd (`s3cmd`) is a free command line tool and client for uploading, retrieving and managing data in any of the 60+ cloud storage service providers that use the S3 protocol. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc.

S3cmd is written in Python. It's an open source project available under GNU Public License v2 (GPLv2) and is free for both commercial and private use. You will only have to pay Amazon for using their storage.

Lots of features and options have been added to S3cmd, since its very first release in 2008.... we recently counted more than 60 command line options, including multipart uploads, encryption, incremental backup, s3 sync, ACL and Metadata management, S3 bucket size, bucket policies, and more!

### What is Amazon S3
### What is S3

Amazon S3 provides a managed internet-accessible storage service where anyone can store any amount of data and retrieve it later again.
S3 is a protocol, first created by Amazon, that allows you to talk to a managed internet-accessible storage service where anyone can store any amount of data and retrieve it later again.

S3 is a paid service operated by Amazon. Before storing anything into S3 you must sign up for an "AWS" account (where AWS = Amazon Web Services) to obtain a pair of identifiers: Access Key and Secret Key. You will need to
give these keys to S3cmd. Think of them as if they were a username and password for your S3 account.
Before storing anything using S3 you must sign up for an account to obtain a pair of identifiers: Access Key and Secret Key. You will need to give these keys to S3cmd. Think of them as if they were a username and password for your S3 account.

### Amazon S3 pricing explained

At the time of this writing the costs of using S3 are (in USD):

$0.026 per GB per month of storage space used

plus

$0.00 per GB - all data uploaded

plus

$0.000 per GB - first 1GB / month data downloaded
$0.090 per GB - up to 10 TB / month data downloaded
$0.085 per GB - next 40 TB / month data downloaded
$0.070 per GB - next 100 TB / month data downloaded
$0.050 per GB - data downloaded / month over 150 TB

plus

$0.005 per 1,000 PUT or COPY or LIST requests
$0.004 per 10,000 GET and all other requests

If for instance on 1st of January you upload 2GB of photos in JPEG from your holiday in New Zealand, at the end of January you will be charged $0.06 for using 2GB of storage space for a month, $0.0 for uploading 2GB of data, and a few cents for requests. That comes to slightly over $0.06 for a complete backup of your precious holiday pictures.

In February you don't touch it. Your data are still on S3 servers so you pay $0.06 for those two gigabytes, but not a single cent will be charged for any transfer. That comes to $0.06 as an ongoing cost of your backup. Not too bad.

In March you allow anonymous read access to some of your pictures and your friends download, say, 1500MB of them. As the files are owned by you, you are responsible for the costs incurred. That means at the end of March you'll be charged $0.06 for storage plus $0.045 for the download traffic generated by your friends.

There is no minimum monthly contract or a setup fee. What you use is what you pay for. At the beginning my bill used to be like US$0.03 or even nil.

That's the pricing model of Amazon S3 in a nutshell. Check the [Amazon S3 homepage](http://aws.amazon.com/s3/pricing/) for more details.

Needless to say that all these money are charged by Amazon itself, there is obviously no payment for using S3cmd :-)

### Amazon S3 basics
### S3 basics

Files stored in S3 are called "objects" and their names are officially called "keys". Since this is sometimes confusing for the users we often refer to the objects as "files" or "remote files". Each object belongs to exactly one "bucket".

Expand Down Expand Up @@ -106,7 +70,7 @@ It is a good idea to use DNS-compatible bucket names. That for instance means yo

Look for "Virtual Hosts" later in this text for more details regarding FQDN named buckets.

### Objects (files stored in Amazon S3)
### Objects (files stored in S3)

Unlike for buckets there are almost no restrictions on object names. These can be any UTF-8 strings of up to 1024 bytes long. Interestingly enough the object name can contain forward slash character (/) thus a `my/funny/picture.jpg` is a valid object name. Note that there are not directories nor buckets called `my` and `funny` - it is really a single object name called `my/funny/picture.jpg` and S3 does not care at all that it _looks_ like a directory structure.

Expand All @@ -126,16 +90,7 @@ Alternatively the ACL can be altered for existing remote files with `s3cmd setac

### Simple s3cmd HowTo

1) Register for Amazon AWS / S3

Go to http://aws.amazon.com/s3, click the "Sign up for web service" button in the right column and work through the registration. You will have to supply your Credit Card details in order to allow Amazon charge you for S3 usage. At the end you should have your Access and Secret Keys.

If you set up a separate IAM user, that user's access key must have at least the following permissions to do anything:
- s3:ListAllMyBuckets
- s3:GetBucketLocation
- s3:ListBucket

Other example policies can be found at https://docs.aws.amazon.com/AmazonS3/latest/dev/example-policies-s3.html
1) Register for an account on an S3 storage-provider

2) Run `s3cmd --configure`

Expand Down