AWS S3 in a nutshell — Cheat sheet and compact overview

What’s up with the Buckets?

What is S3?

  • Data is stored as objects
  • Unlimited storage, you don’t need to care about disk space
  • With the S3 console you can easily upload and access your data

Object

  • Key = Name of the object
  • Value = Data stored as byte sequence
  • Version ID= If enabled the current object version
  • Metadata = Additional information about the object

The storage size for an object can be from 0 bytes to 5 terabytes.

S3 Bucket

The Bucket name needs to be unique across all AWS accounts, like a domain.

Storage classes

  • Intelligent Tiering = Stored objects are analyzed via machine learning to decide which storage class is used automatically. In the end the most cost effective tiering is used for your data without any additional overhead
  • Standard Infrequently Accessed = Cheap for objects that don’t get accessed more than once in a month. Amazon applies additional retrieval fee.
  • One Zone Infrequent Access = Objects are stored only in one availability zone with a lower availability(99.5%). Additional retrieval fee is applied. Cheaper than Standard IA.
  • Glacier = Long term cold storage: Cheap and it can take minutes to hours to read objects.
  • Glacier Deep Archive = Cheapest storage, but up to 12 hours for object access.

Security

  • Logging = Every request to a Bucket can be logged. The log files are stored in a different S3 Bucket :), even different AWS account is possible.
  • Access control = Managed by Bucket policies:
Allow read only access for anonymous users:{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"PublicRead",
"Effect":"Allow",
"Principal":"*",
"Action":[
"s3:GetObject"
],
"Resource":[
"arn:aws:s3:::examplebucket/*"
]
}
]
}

You can generate Bucket policies with the AWS policy generator UI: https://awspolicygen.s3.amazonaws.com/policygen.html

Encryption

To encrypt your objects you can do it on the client side and upload already encrpyted files or use server side encryption from AWS:

SSE Encryption by AWS
  • SSE-AES: AWS manages the key and uses the advanced encryption Standard with a block size of 256 bits.
  • SSE-KMS: Envelope encryption, your key is encrypted by another key and you and AWS KMS manage the keys.
  • SSE-C: Client/Customer provides the key

Data consistency

  • Overwrite or Delete (Eventual consistency) = S3 needs to replicate the changes to multiple availability zones. You might get an old object if you read it immediately after writing.

Cross Region Replication

Versioning

  • Overwriting an object creates a new version and you can restore the previous if necessary
  • Deleting an object inserts a delete marker and you get a 404 response if you want to access the object now
  • If versioning is enabled you can only suspend it, so existing versions stay
  • Multi factor authentication is supported to delete a version of an object. Works only with versioning.

Object Lifecycle Management

  • Works with versioning (Use current and/or previous versions)
  • Delete all files after 20 days

Transfer Acceleration

The s3 accelerated domain name looks like this:

bucketname.s3-accelerate.amazonaws.com

Presigned URLs

aws s3 presign s3://awsexamplebucket/test.txt --expires-in 600

The output url will look like this:

https://awsexamplebucket.s3.amazonaws.com/test.txt?AWSAccessKeyId=AKIAEXAMPLEACCESSKEY&Signature=EXHCcBe%EXAMPLEKnz3r8O0AgEXAMPLE&Expires=1555531131

Summary

Software Architect

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store