Thoughts on S3

Simple...Not

The first S in S3 was supposedly for the word simple. This has been a massive failure, although it has gotten better. S3 can do so much that it is actually hard to wrap your head around it all at once. First, a few esoteric things you need help from other services to accomplish: you can use FSx for Lustre to mount the S3 bucket as an NFS drive and read and write objects as if it were any other NAS (Fuse does this too). You could also use the Transfer Family to set up an SFTP server right on top of the bucket, all you need to supply is a public SSH key and a username. But what else? S3 can host objects written by countless AWS services (reports, logs, and audits). S3 can act as a simple web server (HTTP only for custom domains, but there is always an HTTPS endpoint you could use) but also implement complex redirect rules. S3 can act as the origin for CloudFront to serve up files to the public (which is how this website originally worked before the Lambda@Edge conversion). S3 can have at least a dozen permission models - who owns the objects in the bucket, who pays for them, is it the same as the bucket owner, signed URLs for temporary gated upload and download capabilities. Objects can have metadata directly attached or written to DynamoDB. Objects can be encrypted using multiple different systems, and will soon be encrypted by default (although this is basically checkbox encryption to satisfy an audit or control requirement, not truly effective encryption, but still, a start). If you want to provide cross-account access, you'll need a bucket policy attached, or you can turn on bucket level ACLs and provide grants to other principals, either inside AWS or even by email address. You can version the objects, such that deleting one merely writes a delete marker to the object, but previous versions remain indefinitely, sometimes without you even knowing. Buckets can have lifecycle policies to move the objects to different storage tiers to save money (the Intelligent Tiering Glacier Long Term Storage class is approximately 85% cheaper than the standard price). You can trigger notifications to SNS topics or even directly invoke Lambdas when objects are created, updated, or destroyed, allowing you to manage complex workflows and lifecycles however your business demands. I am probably missing at least half a dozen additional useful features, but you get the idea. It is anything but simple.

Simple Rules

All that said, here is the summary worth taking away: