AWS Classroom Series – 01/Sept/2020

Simple Storage Service (S3)

  • Amazon S3 lets you store & retrieve unlimited amounts of data from anywhere in the world at any time.
  • S3 can be used to store any kind of file.
  • S3 to store
    • Log files
    • Videos
    • Music
    • Images
  • S3 can be used to host static websites (Websites with html & javascript & CSS)
  • S3 stores blocks of raw data.
  • Files in S3 are called as objects
  • Objects are grouped together as buckets
  • S3 bucket can hold objects(files) and folders
  • Individual object(file) can be upto 5TB in size.
  • S3 stores objects as key value pairs. Key is object name & value is your file.
  • Lets try to create S3 bucket from AWS Console
  • Navigate to s3 from AWS Services and click on create bucket Preview
  • Name of the bucket should be globally unique. Preview
  • Bucket is scoped to a region. AWS gives option to replicate buckets to other regions. This option is called as cross region replication.
  • Lets create the following structure
    • three folders music, videos and images
    • upload one mp3 file in music which become an s3 object Preview Preview
    • similarly upload a video file into videos and an image in images folder
  • S3 gives an https url to access the contents. (By default s3 will not allow public access) Preview
  • Lets try to access this location Preview
  • We need to understand how to give access to all or specific individuals. Preview
  • So to understand S3 better we need to know more details

S3 Storage Classes

  • S3 is a service for storing files and the billing of s3 has two factors
    • size of the data
    • access costs
  • Durability: Property which describes that the files are not lost or corrupted
  • Availability: Property which determines the availability of files over https urls
  • Understanding Durability:
    • Lets assume you have 100,000,000,000 objects with 99.999999999% durability
    • What does this mean?
      • This means you could expect to lose only 1 (0.000000001)% of objects over the year
    • Lets assume you have 100,000,000,000 objects with 99.99% durability, you might loose 10,000,000 objects in a year
  • Understanding Availability:
    • Availability is percent of time an object will be available for accessing.
  • The levels of availability and durability of an object depends on its storage class
  • S3 Storage Classes:
    • STANDARD:
      • Durability: 99.999999999% (nine nines)
      • Availability: 99.99%
      • AZ’s: >2 (Data is copied into 2 AZ’s)
      • Cost of GB/month: 0.023$ (Us-west-2 region)
    • STANDARD_IA:
      • Designed for infrequent access. Here storage cost will be less and access cost will be more
      • Durability: 99.999999999%
      • Availability: 99.9%
      • AZ’s: >2 (Data is copied into 2 AZ’s)
      • Cost of GB/month: 0.0125$ (Us-west-2 region)
    • ONEZONE_IA:
      • Designed for infrequent & non critical data
      • Durability: 99.999999999%
      • Availability: 99.5%
      • AZ’s: 1 (Data is copied into 2 AZ’s)
      • Cost of GB/month: 0.01$ (Us-west-2 region)
    • Reduced Redundancy:
      • Durability: 99.99%
      • Availability: 99.99%
      • AZ’s: >2 (Data is copied into 2 AZ’s)
      • Cost of GB/month: 0.024$ (Us-west-2 region)
    • GLACIER:
      • For archival storage, you cannot access the data directly
      • Durability: 99.999999999%
      • Availability: varies
      • AZ’s: >2 (Data is copied into 2 AZ’s)
      • Cost of GB/month: 0.004$ (Us-west-2 region)
  • Storage Classes for frequently accessed data
    • STANDARD
    • REDUCED_REDUNDANCY
  • Storage Classes for infrequently accessed data
    • STANDARD_IA
    • ONEZONE_IA
  • FOR Archival:
    • GLACIER
  • IF YOU ARE NOT SURE
    • INTELLIGENT_TIERING
  • Request and Data Retrieval costs Preview

Access Permissions

  • S3 offers the following methods to control access to objects
    • Bucket policies:
      • Resource-based policy that you apply to a bucket.
      • You can use bucket policies to grant access to all objects in a bucket or specific objects.
      • You can also control which users can read, write and delete objects
      • For this we need to create json based policies
    • User Policies:
      • Policies for Identity and Access Management (IAM) to S3 buckets
    • Bucket and object access control lists
      • This is legacy access control methods to give
        • anonymous read access to all
        • grant specific access to IAM Prinicipals

Note

AWS S3 bucket creation from cli

  • Ensure AWS CLI is configured
  • Refer Here for cli docs
  • The s3 url format is s3://<bucketname>, so to create s3 bucket
aws s3 mb s3://<bucket-name>
  • To view the buckets use
aws s3 ls
  • To remove the bucket
aws s3 rb s3://<bucket-name>

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About learningthoughtsadmin