Aws Classroom Series – 18/Dec/2020

S3 Durability and Availability (S3 Storage Classes)

  • S3 offers multiple storage classes for objects, which can be chosen depending on durability, availability and cost

  • S3 measure durability and availability in Percentages

  • Example: 99.99999999999 percent durability means there is a chance of 0.00000000001% of objects might be lost i.e. you can expect one object to be lost every 10,000 years

  • S3 has two factors that will impact cost

    • Storage size
    • Data access costs Preview
  • Storage Classes in AWS

    • S3 Standard:
      • Durability: 99.99999999999%
      • Concurrent facility fault tolerance: 2
      • Availability: 99.99%
      • Access cost is less
      • Storage cost is more
    • S3 Standard-IA (Infrequent Access)
      • Durability: 99.99999999999%
      • Concurrent facility fault tolerance: 2
      • Availability: 99.9%
      • Access cost is more
      • Storage cost is less
    • S3 One Zone-IA
      • Durability: 99.99999999999%
      • Concurrent facility fault tolerance: 1
      • Availability: 99.5%
      • Access cost is more
      • Storage cost is less
    • Reduced Redundancy
      • Durability: 99.99%
      • Concurrent facility fault tolerance: 1
      • Availability: 99.99%
    • Glacier:
      • Archival Storage which we will not be access
      • To access this storage we need convert the storage class from Glacier to any other (Standard/Infrequent-Access)
    • Intelligent Tier:
      • As time progresses we might want our objects to be moved from one storage class to other. So we need to move the data from one class to other, aws does this automatically depending on your access
  • Lets examine costs using aws pricing calculator Refer Here

  • Scenario-1: We have uploaded videos of 10 TB which are used frequently, Atleast 1000 users will be watching videos in a month which contributes to atleat 1000 TB of access Preview Preview

  • Scenario -2: We have uploaded old documents of size 100 TB which are used in-frequently, at a max you can expect 1 TB of Data to be viewed (Select, Get) and 0 Put

  • Scenario-3: We have uploaded archival data (Which we will never access) of size 100 TB

  • If we don’t use intelligent tiering we can set rules for moving data from one storage class to other using lifecycle rules

  • Scenario: Lets upload a video/image to s3 bucket. Our estimation is this file will be accessed frequently for 60 days and then our file might not be accessed frequently for 180 days and after 180 days we might not even access the file Preview Preview Preview

  • Exercise-1:

    • Create a Bucket with three folders
      • Music
      • Videos
      • Images
    • Ensure you have atleast one file in each folder
    • For images set the storage class to intelligent-tier
    • For videos set the storage class to standard
    • For music set the storage to reduced redundancy while creating
  • Exercise-2:

      • Create a Bucket with three folders
      • Music
      • Videos
      • Images
    • Ensure you have atleast one file in each folder
    • Create a lifecycle rule to move objects to One-Zone IA after 200 days and after 300 days move to glacier.

Scenario: LT-document management

  • LT is building a solution where customers can upload documents. LT will internally store the documents in S3 bucket
  • This product should support uploading & accessing multiple versions of documents
  • Some of the documents should be made available to public, where as some documents should be private.
  • The documents can be accessed by thousands of users parallely, some will be reading some will be writing documents, how can we ensure the documents read by all the users parallelly show the same content.
  • Solution: To solve the above Scenario we need to understand
    • Access levels
    • Eventually Consistent Data
    • Bucket Versioning
    • Prefixes and Delimiters

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About learningthoughtsadmin