AWS Classroom Series – 25/Nov/2021

Amazon Elastic File System (EFS)

  • EFS provides a file system interface and file system semantics to AWS EC2 instances. When EFS is attached to EC2 instances, it acts just like a local file sytem but EFS is a shared file system which means you can mount the same file system across multiple EC2 instances with low latencies
  • Attributes of EFS
    • Fully managed: We don’t need to maintain any hardware or software
    • File System access semantics: We get what we expect from a regular file system including read-after-write consistency locking.
    • File System Interface: It exposes file system interface that works with standard os APIs. EFS appears to be like any other file system to your OS
    • Shared Storage:
    • Elastic and scalable: EFS elastically grows to petabyte scale. We dont have to specify a provisioned size up front, We just create a file system, and it grows and shrinks automatically as we add/remove data
    • Performance: It is built for performance across wide variety of workloads. It provides consistent, low latencies, high throughput and high IOPS
    • High Available & Durable: The data in EFS is automatically replicated across AZs with in a region.
  • EFS also works with NFS protocol, Using Direct connect and VPC, we can also mount EFS on your on-premises server via the NFS 4.1 protocol. This a great use case for transferring a large number of data from the servers running on-premise to AWS cloud
  • EFS Working process Preview
  • Performance Mode of EFS
    • General Purpose Mode:
      • This is default mode for EFS
      • This is optimized for latency-sensitive applications & general purpose file-based workloads
      • This mode is best option for majority of the usecases
    • Max I/O mode
      • This is optimized for large-scale and data-heavy applications, where tens, hunderds or thousands of EC2 instances are accessing the file system
      • This scales to higher level aggregate throughput and operations per second with a trade-off of slightly higher latencies for file operations Preview
  • EFS mounts don’t work on Windows Servers so AWS has a service called FSx
  • Lets create a Fsx for windows Preview
  • Fsx Also supports third party storage servers like Netapp, Lustre

Understand General Purpose Volumes

  • Here lets focus on GP2 which are SSD-backed, low cost and default in all regions.

  • Lets look at the documented performance characteristics of a GP2 volume

    • Max IOPS/Volume => 16000
    • Max Throughput/Volume => 250 MB/s
    • Max Burstable IOPS => 3000
  • The actual max IOPS for any specific volume is based on size 3 IOPS/GB i.e only volume or size 5.3 TB or larger can hit a max IOPS

  • Baseline IOPS and Burst Limits

    • Lets consider a smaller volume 100 GB GP2 Volume, at this size the volume has baseline performance of 300 IOPS, but it can burst upto 3000 IOPS using a bucket credit system
  • Throughput:

    • IOPS is a tricky metric and you typically care more about throughput-speed at which data comes on and off the volume
    • GP2 Volumes have a throughput limit based on size

    |Volume Size| Throughput Limit | | < 170 GB | 128 MB/s | | 170-334 GB | 250 MB/s (burst) | | >334 GB | 250 MB/s |

  • So take, use GP3 volumes with higher throughputs to save costs rather than provisioned IOPS

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About learningthoughtsadmin