AWS Classroom Series – 25/Nov/2021

Amazon Elastic File System (EFS)

• EFS provides a file system interface and file system semantics to AWS EC2 instances. When EFS is attached to EC2 instances, it acts just like a local file sytem but EFS is a shared file system which means you can mount the same file system across multiple EC2 instances with low latencies
• Attributes of EFS
  • Fully managed: We don’t need to maintain any hardware or software
  • File System access semantics: We get what we expect from a regular file system including read-after-write consistency locking.
  • File System Interface: It exposes file system interface that works with standard os APIs. EFS appears to be like any other file system to your OS
  • Shared Storage:
  • Elastic and scalable: EFS elastically grows to petabyte scale. We dont have to specify a provisioned size up front, We just create a file system, and it grows and shrinks automatically as we add/remove data
  • Performance: It is built for performance across wide variety of workloads. It provides consistent, low latencies, high throughput and high IOPS
  • High Available & Durable: The data in EFS is automatically replicated across AZs with in a region.
• EFS also works with NFS protocol, Using Direct connect and VPC, we can also mount EFS on your on-premises server via the NFS 4.1 protocol. This a great use case for transferring a large number of data from the servers running on-premise to AWS cloud
• EFS Working process
• Performance Mode of EFS
  • General Purpose Mode:
    • This is default mode for EFS
    • This is optimized for latency-sensitive applications & general purpose file-based workloads
    • This mode is best option for majority of the usecases
  • Max I/O mode
    • This is optimized for large-scale and data-heavy applications, where tens, hunderds or thousands of EC2 instances are accessing the file system
    • This scales to higher level aggregate throughput and operations per second with a trade-off of slightly higher latencies for file operations
• EFS mounts don’t work on Windows Servers so AWS has a service called FSx
• Lets create a Fsx for windows
• Fsx Also supports third party storage servers like Netapp, Lustre

Understand General Purpose Volumes

• Here lets focus on GP2 which are SSD-backed, low cost and default in all regions.

• Lets look at the documented performance characteristics of a GP2 volume

  • Max IOPS/Volume => 16000
  • Max Throughput/Volume => 250 MB/s
  • Max Burstable IOPS => 3000

• The actual max IOPS for any specific volume is based on size 3 IOPS/GB i.e only volume or size 5.3 TB or larger can hit a max IOPS

• Baseline IOPS and Burst Limits

  • Lets consider a smaller volume 100 GB GP2 Volume, at this size the volume has baseline performance of 300 IOPS, but it can burst upto 3000 IOPS using a bucket credit system

• Throughput:

  • IOPS is a tricky metric and you typically care more about throughput-speed at which data comes on and off the volume
  • GP2 Volumes have a throughput limit based on size

  |Volume Size| Throughput Limit | | < 170 GB | 128 MB/s | | 170-334 GB | 250 MB/s (burst) | | >334 GB | 250 MB/s |

• So take, use GP3 volumes with higher throughputs to save costs rather than provisioned IOPS