S3 performance and data transfer: SOA-C03 study notes and practice questions
What to know about S3 performance and data transfer for the SOA-C03 exam, from the Monitoring, Logging, Analysis, Remediation, and Performance Optimization domain (22% of the exam guide), then 1 free question to test it. Part of the SOA-C03 study guide.
These questions ask how to make S3 uploads, downloads or request rates faster, or how to move a lot of data.
Uploads and downloads
Multipart upload splits a file into parts that upload in parallel and retry individually. Required above 5 GB (single PUT limit); recommended from about 100 MB. The AWS CLI uses it automatically for large files.
Add a lifecycle rule to abort incomplete multipart uploads, or orphaned parts keep billing.
S3 Transfer Acceleration sends data through the nearest edge location and the AWS network; it helps distant clients. Bucket names must be DNS-compliant without dots, and clients use the accelerate endpoint.
Byte-range fetches download parts of an object in parallel, or only the part you need.
Request rates
S3 supports at least 3,500 write and 5,500 read requests per second per prefix, with no limit on prefixes. If an application gets 503 Slow Down errors, spread keys across more prefixes and retry with backoff.
Moving data
Tool
Use
AWS DataSync
Online transfers from NFS, SMB, HDFS, other clouds or between AWS storage, with scheduling, verification and bandwidth limits
AWS Snowball Edge
Offline transfer of terabytes to petabytes when the network is too slow
S3 Batch Operations
Run an action (copy, restore, tag, invoke Lambda) on millions of existing objects
S3 Replication
Copy new objects to another bucket or Region automatically
Watching performance
S3 request metrics in CloudWatch (per bucket, opt-in) show latency, errors and request counts. S3 Storage Lens gives organization-wide usage and activity.
Common traps
Transfer Acceleration for repeated downloads of the same files. Use CloudFront.
Retrying a failed 50 GB upload in one PUT. It can’t work; use multipart.
DataSync vs Snowball. Enough bandwidth points to DataSync; weeks of transfer time points to Snowball.
Large files over long distances need two fixes. Multipart upload sends parts in parallel and retries only what fails, and it is required above 5 GB. Transfer Acceleration shortens the slow public internet part of the path by entering the AWS network at a nearby edge location. Prefixes, storage classes and versioning do not address either problem.
Want more on S3 performance and data transfer? The 5 full-length practice sets have 5 more S3 performance and data transfer questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →