In this video, Jeff Winesett talks about AWS object storage S3 and Glacier. S3 and Glacier are ideal for storing data that have high durability and reliability needs.
- [Instructor] Amazon offers different storage options to accommodate the different types of data needed to be stored in the cloud. Amazon EFS offers file storage, Amazon EBS and EC2 instance storage offers block storage, and Amazon S3 and Glacier offer object storage. S3 is an object store. In an object store, objects are stored in a flat organization which are referred to as buckets in S3.
These objects are retrieved by a unique identifier called keys. A key can be any string and can be constructed to mimic hierarchical attributes, meaning that it can sort of be made to look like a directory structure but it should not be confused with one. Objects are really any blob of data, such as backups and archives, log files, videos, images and audio files. This is typically contrasted with a traditional file storage system where files are organized in a directory hierarchy and housed in a central repository of metadata about the files.
This metadata and structure is then used to search on and retrieve the information. In object storage, objects are stored in a non-hierarchical flat, address space which allows for almost endless scalability and very fast object retrieval. Of course, there are some feature trade-offs with object storage compared with file storage. For example, S3 does not natively offer the ability to search across all objects in storage.
S3 is built to be incredibly durable. This is achieved by redundantly storing objects on multiple devices across multiple availability zones in an Amazon S3 region. For non-critical data that may not require these extremely high levels, Amazon also offers reduced redundancy storage, or RRS. This is an option which reduces storage costs and can be used for objects that could easily be reproduced in the event of a failure.
For example, RRS could be a cost-saving option when storing multiple sizes of an image from an original or different video encodings from an original video source. Store the original on S3 and use RRS for the thumbnails. S3 provides, in theory, unlimited storage capacity. It will scale and grow as demanded by your storage needs. There is no defined limit to the amount of content you can store.
There is, however, a five terabyte limit on the size each object can be but not a limit on the number of these objects you can store. Objects can be accessed via a Rest API which makes it easy to implement in almost any application. One thing to note about S3 is that S3 object storage works on an eventual consistency model. While the objects stored in S3 are highly available and highly durable, the data consistency is achieved only eventually.
The high availability and durability is achieved by replicating the data across devices both within an availability zone and across availability zones. Whenever an update is made to an object there is a period of time until the change is propagated to all of the replicas before requests will return the latest version. Take, for example, deleting an object in S3. It may take some time to propagate to all the replicas and therefore there is a brief period of time where objects could still be returned even after a delete request has been made.
So, while object storage is a great fit for data that doesn't change much, like backups and archives, log files, video images and audio files, this eventual consistency can make object storage unsuitable for data that changes frequently. Another object storage option available within AWS is Amazon Glacier. The Glacier storage is like an extension of S3 but is intended for archiving data, for data that will be accessed very infrequently and where retrieval time of several hours is acceptable.
The same extremely high durability is guaranteed with Glacier just as with S3 but the storage costs are much less. Objects in Glacier are managed through S3. Objects are not associated or managed with Glacier storage directly upon upload. Rather, existing Amazon S3 objects are transitioned to Glacier when the data is ready for archiving. Implementing elasticity rule number three: take advantage of the available object storage options in AWS to improve scalability, availability, durability and performance of storing and accessing application assets.
- Benefits of cloud services
- Making architectures scalable
- Examining cloud constraints
- Virtual servers, EC2, and Elastic IP
- Using the Amazon machine image
- Elastic load balancing
- Using CloudWatch for monitoring
- Security Models
- Elastic block storage
- S3, CloudFront, and Elastic Beanstalk
- Handling queues, workflows, and notifications
- Caching options and services
- Identity and access management
- Creating a custom server image
- Application deployment strategies
- Serverless architectures