Get an S3 overview.
- [Instructor] Launched in March of 2006 S3, or Simple Storage Service, is one of Amazon's longest running AWS offerings. It's purpose is to provide easy-to-use cloud storage that is scalable, highly available, and reliable. Before we start, it's important to know that S3 isn't actually a file system in the traditional sense. Rather, S3 provides what is known as Object Storage. What distinguishes Object Storage from File Storage? Well storage devices come in a few flavors having to do with the level at which storage systems manage data.
Block Storage operates at the level of disc, blocks and sectors and this is the lowest level to the hardware. Sectors originally refer to actual sectors on the circular magnetic discs in platter based hard drives. In the age of solid stay drives sectors are more of a logical construct, but the concept remains the same. Block Storage is generic and can be formatted to apply a file system on top of it. File Storage is probably what you're most familiar with. A File Storage device uses a file system, software that manages files in directory structures sometimes with permission bits that describe who can read, write and list the files.
Examples include FAT32 and NTFS for Windows, HSF for MAC, and EXT4 for Linux based systems. Object Storage is a bit different. An object might be a file from your computer or service file system, but once placed into Object Storage it's an object. Data with metadata attached. In S3 this metadata often takes the form of tags, the same kind of tags that appear in resources throughout AWS. In Objects go into Buckets.
A Bucket is called that because you can basically throw anything you want into it. In a real bucket there's no structure, there're no shelves. Likewise, an S3 Bucket has no native concept of folders or hierarchies. It's all flat name space for objects inside. Of course, unlike a real bucket S3 Buckets can hold a virtually unlimited amount of objects. S3 can easily scale to hold billions of objects. It's also very important to understand that unlike with a file system, your server can't natively mount to Object Storage.
Rather, your host will talk to S3 of the restful API's through an SDK like the ones for Python or Java or using some abstraction layer. For instance, Ruby applications can use a gem called FOG to abstract S3 and treat it more like a file system. There are also hardware and software gateways that can provide this abstraction for an entire server such as AWS's File Gateway. In the case of File Gateway the middle layer actually does provide an NFS mount so your instance can take advantage of S3 storage while still talking to a typical file system.
Back to Buckets. The name space for Buckets is global, which means that each time you create a Bucket you'll need to come up with a name that is unique worldwide. Object names within a Bucket can contain letters, digits, or a certain special characters including slashes. These object names are sometimes referred to as Keys. Speaking of slashes and so there's no concept of folders in Object Storage and that's true. However, S3 goes a long way toward pretending that there is. Take a look at these two object names.
Knowing about the flat name space concept, you may recognize that these are just object names that happen to share some of the same characters. With S3 we'll visually represent them as two objects within a Files folder. This convenience is a bit confusing when you know the truth, but in practice it rarely causes issues. In fact, the folder metaphor runs pretty deep depending on which interface you use. These days the web console even has a Create Folder button and it doesn't even let you set key names with embedded slashes.
So, what distinguishes S3 from any other Object Storage? In the next video we'll get into S3's incredible reliability, security and data management features.
Join AWS architect Brandon Rich and learn how to configure object storage solutions and lifecycle management in Simple Storage Service (S3), a web service offered by AWS, and migrate, back up, and replicate relational data in RDS. Find out how to leverage flexible network storage with Elastic File System (EFS), and use the new AWS Glue service to move and transform data. Plus, learn how Snowball can help you transfer truckloads of data in and out of the cloud.
- What is data management?
- AWS S3 basics
- S3 bucket creation
- S3 upload and logging
- S3 event notifications
- S3 data lifecycle configuration
- Working with Amazon Elastic Block Store volumes
- Creating and mounting an EFS
- Creating an AWS RDS instance
- RDS backup and recovery
- Moving data with AWS Database Migration Service
- Moving data with Data Pipeline and Glue