Review batch architecture for ETL

- [Instructor] The first Hadoop pipeline architecture…we're going to examine is a traditional one.…This is using Batch Extract, Transform, and Load…so it's not using streaming,…it's not using just in time.…However, it is leveraging some services…and processes in the cloud.…Now, this one happens to be running on the Amazon Cloud…and it's around augmenting healthcare data,…cleaning healthcare data.…And you can see from the associated link,…if you want to read the underlying use case in more detail,…but let's look at the architecture.…So we start with a manifest file…and this is stored in S3 file storage…and that's going to trigger compute or processing…and this is run in the microservices or Lambda architecture.…

Now, that's separate from the whole data world.…It's interesting to note…that Lambdas are being more and more used…in these data pipelines because it's efficient.…It's more efficient than using a virtual machine.…It's really a topic for a separate course,…this whole idea of docker and containers and microservices,…