Explore key business benefits of building applications using serverless architecture patterns, and see an example from bioinformatics.
- [Instructor] So what are the benefits of serverless? Well, it boils down to agility and cost savings. You should be able to manage quicker, smaller deployments so more frequent deployments. I have lots of customers that can do A/B Feature Testing, push out a version of a feature to a subset of the population because it's a smaller set of features that have to be pushed rather than a monolithic push. Pay only for what you use, you can reduce service costs really substantially, particularly around core compute as we saw in previous movies.
Simplified scalability and management. If you don't have to set up, for example, around compute in case of Amazon EC2, load balancers and configure scaling via auto-scaling configuration, the people that would be doing that can be freed up to work on providing value to your customers by looking at different aspects of the application. Now does that require re-training sometimes? Yeah, it does. But what I have found with teams is if they can be involved with the new architecture and learn how the services work, they can provide value pretty quickly.
Additional benefits with serverless is the vendor, in most cases, your public cloud vendor will manage your availability, uptime, scalability, and patching. And if you adopt a microservices architecture, you will use a combination of your functions and you will also use vendor services so you will write less code. As I mentioned, serverless is not just Lambda and S3, if you're thinking about Amazon for example. Serverless is really becoming tens of even hundreds of serverless services around storage, compute, and other services.
In particular, I'm seeing a number of services be made available across the vendor spectrum around advanced data processing and visualization. So everything from aggregation of data that's sitting in an object store via services like Amazon Redshift Spectrum all the way to Visualization On Demand with services like Power BI for Microsoft Azure. One of the serverless service areas that I've done a lot of work with and I think is not very well understood in the industry and it provides a lot of value to my customers is serverless SQL.
So Google was actually the pioneer here with Google BigQuery and just as a reference example, here's a genomics query that I worked with where four terabytes of data was processed and this was stored in an object store, it happened to be the BigQuery object store, 85 million rows was aggregated down to 2,500 rows in around 30 seconds. And this was less than $50 U.S. to process. No servers, no clustering, no indexing, no query optimization, this is a area of serverless that I think many, many customers will be interested to explore as the volumes of data grow and in fact, in this course, we have a section of architectures that focus on big data pipelines because that's where a lot of my work has been focused and a lot of my customers may getting value.
So for an example of this, a group that I've been working with in Australia, CSIRO bio informatics team built this serverless architecture for genomic variant information after they attended an AWS submmit. So this is an example of a small team, they're between five and 10 people, it depends on how many grad students they have working with them and they looked at this new architecture and they said, hmm, we have a bursty workload that has large amounts of data and varying amounts of compute and we do not have the resources to stand up or manage servers so this seems like a great use case.
This is the actual architecture they built in the Amazon Ecosystem and it very much reflects that first reference architecture I showed you earlier in this course. They used S3 object store for file persistence, they used Dynamo DB for a semi-structured data and they used a number of purpose-driven Lambdas for the compute on both the file and the semi-structured data. They made this available for a public website via the API Gateway. Now this architecture although in many ways standard was pretty revolutionary for their industry.
In fact, so much so that Amazon actually recognized them with a blog post. So you can see this is already two years old and this is on Jeff Barr's blog and after they built this solution, it's won a number of awards and most importantly it's been used very widely in the industry. And because it was built sort of classically serverlessly if there is such a thing, they have had virtually no service cost, I think they're still on the free tier, given their amount of usage. So this is a great story, not only for the bio informatics industry but I think it's a great reference use case around the idea of a small team, a startup team starting with no cost and going 100% cloud all in on cloud, all in on serverless and delivering value to their customers really, really quickly and easily.
- Defining serverless architecture
- Serverless services
- Serverless functions
- Cloud-native serverless architecture
- Serverless architecture for big data and machine learning
- Emergent serverless architecture