Learn how to finesse the earlier architecture blueprint for the mobile couponing use case with technology options and finer details to arrive at the final architecture.
- [Instructor] Let us now detail the architecture…based on the technology options we discussed…in the earlier video.…We will build a custom mobile gateway solution.…This gateway should be horizontally scalable…and capable of processing multiple requests in parallel…and push them into Kafka.…We will use Kafka for the real-time queue.…Given that the sequencing of location information…coming into the queue is not important,…we can create multiple Kafka partitions…and individual Spark subscribers into each partition.…
This will help horizontally scale the solution.…We will use Apache Spark…as the coupon recommendation engine.…Spark in this case is a mere data querying…and matching engine,…but it helps in scaling the solution.…The recommendations should be queried and decided…within map operations so that the work is distributed…amongst Spark partitions.…The coupon queue would also use Kafka.…This queue will have a smaller load…since not all location information coming in…will find coupon matches.…
There is no coding involved. Instead you will see how big data tools can help solve some of the most complex challenges for businesses that generate, store, and analyze large amounts of data. The use cases are drawn from a variety of industries, including ecommerce and IT. Instructor Kumaran Ponnambalam shows how to analyze a problem, draw an architectural outline, choose the right technologies, and finalize the solution. After each use case, he reviews related best practices for real-time streaming, predictive analytics, parallel processing, and pipeline management. Each lesson is rich in practical techniques and insights from a developer who has experienced the benefits and shortcomings of these technologies firsthand.
- Components of a big data application
- Big data app development strategies
- Use cases: fraud detection and product recommendations
- Technology options
- Designing solutions
- Best practices