Learn how to finesse the earlier architecture blueprint for the social media sentiment analysis use case with technology options and finer details to arrive at the final architecture.
- [Instructor] Let us now fine tune…the outline we laid out…for social media sentiment analysis architecture.…We start off with the real-time subscriber for Twitter.…We will build a custom subscriber application for Twitter.…It can be built with Java or Python.…The list of hashtags to monitor…will be configurable in this application.…You can create separate subscription threads…for each of the hashtags.…All posts that has received…will then routed to this queue.…
We will build a real-time subscriber for Facebook.…Actually both Twitter and Facebook subscribers…can be built within the same JVM…with different threads monitoring…different platforms and hashtags.…The real-time streaming queue…will be built on Kafka.…There will be one single Kafka topic…into which all posts from all social media platforms…will be pushed in after standardization…of the message.…The text mining engine will be built in Apache Spark.…
Apache Spark can scale within a cluster.…The number of Spark and Kafka partitions…should be the same.…The sentiment analysis engine…
There is no coding involved. Instead you will see how big data tools can help solve some of the most complex challenges for businesses that generate, store, and analyze large amounts of data. The use cases are drawn from a variety of industries, including ecommerce and IT. Instructor Kumaran Ponnambalam shows how to analyze a problem, draw an architectural outline, choose the right technologies, and finalize the solution. After each use case, he reviews related best practices for real-time streaming, predictive analytics, parallel processing, and pipeline management. Each lesson is rich in practical techniques and insights from a developer who has experienced the benefits and shortcomings of these technologies firsthand.
- Components of a big data application
- Big data app development strategies
- Use cases: fraud detection and product recommendations
- Technology options
- Designing solutions
- Best practices