Discuss some best practices while building data transport modules within the big data architecture.
- [Instructor] In this video, we will look at some…of the best practices of predictive analytics…in the big data architecture world.…We will leave the finer details of model building…to data scientists.…Being architects, we will focus upon how the big data…architectures we build can help data scientists…build models and use these models to predict efficiently.…Big data architects should care about prediction efficiency.…Data scientists should care about prediction effectiveness.…
Scalability is an important architectural…component in predictions.…It should be possible to run predictions…on a number of transactions simultaneously,…and they should be easily scalable.…Keep the prediction process as asynchronous as possible.…This is because when the process is asynchronous,…we only need to bother about average response times…and not worry about the peaks.…Perform benchmark tests with real production data loads…and see if the prediction times obtained…will fulfill the goals for the use case.…
Also benchmark user response times…
There is no coding involved. Instead you will see how big data tools can help solve some of the most complex challenges for businesses that generate, store, and analyze large amounts of data. The use cases are drawn from a variety of industries, including ecommerce and IT. Instructor Kumaran Ponnambalam shows how to analyze a problem, draw an architectural outline, choose the right technologies, and finalize the solution. After each use case, he reviews related best practices for real-time streaming, predictive analytics, parallel processing, and pipeline management. Each lesson is rich in practical techniques and insights from a developer who has experienced the benefits and shortcomings of these technologies firsthand.
- Components of a big data application
- Big data app development strategies
- Use cases: fraud detection and product recommendations
- Technology options
- Designing solutions
- Best practices