Discover how to build a real-time stream processing pipeline with Apache Fink. Learn about the platform's windowing, event-time processing, and state management features.
- [Kumaran] You are on LinkedIn Learning right now, or perhaps, Lynda.com. You're watching this course. Hey, thanks for that. But along with you, there are tens of thousands of other users who are logged on to this website. It is in LinkedIn's best interest to monitor this activity in real time, not just from an operations point of view, but also to analyze student behavioral patterns. For this, they need a real time streaming pipeline that allows for fast and scalable analytics. Now, I'm not sure what LinkedIn uses, but I want to introduce you to Apache Flink. Apache Flink is becoming the preferred platform for building real time streaming pipelines today. It's ease of use and extensive streaming functionality, coupled with fault tolerance, have made it the favorite for many data engineers and architects. Apache Flink is an essential skill today for any developer in the big data world. My name Kumaran Ponnambalam. In this course, I will show you how to build a real time streaming pipeline in Apache Flink. I will show you how to use Apache Flink to process and aggregate real time data streams. I will demonstrated windowing, even time processing, and state management features of Flink. I will then help you use these skills in a use case project. We will use Java and IntelliJ IDEA for building the course exercises. Please refer to the other Flink courses for Flink basics, architecture, batch processing, SQL, and machine learning. That being said, let's explore how to build real time processing pipelines with Apache Flink.
- Streaming with Apache Flink
- Using the DataStream API for basic stream processing
- Working with process functions
- Windowing and joins
- Setting up event-time processing
- State management in Flink