From the course: Apache Flink: Real-Time Data Engineering
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Splitting a stream - Flink Tutorial
From the course: Apache Flink: Real-Time Data Engineering
Splitting a stream
- [Instructor] In this video, I will show you how to split a single DataStream into two streams. The code for this example is in the StreamSplitAndCombine class under the chapter two package. The class has the default streaming environment set up reading a CSV file into a DataStream and executing the pipeline parts as discussed in the previous examples. Let's jump right into stream splitting. We want to split the auditTrail stream into two streams based on the entity type attributes. There are two entity values. Customer and salesRep. We want to obtain them as separate DataStreams. If we use the filter function, we need two separate filter operations. Instead, we will use the process function to do it in one go. First, we need to define a unique output tag for the side output that is unique within the job context. We do so using the OutputTag class and passing a string name to it called sales-rep. Next, we do a…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.