A data stream may need splitting to cater to multiple processing use cases. Learn how to split data streams vertically and horizontally in Flink.
- [Instructor] In this video, … I will show you how to split a single DataStream … into two streams. … The code for this example … is in the StreamSplitAndCombine class … under the chapter two package. … The class has the default streaming environment set up … reading a CSV file into a DataStream … and executing the pipeline parts … as discussed in the previous examples. … Let's jump right into stream splitting. … We want to split the auditTrail stream into two streams … based on the entity type attributes. … There are two entity values. … Customer and salesRep. … We want to obtain them as separate DataStreams. … If we use the filter function, … we need two separate filter operations. … Instead, we will use the process function … to do it in one go. … First, we need to define a unique output tag … for the side output … that is unique within the job context. … We do so using the OutputTag class … and passing a string name to it called sales-rep. … Next, we do a process function … on the auditTrailStr DataStream. …
- Streaming with Apache Flink
- Using the DataStream API for basic stream processing
- Working with process functions
- Windowing and joins
- Setting up event-time processing
- State management in Flink
Skill Level Advanced
1. Apache Flink
2. DataStream API
4. Event Time Processing
5. State Management
6. Use Case Project
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.