Data in a stream needs to be aggregated based on keys. Use keyed streams for partitioning and aggregating data in Flink.
- [Instructor] In this video I will discuss … partitioning of streams by keys … and show an example for the same. … In partitioning, we apply the keyBy operator … to partition data by one or more attributes … in the data stream. … Flink distributes the events in a data stream … to different task slots based on the key. … Flink users are hashing algorithms … to divide the stream by partitions … based on the number of slots allocated to the job. … It then distributes the same keys to the same slots. … Partitioning by key is ideal for aggregation operations … that aggregate on a specific key. … Each key is then aggregated locally … in the task slot without the need to shuffle data … between the slots. … This helps optimize the performance of this job. … The code example for using keys … is in the keyed stream operations class … under the chapter two to package. … The setting of the flick environment … and the reading CSV into the data stream, … and setting up the pipeline are all the same …
- Streaming with Apache Flink
- Using the DataStream API for basic stream processing
- Working with process functions
- Windowing and joins
- Setting up event-time processing
- State management in Flink
Skill Level Advanced
1. Apache Flink
2. DataStream API
4. Event Time Processing
5. State Management
6. Use Case Project
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.