Transformations and aggregations are required for processing incoming streams and to generate analytics. Use the DataStream API for basic stream processing.
- [Instructor] In this video, we will perform … some computations on the audit trail object … populated in the previous video. … We have a printCount function, … which is a convenience function … to print the count of records … received by a data stream every 5 seconds. … This function shows the basic use of some … of the data stream operators. … In this function, we first use a map … to get the message fast to the function … and a count of 1. … We use a timeWindow of 5 seconds … to aggregate the data at this interval. … We will review timeWindows in detail … in the windowing chapter later in the course. … Then, we use a reduce to add up the counts. … Given that we have a window, … the reduce will only receive data … and emit output every 5 seconds. … Finally, we have a map function … to do a pretty print of the output … of reduce every 5 seconds. … We will use this function in many … instances in this course. … Back to the Mainstreaming Operations Class. … In the next operation, … we do a similar kind of summary. …
- Streaming with Apache Flink
- Using the DataStream API for basic stream processing
- Working with process functions
- Windowing and joins
- Setting up event-time processing
- State management in Flink
Skill Level Advanced
1. Apache Flink
2. DataStream API
4. Event Time Processing
5. State Management
6. Use Case Project
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.