Walk though the organization and packaging of the exercise files. This is an important prerequisite for the rest of the course.
- [Instructor] The exercise files for this project are available in the real time data engineering.zip file. I have extracted the contents of this zip file into the Exercise Files folders on my desktop. In this video, I will walk you through the steps to import this project into your setup. First, make sure that your IntelliJ Idea environment and Maven are set up properly. From IntelliJ Idea, click on the Import Project option, and then choose the extracted file folder. Click OK. Select Import project from external model, and choose Maven. Click Finish. Import will take some time as Maven resolves the required dependencies. Import Changes as required. You should be able to get a clean build at the end of the process. Please use Java 8 as the SDK for this version of Flink. You can set it up in the Project Structure, Project SDK. One key thing to note is about Maven dependencies for this project. The scope of most of the dependencies is marked provided by default. You can look at them by going to Modules. This is because when you deploy the project on a Flink cluster, Flink installation provides these dependencies, and you should not be packaging them in your build, but that may cause a problem with executing programs inside the IDE. If you get a class not found error when running any of these examples, please check the following. Make sure that all these Scope options are set to Compile. Once this is setup, we are good to go. We are all set to do some serious Flink streaming coding.
- Streaming with Apache Flink
- Using the DataStream API for basic stream processing
- Working with process functions
- Windowing and joins
- Setting up event-time processing
- State management in Flink