From the course: Data Science Foundations: Data Engineering
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Data science system overview
From the course: Data Science Foundations: Data Engineering
Data science system overview
- [Instructor] All right let's get started here by doing an overview of a data science system. Here's an ideal world view of how your big data system really should look. We have on the left data providers, and we can think of these as different types of events coming in. So first we have real-time events, and these stream in data. Then you have data at rest, so this is data that's sitting there, and this gets batched in, in usually a nightly or a daily process. All of this data flows into the Data Hub, and I'm using that term and it's something that you'll hear a lot of other people talk about. Think of this as your enterprise data repository, where all of the data from your company ends up. It may not start there but it definitely lands there at some point. Now from here, the data from the real-time stuff streams in, meaning as it comes in, it's being processed, and the data at rest is batched in, so it's just kind of copied over in some sort of interval. Then you have your analysts,…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.