Learn how to identify and handle inferred members when loading a star schema.
- [Instructor] Sometimes, when we have data coming in, there aren't dimension records for the associated event record, so we call these inferred members, and I like to think of it as the situation where when we have new dimensions, so let's say a new client appears, prior to that client record existing in our lookup table. So just like we looked at the client table previously or the zip code table, if we have a new record in there, we need to handle that new record somehow, and it looks like this. We have our client and our sales table, and we're loading these with new sales data coming in.
What we have to do first is check the client table, so, if there are new clients that weren't already loaded in our new sales feed, we need to add those to the client table and then get back that business key, that client ID. Then we can finally load that new sales data into the sales table. This is what's known as inferred member support. It's sort of a traditional data warehousing term, but when you're loading data here, you're going to encounter this, where events show up with corresponding data that's related, such as a zip code or a client, that doesn't already exist in those other tables, so how do you handle that? You just need to load that data first, so there's sort of an order of operations here when you process new event data.
- Working with systems and schemas
- Managing of a good data pipeline
- Setting up an environment
- Loading and profiling data
- Testing quality
- Adding data types
- Handling missing values and inferred members
- Performing master data lookups
- Loading schemas and tables
- Creating views