Join Alan Simon for an in-depth discussion in this video Addressing the analytics gap, part of Transitioning from Data Warehousing to Big Data.
- The gap between what today's organizations need for data-driven insights and what traditional BI and data warehousing provide us, continues to widen and that means we need to do something to address that disparity. Here we see five must-have capabilities for today's analytics and we see that for each one of them traditional data warehousing and BI are rather challenged when it comes to delivering those capabilities to us. Even with today's relational databases, our data warehousing methodologies usually call for us to make hard choices about what source data we do and don't need and what we actually bring into the data warehouse.
But inevitably we come across critical needs for data that we've left out of our warehouses and we wind up having to scramble to add that content into the data warehouses without disrupting the structural integrity of the environments or the usefulness. We've also seen time and again that no matter how thorough of a job we do with our upfront requirements analysis process in identifying our data needs, we will find new data sources that we decide can provide significant value to us if we could only access that data.
With data warehousing the way we've practiced it, that's a challenge. We've already taken a brief look at the challenges of Operational Business Intelligence that often need real-time access to data to answer questions of the type tell me what is happening right now. We've also seen how data warehouses have been built since the very beginning primarily to contain structured data, and any of our analytical needs for semi-structured or unstructured data have usually required us to build alternative solutions that are very difficult to integrate with our core data warehouses and data marts.
And finally our data mining needs for predictive and discovery analytics usually require us to build totally separate specialized stores of data. And more often than not, we wind up with significant architectural issues with separate data feeds directly into those analytical environments that start showing up. Every day people all across your organization are coming up with new Ideas for how to drive decisions and actions from your data. But as we've seen with data warehousing, the time between when those ideas are first thought of and when they could be put into action, tend to be on the longer side due to the technologies and methodologies we've had to use.
What we need to do then is significantly decrease the time between our ideas for analytics and when we actually have those insights at our fingertips. Our data warehousing and BI systems have served us well for many years, but the ways in which we've built those systems and the technologies we've used are just about at their limits when it comes to giving us the analytics we need so desperately. The good news though is that Big Data technology and architecture, specifically Hadoop and its entire ecosystem, is coming to the rescue to help us close that gap and significantly reduce that idea-to-insight cycle.
- Exploring big data, Hadoop, and analytics
- Examining the shortcomings of traditional data warehousing
- Comparing big data architectures for next-generation data warehousing
- Understanding alternatives
- Building a roadmap
- Managing big data-driven projects
- Monitoring and measuring success