Join Alan Simon for an in-depth discussion in this video Tools and technologies, part of Big Data Foundations: Building Architecture and Teams.
- Your organization uses tools and technology throughout your enterprise for many different purposes. When it comes to enterprise data, tools and technology are equally as important, and the very nature of enterprise data management invariably means that you will be using many different ones for many different purposes. You'll use tools for accessing data, storing data, interchanging data, and maintaining data. Let's look at each of these in more detail.
Your business users may have the most personal familiarity with the tools used to access data, and, in fact, to them, those tools and the data delivered through those tools may actually represent the entire enterprise data environment in their eyes. The reports and the analytics delivered and used for both operational and strategic decision making are critically important, and almost every organization today uses at least one commercially available product to deliver these analytics rather than having reports and dashboards built and delivered by custom developed programming.
But even within this category of reports and analytics, you'll have different subcategories that need to be focused on as you match up your business capabilities with the tools that will be part of your future state. Make sure that you account for operational reports, including those that might actually be produced by your transactional applications themselves. Also, pay attention to regulatory reporting as governed by your industry or governmental reporting based on where your business might be located. Beyond operational and regulatory reporting, make sure you also pay attention to the tools that are used to produce dashboards and visualizations as these will very much be the face of enterprise data to many of your users.
For all of these categories, make sure you pay attention to what your scorecards told you about the tools that you're using today. Some might be working very well while others are widely known throughout your enterprise as being particularly problematic. Beyond the actual scores that identify certain hot spots, make sure that you dig into the story behind those numbers. What did the comments on the surveys and scorecards tell you? What did follow-up meetings and interviews tell you? Look for signs that certain tools might be difficult to use or maybe even unreliable or that response time is too long and unacceptable or perhaps even that too many tools are in place for too many purposes, resulting in confusion and frustration.
You will also have tools that you use to store and manage your data. Throughout your enterprise, you'll have dozens or hundreds or maybe even thousands of relational databases used for operational purposes such as reporting and analytics. You may also have big data environments, Hadoop or MongoDB or some other engine that uses non-relational technology. Make sure you also pay attention to datastores for unstructured and semi-structured content such as document repositories and multimedia storage engines.
All of these do need to be considered as you look at your current environment and plan what will come next. As with your tools, you're looking for feedback that any given environment is difficult to use or maintain or might be unreliable. Performance issues may be highlighted along with the possibility that far too many data storage environments exist and the portfolio needs to be scaled back and rationalized. Another set of tools will be those responsible for data interchange across your components.
If you have traditional data warehouses and data marts, you likely will have tools for data extraction, transformation, and loading, or ETL. Big data environments usually have similar tools, though they perform ELT, or extraction, loading, and then transformation. Even non-tool ETL and ELT that has been accomplished and built by custom coding needs to be looked at along with special purpose one-to-one interfaces that might exist. As you cycle through all of these different tools, keep referring back to your scorecards to guide you on where you need to pay particular attention.
A final set of tools will be those which help you maintain the data. These are usually used by IT personnel rather than business users and help with critical functions such as profiling your data, backing up databases and then recovering them if necessary, planned archiving, as well as auditing data access. The more you continue to examine your current state and build a list of preliminary ideas for your future state, the more your overall enterprise data picture begins to take shape.
Join Alan Simon as he shows how to build a top-notch enterprise data team, evaluate your data architecture, solicit feedback from business users and IT staff, and create a plan to manage your data going forward. He helps you build a portfolio of projects that align with business needs, and migrate your current data into a more modern big data environment.
- Compare and contrast big data technology and data warehousing.
- Explain the pros and cons of implementing enterprise big data technology.
- Understand the benefits of using a data-architecture team.
- Differentiate between master facts and master data.
- Identify the strategies used to sell a portfolio of data management initiatives.
- Recognize the roles and skills of each member of a data team.
- Recall the information required in a maturity model.