Join Doug Rose for an in-depth discussion in this video Set success criteria, part of Learning Data Science: Using Agile Methodology.
- In a 1921 interview, Thomas Edison said that his assistant was discouraged by their failed experiments. The famous inventor was cheery and assured them that they hadn't failed. They were always learning something new. Once they learned something, they were free to try it a different way. Today we know Thomas Edison was right because of his many successes. A few of them are still in use today. There were also a few experiments that were lost to history. Not many people still play on their concrete piano.
We have the benefit of looking at this legacy through a string of successes. We don't see the failed experiments that took up most of his time. If you were in Edison's lab, it might be difficult to imagine his successes. In fact, it might not look like Edison would ever be successful, because there were many more failures than successes. If Edison were using modern project management, he would've run into a couple of challenges. How would he define a success? He couldn't look for the things that work.
If you did, you need a lot of patience. His experiments would last for months, or years, before producing any real deliverable. Instead, you'd want to look at it the same was as Edison. Maybe you just want to ask a simple question: Did we learn anything new? This is the same way you want to approach your data science team. The team will run many experiments on your data. Many of them will be failures, they'll be dead ends, they might not have interesting reports. Not every experiment leads to insights.
It also might be true that most of your discoveries won't have very much value. You might make your own concrete pianos. Maybe you'll find out that most of your customers are pet owners. That's interesting, but doesn't exactly provide great insight. The data science mindset may be challenging in many organizations. In some organizations, it might even be annoying. That's why it's a good idea to create success criteria, and objectives for the team. This is a nice compromise between project management and data science.
The first thing is to make sure that your team is transparent as possible. Fight the urge to stay isolated from the rest of the organization. Often, if people in your organization don't understand what you're doing, then it doesn't take long for them to question your team's value. Second, try to solve large problems. You'll want to make sure that your research lead is ambitious enough to tackle interesting questions. If the questions are too timid, then it might be difficult to show results. It might just look like you're going for low-hanging fruit.
Finally, try to show what the team is learning through regularly scheduled storytelling sessions. These could be questions that the team is working on, or even a few insights. I once worked for a university that hired a group of unstructured data specialists. The team worked in an office near the administrators that hired them. No one else in the university knew what they were doing. Most didn't even realize who they were. The problem was that this data scientist team had trouble asking interesting questions.
No one else in the organization would take their time to meet with the research lead. Things would have gone much smoother for this team if they were asked to sit with the faculty. Then they would have worked with everyone from the beginning to come up with interesting questions. Then they could have had demonstrations that addressed these questions. If you're the research lead for a data science team, work hard to keep the questions closely connected to the rest of the organization. Stay transparent about what you find. Give frequent demonstrations of insights.
Try to draw in the rest of the organization, so they understand the value in data science. If you're the project manager for a data science team, try to work hard to make sure that the team is sitting with everyone else. Some of your best demonstrations might be people dropping in and asking interesting questions. The better connected you are with the rest of the organization, the easier it will be to ask interesting questions.
This course shows how to structure your work within a two-week sprint. See how to work within a data science life cycle (DSLC)—a methodology for cycling through questions, research, and reporting every two weeks. Explore key practices to help your team break down the work so it fits within a two-week sprint. Learn how to use tools like question boards to encourage discussion and find essential questions. And most importantly, learn how to grow your team's shared knowledge and avoid common pitfalls.
- Defining data science success
- Determining project challenges and criteria for success
- Using a DSLC
- Iterating through DSLC sprints
- Creating a question board
- Breaking down your work
- Adding to organizational knowledge
- Avoiding pitfalls