How automation can go wrong

- In this lecture, Tom Davenport and I are going to be talking about automation. Keeping the pros and cons of automation in mind is going to be useful if you're looking at bringing cognitive technologies into your organization. Or if you're designing systems or work processes that might involve cognitive technologies. The history of automation goes back hundreds of years, from manufacturing to aviation to clerical work. Cognitive technologies extend the reach of automation. As we've seen, it's possible to automate tasks that traditionally required human perceptual and cognitive abilities. Automation can be great. It can make it possible to produce goods faster, cheaper, and better. And it can relieve us of tasks we don't want to do. But it can also come with some surprising drawbacks. Decades of research show it doesn't always deliver the intended benefits, and it can have unintended consequences too. Here are some things that research has taught us about automation. First, automated systems don't always work. Using automation to eliminate flawed human performance may seem like a compelling idea. But automated systems can have flaws too. Tom, you were talking about an example of this recently. - We can talk about flaws and automation on a small scale and a large scale. We're all familiar with the frustrations of speech recognition on our smartphones, or an automated call center where we're desperately trying to press zero and get to a human so we don't have to deal with it anymore. But on a larger and much more important scale, we have these highly interconnected, large automated systems like financial trading and energy management, and we just don't understand the dynamics of them. Sometimes as in the flash crash about five years ago, they go awry and we don't understand why. We still don't understand why the flash crash happened and stock prices dropped precipitously. So I think in order to really adopt these systems on a large scale, we'll have to develop a much greater understanding of their dynamics, both social and technical. - It's a great point. A second drawback that automation has shown is that humans are bad at monitoring automated processes. Studies show that it's effectively impossible for even a highly-motivated worker to pay attention to information that hardly changes for more than a half hour or so. So tasking people with monitoring an automated process can cause errors and anomalies to go undetected. Third, people tend to lose skills if they don't practice them regularly. Tom, you were talking about this one as well. - Well I think that issue could become quite germane to all of us if we get these autonomous vehicles that we've been hearing so much about. Will we no longer be able to drive? And what happens if the autonomous functions of the car don't work and we have to steer it? Or what if we don't even have a steering wheel? I think it raises all sorts of concerns for social policy and regulation and so on. - Fair enough. There's a fourth issue with automation that's a little bit more nuance but it's worth bringing up. Studies have shown that poorly-automated systems can undermine worker motivation. It can cause alienation, can reduce satisfaction, even productivity and innovation. And it's well-known that businesses whose workers are engaged, tend to perform better than those whose workers aren't. So the drawbacks of automated systems can include bugs, deskilling, ineffective process monitoring, and a loss of motivation and productivity. Automation is undeniably valuable. But we need to know how to do it right. This is more important than ever with the growth of cognitive technologies. How to do automation right is the focus of our next lecture.