Join Ernest Mueller for an in-depth discussion in this video Operate for design: Logging, part of DevOps Foundations.
- Logging is hard to get right, but the rewards are huge. Logs are a great DevOps monitoring tool because while they can be consumed by an operations team, they can also be fed back to developers to provide meaningful feedback. Before getting into it, remember the five Ws of logging: what happened, when it happened, where did it happen, who was involved, and where the entity came from. Logging can be used for lots of purposes, ranging from audit to forensics, to troubleshooting, resource management, to intrusion detection, to user experience. When logs are happening on a single machine, it's relatively easy to understand how to use it. However, in modern infrastructures with microservices, containers or serverless, there's not a single machine anymore. And in the case of serverless, there's no machine at all in a classical sense. One of the first goals in any environment is centralized logging. Sending all the logs via syslog or store and forward agents to a centralized location, that's key, but how you do it is important. With this in mind, there are five principles I'd like to cover. First, do not collect log data that you're never planning to use. Now, if you can say, I might need this to investigate an outage or audit something in the future, then, okay, it's definitely in scope. But if you can't think of a feasible use for it, don't collect it. Number two, retain log data for as long as it's conceivable that it could be used or longer if it's prescribed by regulations. This is pretty self-explanatory. The impact of keeping too much overloads cost for resources, for maintenance, and it inhibits overall growth. Three, log all you can, but alert only on what you must respond to. At Signal Sciences, where I work, we log a lot. However, we only alert our development and operations folks based on customer experience problems or security problems. Our goal is to have customers use our application and never get an application error. So, naturally, anytime we send a customer an HTTP 500 error code, we also alert our staff. You also want to make sure that you clearly define your log levels: error, warn, info, et cetera. You know, one tip I'll give you here is that there should never be such a thing as a standard error. Errors should be something that requires action. If it doesn't, make it a warn. All right, fourth, don't try to make your logging more available or more secure than your production stack. This is a very lean approach. And if you overbuild capacity, or in this case defense and availability, you have misallocated resources in your system. Logging should meet business needs, not exceed them. Fifth, logs change, as in their format or their messages. New versions of software bring changes. Creating a feedback loop that encourages everyone to take ownership of their logs in the centralized logging system, that's important. So this wraps up our section on logging. Logging creates a fast feedback loop from operations to design. There's a lot of great books out there on the market. They'll help you to this end. We mentioned Logging and Log Management, but also Web Operations and the Practice of Cloud System Administration are good places to find out more. In closing, monitoring, metrics and logging are three feedback loops that bring operations back into design. Remember, the other processes that can create feedback loops such as incident command system, blameless postmortems, and transparent uptimes, which we've discussed elsewhere in the course.
In this course, well-known DevOps practitioners Ernest Mueller and James Wickett provide an overview of the DevOps movement, focusing on the core value of CAMS (culture, automation, measurement, and sharing). They cover the various methodologies and tools an organization can adopt to transition into DevOps, looking at both agile and lean project management principles and how old-school principles like ITIL, ITSM, and SDLC fit within DevOps.
The course concludes with a discussion of the three main tenants of DevOps—infrastructure automation, continuous delivery, and reliability engineering—as well as some additional resources and a brief look into what the future holds as organizations transition from the cloud to serverless architectures.
- What is DevOps?
- Understanding DevOps core values and principles
- Choosing DevOps tools
- Creating a positive DevOps culture
- Understanding agile and lean
- Building a continuous delivery pipeline
- Building reliable systems
- Looking into the future of DevOps