In this video, Meaghan Lewis explains the test pyramid and why it is an ideal way to structure tests. It is a guide intended to visualize how tests should exist at various levels within the application. Use the pyramid to help guide and evaluate whether your automated test suite has the right level of testing.
- [Instructor] Another model I always refer back to when planning test automation is the test pyramid. The test pyramid explains an ideal way to structure tests. It is a visual representation of the recommended amount of test coverage that should exist across each type of test. This concept was introduced by Mike Cohn in 2009 in the book Succeeding with Agile. The original test pyramid consists of three levels.
There are unit test at the base of the pyramid, integration test in the middle, and UI test at the top. These are the test types that are the usual suspects on software projects. At a minimum, I recommend that projects have at least these three types of automated test, but can have additional types of test as well. The test at the base of the pyramid will be the fastest running test. As you move up the pyramid, the test becomes slower.
Similarly, the tests at the base are the most isolated in what they test, and moving up the pyramid, the tests become more involved and more integrated with different services. Unit tests are always at the base of the pyramid. They test a single function by calling that function and passing in various values. Unit tests confirm that the right results are returned for each function. Unit tests are the fastest test and run in a matter of milliseconds.
There should be the greatest number of these tests. And data for a unit test is typically mocked or stubbed, which are ways to create objects with certain values. Jumping up to the middle of the pyramid are the integration or service-level test. They test multiple services in conjunction to ensure that all parts work seamlessly together. They typically involve testing integrations between the database, file systems, and other applications.
APA testing is also very popular here. Integration tests take a bit longer than unit tests, typically taking anywhere between 10 or 100 milliseconds to run and they create their own data. There should be a good amount of these tests to cover each service. User interface, or UI, tests are at the top of the pyramid. They are extremely valuable because they test end-to-end workflows and simulate user actions, like clicking and typing.
These tests run through the browser and, therefore, can take many seconds and sometimes up to a minute to run, depending on the workflow. There should be just a handful of these tests that cover each primary user workflow. The pyramid is intended to be a model of what a healthy, fast, and maintainable test suite looks like. It encourages a team to be wise about a strategy of what to automate at each level.
Be cognizant of the amount of tests that are written or else there can be negative effects. For example, sometimes teams want all unit test on a project and don't care about any other type of test. The shape of testing here ends up looking like a big square. While the tests execute quickly, there are likely going to be some testing holes where bugs can slip through. Or maybe a project has a large number of UI tests, testing every single part of the user interface.
There are a minimal number of integration and unit tests. Perhaps everything looks okay in an application, but at a lower level it's not functioning as it should. The shape here can end up being an inverse pyramid. In this case, the tests likely have a slow execution time and are a pain to maintain. There are many more shapes that can manifest based on the amount and level of test implemented, and sometimes that's okay. Just be sure to consider the pros and cons of the shape and what it implicates.
Besides different amounts of tests, the test pyramid can also include many other types of tests as well. I encourage you and your development team to think about what the test pyramid for your project will look like. Remember that the shape of your test might not resemble the pyramid exactly and might have more than three levels.
- The test pyramid
- Unit, integration, and UI tests
- Creating an automation strategy
- Choosing test tools
- Deciding what to automation
- Identifying the risks and cost of automation
- Implementing test automation
- Using continuous integration
- Measuring code coverage