Join Kit Eason for an in-depth discussion in this video Introducing NCrunch: A marvelous test runner, part of F#: Automated Testing for Developers.
- In the last chapter, we saw how to use unit tests to drive simple implementation of a human name parser. You could certainly take this further, using the concept I've already explained, but I wanted to take a few minutes to look at a couple of tools you can use to make the experience even slicker. The first of these is, brace yourself, a commercial tool called NCrunch. NCrunch is a more fluent substitute for Visual Studio's test runner. We're all used to getting much of our software tooling free of charge.
In Visual Studio community edition, which is what I'm using for this course, it's completely free of charge, as are all the other libraries I'll show you in this course. NCrunch is charged for, and here's the pricing. But, in my opinion this is money well spent. I paid for it personally without hesitation. I should note that I don't get anything out of promoting NCrunch here or anywhere else. It is just a tool I love. Well, what do you get for your hard-earned money? Well it only takes a few minutes to install, so let's go ahead and do that.
Then I can show you the magic. So we're going to go to Download. And I got to download the Installer for the version of Visual Studio I'm using. And that downloads an msi, which I can open. That was pretty painless. Let's go back to Visual Studio. Restart it. And now we have an NCrunch menu. So I'm going to reopen the solution I had open before.
And I'm going to enable NCrunch. First menu item. And then we get this wizard. This center section is pre-populated because I have in fact used NCrunch before, but I'm going to pretend that doesn't exist. I'm going to go right the way through the wizard. And, I'm going to let it run my tests alongside each other. If you're writing unit tests, generally they should be completely independent. They should have their own setups, their own freeing of resources, so really they shouldn't depend on each other. So, if you can't run your tests in parallel, it's actually telling you something about your testing philosophy, that it's wrong.
So, run my tests alongside each other. And then you get a bunch of execution modes. Run my tests automatically when changes are detected. Let me run my tests manually, which means you take complete control over when tests are run. Only automatically run impacted tests, which means that NCrunch attempts to workout what each code change you make, which other bits of code those code changes affect, and only runs the tests that those code changes are covered by. Or there is this concept of pinning tests to a test window.
I'm going to go with only automatically run impacted tests. That works well for relatively simple scenarios, but the sort of impact detection can get fooled by rather complex projects. But this is fine for now. And I'm going to let my tests run. I can ignore any, like maybe slow-running tests or whatever as I need to. And now we can NCrunch tests window, which I'm going to maximize in the environment. And there's a number of things to observe going on now.
Down here we've kind of got a progress indicator, which is green. That's telling me all my tests are passing. When it's blue, that means a build is occurring. When it's orange, it means the build failed. Usually because your code doesn't compile. And when it's red, that means you've got some failing tests. Then up in the test window, by default, passing tests are concealed, but you can change the filter by clicking on this green tick. And you can see your passing tests. And then you can expand that to see the individual tests.
And then we can actually double click to navigate to the actual test code. We're getting some red squigglies there because I haven't explicitly built this. There we go, the red squigglies have now gone away, as Visual Studio catches up. So what happens if I break some test code to actually cause a failure? Let's go to my implementation. And let's go back to that bit, where we checked whether p1, part one of the incoming string was a title. And let's just reverse the logic by piping that into not.
Now can you see down at the bottom right hand side of the screen? Down here, we've got an orange circle. That's because the code can't be built at the moment. Watch very carefully. I'm going to type n o, watch the bottom right, t, so that's now compiling code. And you saw that the tests, well at least one of the tests failed. In fact seven tests failed as a result of that change. So let me just go through that again. I'm just going to make it so it doesn't build orange. N o t space, red.
Now I didn't have to explicitly run tests. I didn't have to explicitly compile. That was completely live as I typed. I simply typed the code changes I want. So for instance, if I now get rid of that, my tests pass again. And what's going on is basically that NCrunch starts a secret build, every time you change any of your code. And if that build completes it runs tests against the build. And obviously as you're typing all the time, it's canceling builds an restarting builds, until there's enough of a pause for it to catch up and actually run tests.
If we look at our implementation, you can see that we've now got these colored blobs, beside every line that's covered by a test. So if I again change this to reverse the logic, we can see all the lines that are covered by a test that is failing, including the actual line that I changed there. I can click on the blob, and that will give me a list of tests that cover that line. One of them passes, because there must be some path through this code where that's correct.
And one of them fails. And I can double click on this and actually go to the test code. The other place I can look is by clicking here, and that will go back to the NCrunch Tests window. I can navigate to the tests. And I can also see details of how the tests failed at the bottom here. Let's unbreak that code again. Let's add another kind of bug. One that raises an exception.
Obviously this is fairly silly implementation code but it's just to illustrate a principal. So if the incoming rawName happens to start with the word Prof, and I missed out the then, then we raise an exception. And there, the reason I did that is there you can see you get a red X at any line of your code that raises an exception. Click on that, and see the test that is failing. And we can even see the message that occurs.
Exception of type System.Exception was thrown. Now if you think back to the last chapter, where we were continually pressing Control R, Control A, and then navigating to the Tests window, this gives a much more fluent experience to the process of true test-driven development, and I really love it.
Kit Eason explains how to use xUnit—a .NET unit-testing package—to do some test-driven development, and demonstrates how to improve your test run experience using NCrunch, a test runner. He shows how to use FsCheck to generate test cases, and how to use Expecto to move into the world of tests as first-class values. He also covers the use of Canopy to automate the testing of web user interfaces, and of mocking to tame dependencies.
- Solving issues with early versions of Visual Studio 2017
- Classical unit testing with F#
- Creating a testable project and installing Paket and xUnit
- Test driving an implementation
- Improving your test run experience with NCrunch
- Going beyond test cases with FsCheck and Unquote
- Making tests first class using Expecto
- Using mocking to tame dependencies
- Web UI testing with Canopy