Know what you want to learn

- It's very unlikely you're running a survey just for the fun of it. Normally, there's a particular problem you need to resolve and questions you need answers to. The best way to make sure your questionnaire will provide the type of data you need is to write a draft of your report before you run the survey. For each statement you want to make about the data, write or sketch out what the answer would look like. I call these desired outcome statements. Now you can create the questions for your questionnaire in a way that will give you the input you need to make these statements, and those statements will tell a story about your survey respondents that gives you the information you need to make decisions. To be clear, I'm not saying you should word the questions to make sure the answer comes out the way you want, but I am saying you need to give yourself the opportunity to find answers to the questions you care about. In other words, your desired outcome statements are like a scientific hypothesis. A statement of what you believe to be true, then your questionnaire provides a test for whether they're true or not. A good test is unbiased, but it will show the answer, if it exists. In the same way, your questions should be unbiased, but they should let you see the answer, whatever it turns out to be. It's easier to make statements about numbers and proportions than it is about people's written responses. For instance, a simple desired outcome statement might be, 80% of our users are either mostly or very satisfied with our application. I put the percentage figure in brackets because although that's what we want the number to be, we won't know until after we run the survey. So we'd like to be able to write that in our report, and we need to make sure that we ask a question that will give us an answer we can use in that desired outcome statement. In this case, a simple satisfaction rating scale will do the trick. After we run the survey, we can count the number of responses that were mostly satisfied and the number that were very satisfied and calculate that as a percentage of all the responses. Hopefully, it's in the range we were expecting, but even if it wasn't, we've still learned something important that we need to explain in our report. If you ask qualitative questions that have written responses, work out how you're going to analyze and report the results before you field the survey. It can be disheartening to work through a whole mess of written responses, so make your life easier from the beginning. Make it clear what type of answer you want by asking a focused question. For instance, if you're asking for feedback, state exactly what part of the product you want that feedback on. Qualify the text answers by collating them in conjunction with other fields in your survey. For instance, comparing the written responses between satisfied and dissatisfied users might help you spot a pattern that can increase satisfaction rates. The desired outcome statement you create might look like this. Respondents listed the following reasons for being dissatisfied with the support process, and then the reasons for being dissatisfied. Those who were satisfied said it was because and then the reasons for being satisfied. That suggests we should focus on and then the things that can reduce dissatisfaction reasons and increase satisfaction reasons. Because we're working with qualitative data in this desired outcome statement, we can't create goals to fill in the blanks inside those brackets yet like we could with the 80% figure earlier. But still, writing out the statement now lets us design our questionnaire with the appropriate questions. In this instance, in order to be able to make that statement, you'll need a question that asks for satisfaction with the support process and a text field that asks people to give their reasons for the satisfaction rating for the support process. So you can see how important it is to think through what you want to be able to say as a result of running your survey before you even start. Desired outcome statements let you create a more clearly-focused questionnaire that will make more sense to respondents as they fill it out, on which you can be confident will provided data that you can use to tell a good story.