In our previous blog post, we talked about a few of the things you should always strive to do when designing a quantitative questionnaire for your market research. We thought we’d follow that post up with some guidelines to help you recognize some of the things that you should virtually NEVER do when designing a quantitative questionnaire.
1. Never begin to design a questionnaire until you have a solid understanding of the client’s business and research objectives.
Before you jump to the survey design, re-read the client’s stated study and business objectives in the RFP or research proposal. What end results does this study need to achieve? How can your questionnaire be a means to that end? Ideally, any vague or confusing areas in the RFP should have been clarified before the proposal was sent out … but if there is still something that is unclear, now – BEFORE you begin to write your first draft – is the time to have the account manager or the client explain it in full. Skim the objectives again periodically as you work, to keep yourself on-track. Neglecting to adhere to the client’s list of objectives is an excellent way to guarantee that there will be major re-working of the questionnaire required; it will also likely guarantee that the client will question your attention to detail and communications skills.
2. Don’t underestimate the importance of the screening questionnaire.
Research groups often assign their most junior team members to draft screening questionnaires, as a stepping stone toward eventually being able to design the main instrument. But what could be more critical than ensuring that your screening efforts are getting the right respondents to participate? It’s important to remember that your data will be virtually worthless if you don’t make sure that you are including only respondents for whom the survey is relevant, and whom the client considers to be of interest. So, sure – let the less-experienced researchers dip their toes into the waters of questionnaire design by asking them to draft your quantitative screening questionnaires. But please make sure that those screening questionnaires are carefully vetted and proofed by more experienced members of your team!
3. Don’t forget to allow for every response that a participant might give to a question – even if you do so by adding “don’t know”, “none”, and “other – specify” options.
One of the quickest ways to frustrate a respondent and hasten their departure from a survey is to make them search for a way to include a response that just isn’t among those listed. It’s exasperating for the participant, adds to the time required to complete the survey, and sends the message that their responses are not very valuable.
4. Never use only one question technique throughout the questionnaire for capturing data; approach the topics you want to cover in various ways to keep the respondent engaged and on-task.
There’s nothing more mind-numbing than plodding through an endless ocean of questions asking “Which rating on the ten-point scale displayed best describes how compelling you consider each of the following statements?” … for 50 or even 100 different message statements. To break up a “wall” of similar questions, try inserting an open-ended question or two to clarify why respondents rated a particular item or statement as they did. Include a couple of quick, easy “yes/no” questions if possible. If you can use an approach that respondents may not often see – such as a Max/Diff methodology, or using “sliders” for rating scales rather than just requiring the respondent to type in an integer response – that’s even better! In addition to yielding additional interesting, valuable data, the objective here is to “reset” the respondent’s brain so that they don’t begin to click on the same rating at every prompt, or answer the questions by blindly clicking on random responses, just to get through the task.
5. Never forget to run Spell Check and proofread the each iteration of the questionnaire document before sending it to the client.
Before your questionnaire is programmed using survey data collection software, it will be reviewed in text form by the client. In addition to checking for skip logic and adherence to study objectives, basic proofreading for spelling, punctuation, and syntax is a must before you send over that file. Verify that all company and brand names – particularly those that are associated with your client – are spelled correctly. Double-check to make sure that you didn’t forget to delete companies and brands that might have been included in any older documents that you used as style or content templates. This probably sounds like a very basic, “Client Service 101”-type rule, but you’d be surprised how often clients receive deliverables from research suppliers that include typos, or mentions of the wrong companies or brand names (including the names of the client’s direct competitors!)
After reading our two-part series on Designing Quantitative Questionnaires, you’ll probably understand why designing an effective questionnaire for a quantitative study often compared with completing a jigsaw puzzle: If even one piece of the puzzle is missing, the whole project is incomplete!
Now that we've talked about questionnaire design, we should talk about what to consider when setting up an email campaign to reach the most respondent. Download our white paper now!