Developing a questionnaire
In this section:
Basic rules for questionnaire construction
- Use questions that are interpreted in the same way by members of different stakeholder groups.
- Use questions that allow for an entire range of options to be commented on.
- Don't force participants to choose between options where they may not agree with any of them.
- Have an ‘open’ answer category after any list of options.
- Use positive statements and avoid negatives or double negatives.
- Do not make assumptions about the participants.
- Use clear and comprehensible wording, easily understandable for all educational levels.
- Use correct spelling, grammar and punctuation.
- Avoid questions that contain more than one question (e.g. Do you think one-level and two-level buildings are suitable for this site?).
- Questions should not lead the participant towards any particular response.
In general, questions should flow logically from one to the next. So the questionnaire should be developed to lead the participant through the topic in a logical manner. Wherever possible, a questionnaire should follow a natural flow, reflecting a train of thought, a logical conversation or a chain of events.
Also having relevant information immediately before or beside each question means that the participant has an opportunity to read factual items before expressing their point of view. Online presentation of information also offers the opportunity to provide layered information so the participant can continue to explore the information appropriate to their level of interest.
People tend to look at the first few questions before deciding whether to respond to the questionnaire. By putting the questions of high interest first along with the most important issues, even partially completed questionnaires will contain useful responses.
A typical questionnaire flow might be:
- information about this questionnaire and how the responses will be used
- warm-up questions
- transition information
- skip instructions or routing
- complex or contentious areas.
Warm-up questions are usually simple to answer, help capture interest in the survey, and may help develop trust and confidence that the participant's views will be taken seriously. Good examples are questions which start with a good précis of the proposals and then ask: "Do you generally agree or disagree with these proposals. Please give us your main reasons for your choice above". This should be followed by an open text box so they can state their main reasons.
Transition information is used to separate the topic if there are several topics covered and to explain how they are related (or not).
Skip instructions direct the participant to the next relevant question – for example "If yes, then answer question 3. If no, then continue to question 4."
Most online questionnaires will use the response to a question to automatically route the participant to the next question to be answered. If authoring a questionnaire in an online tool, make sure you use this feature.
Complex and contentious topics are usually towards the end because the respondent is in ‘response mode’ and may have picked up more information and be willing to expand on their views.
Do use a progress bar in authoring the questionnaire. The progress bars lets the respondent know where they are in the questionnaire, so they are often more willing to give thoughtful and comprehensive responses.
Demographic questions should be at the end if used at all and should not be compulsory. It's useful to explain why we are collecting that kind of information. (That is normally to make sure we are reaching a wide range of members of the community and getting the views of a wide range of people). These kind of questions can give the impression that we are collecting very personal information and can make participants uncomfortable and concerned about how their views are going to be taken into consideration.
There are two main types of questions:
- open – where space is given for the respondents to answer in their own way
- closed – where questions are followed by structured responses (for example, simple YES or NO tick boxes or multiple choice answers).
Open questions reveal more information than closed, but can take more time and be more difficult to interpret. Both open and closed questions can be used in conjunction with each other to provide additional information and help the interpretation of responses.
The Market Research Society in the UK has identified four major issues that are known to have a negative impact on the quality of responses and participants' attitudes
- very long questions
- questions that ask the same thing but in a different way
- insufficient opportunity for respondents to comment
- excessive personal information requests.
The order of questions will have an effect on the answers that are given.Space should be provided to allow respondents to enter their comments on any other issues not covered by the questionnaire. This encourages respondents to feel that their views are valued and it provides a useful guide to aspects of the topic that may not have been adequately covered in the questionnaire.
How are you going to attract participants to complete your online questionnaire? Pulling participants through to your questionnaire will be covered in your communications plan. At this stage you might want to consider if you need printed paper versions for some people.
Participants will need complete and clear information to be able to comment effectively. The better informed people are, the more useful their comment will be.
Testing and piloting
Test and test again to make sure the questions make sense. It's worth testing the questionnaire with a few stakeholders before making it public. Stakeholders are useful because they will give you real feedback about whether your questionnaire will get the result you need.
Send us your feedback
This guidance is a work in progress, so please email us your feedback on how useful you found it, what was missing, how it could be improved.
- Last modified: