Survey Questionnaire Design

What is the first step in designing a survey questionnaire?

Not writing questions. List the objectives, and next to each one write the decision it informs and what would be done differently under each plausible answer. Objectives that survive this become measurement targets. Objectives that do not are topics of interest, and they are where questionnaires go to die.

What is an objective-to-question mapping grid?

A table with one row per question, showing the objective it serves, the decision that objective informs, the construct being measured, the question, the response frame, the base, and the specific analysis output it feeds. Two rules govern it: no question without a parent objective, and no objective without a question. It is built before wording is drafted, because designers who draft first tend to rationalise the questions they have already written.

How long should a survey be?

Short enough that the answers at the end are as good as the answers at the start. Quality decay is not uniform: break-off, straightlining and speeding concentrate in the back half of the instrument and among the least engaged respondents, so the loss is not random. Set a length budget in advance, estimate against it from the question inventory, and if the estimate is over, cut whole objectives rather than compressing questions, because compression buys a minute and costs data quality invisibly.

What makes a survey question biased?

Most commonly one of: it suggests its own answer (leading), it carries an emotive or assumed frame (loaded), it asks two things at once (double-barrelled), it uses an undefined quantifier such as "regularly", or it uses the client's vocabulary rather than the respondent's. Bias also lives in the answer options, which get a fraction of the scrutiny that question stems get: overlapping ranges, unbalanced scales and missing options change the distribution before anyone answers.

Should a survey question include a "don't know" option?

Only where not knowing is a real and meaningful state, such as a factual question or a question about a brand the respondent may never have encountered. On an attitude question where everyone holds some view, it becomes an escape hatch from effortful thinking and inflates non-response. Where it is included, it is offered as a separate option rather than a scale point, and it is excluded from the base at analysis.

Why must unaided questions come before aided ones?

Because the contamination is irreversible. Once a respondent has seen a list of brands, features or attributes, their spontaneous awareness of those items cannot be measured again in that interview. This is the one sequencing rule with no exceptions anywhere in an instrument.

Can AI design a research questionnaire?

It can draft one quickly and fluently, which is exactly the risk: a generated instrument tends to be well ordered, longer than it needs to be, and to contain nothing that was removed. Used properly, AI should be required to produce the mapping grid and the removed-questions log alongside the questionnaire, and should never invent the substantive content of an option list (brands, features, competitors, price points) that was not supplied. A design with no removals has not been designed.

How do I reduce social desirability bias in a survey?

Confidentiality assurances help less than design does. Use self-completion mode where possible, normalise the undesirable answer in the preamble ("some people do, some people don't"), consider indirect or third-person framing, and build option lists that make the less flattering answer easy to select. Assume that reported voting, exercise, giving, healthy eating and safety behaviour are over-reported.

When is a survey the wrong instrument?

When the objectives are not yet settled, when you cannot write the answer options because you do not know what people would say, when behavioural data already measures the thing more accurately than recall can, when the objective is a trade-off or a price point that needs a designed choice exercise, or when the sample cannot support the subgroup comparisons the objectives require. In each case the fix is upstream, and question quality will not rescue it.

What should a questionnaire pilot look for?

Cognitive testing first, asking a small number of people from the target population to say aloud what each question means to them and how they reached their answer, which is the only reliable way to find ambiguity and jargon. Then a soft launch inspected for completion time against estimate, break-off by question, straightlining, "other" verbatims that reveal missing options, "don't know" rates that reveal unanswerable questions, and any key measure whose distribution is too skewed to support the planned analysis.