Sampling Strategy

How do I calculate the sample size I need?

It depends on which logic applies. For estimation you need the precision you require, the expected value of the estimate, the confidence level and any design effect. For detecting a difference you need the smallest difference that would change a decision, the split between groups, the variability, the threshold, the power and the number of comparisons planned. Without those inputs no number is derivable, and one produced anyway is a guess with a decimal point.

Why does my sample work for the total but not for the subgroups?

Because it was sized from the total. Subgroup requirements should drive the total, not the other way round, and crossed cuts multiply fast: six sectors by three size bands is eighteen cells, and the larger groups take most of the sample. List every cut that will appear in the report, work out the base each needs, and build upward.

What can I claim from a non-probability sample?

Description of the achieved sample, and comparisons within it with the selection mechanism stated. Not a margin of error, not a confidence interval interpreted as population error, and not a population estimate presented as such. No sample size changes this, and weighting to population totals corrects composition on the weighting variables without converting the design into a probability one.

Is a quota sample representative?

It is composition-matched, which is worth having and is not the same thing. A quota fills each cell with whoever in that cell was willing and available, so it controls who is in the sample without correcting why they answered. Its particular risk is that a total which matches population figures hides non-response bias inside it.

How many interviews do I need for qualitative research?

Enough to cover the variation the question spans, and then enough more to see whether new sessions are still producing new material. Build a variation matrix of the characteristics across which experience plausibly differs, allow enough sessions per cell to see repetition, and express the answer as a range with a minimum and a defined review point. Any number fixed in advance is a planning estimate, and should be labelled as one.

What is coverage error?

The gap between the population you want to describe and the frame you can actually reach. It matters not because someone is missing but because the missing group differs on what you are measuring. Write down who is undercovered, who is overcovered, whether anyone can appear twice, and the likely direction of the resulting bias. No sample size and no weighting repairs it.

Does weighting fix a bad sample?

It corrects known composition on the variables you weight by, and it costs effective base size everywhere. It does not correct unknown selection, it does not reach people the frame never contained, and it does not turn a non-probability sample into a probability one. Report the effective base, not the respondent count, in every base and every test.

What do I do if a quota cell will not fill?

Decide before fieldwork, not in week three. The options are: extend fieldwork, which shifts the sample toward late responders; relax the quota, which changes the population that cell describes; substitute an adjacent group, which changes what the cell can be called; or report short with the base shown and the limitation attached. Each costs something. Choosing calmly in advance produces a better choice.