Recruitment and Sample Sourcing

How do I work out whether a research sample is feasible?

Work backwards from the completes you need: divide by the expected completion rate, then the qualification rate, then the response rate, and check the implied reach against the plausible size of each source. Two things catch people out. Incidence is compound, so five criteria each passing 60% do not pass 60% overall, and the honest estimate depends on how correlated they are. And feasibility is governed by the smallest reportable subgroup, not by the total.

What is the difference between coverage error and non-response bias?

Non-response bias comes from people who could have been reached choosing not to take part. Coverage error comes from people the source could never have reached at all. Non-response is visible and worried about; coverage error is invisible, and it is the one that no sample size, quota structure or weighting scheme repairs.

Is it acceptable to use a client's customer list as the sample?

Often, and it is sometimes the only frame that reaches people with a real relationship to the subject. What matters is how the list was built. A list of all account holders and a list of marketing subscribers are different populations with the same name. A customer list cannot reach non-customers, and it usually excludes those who opted out of contact, who are the group most likely to be dissatisfied. Any absolute measure from it describes contactable, consenting, surviving customers, and the finding should say so.

What are professional respondents and how much do they matter?

Three different effects get lumped under the term. Conditioning: experienced respondents learn how instruments work and answer more consistently and less spontaneously. Motivation drift: where the incentive is the main reason for taking part, effort falls. Deliberate misqualification: a minority claim eligibility they do not have, and this rises with incentive value and with how obvious the screening criteria are. Each needs a different control, so it is worth separating them rather than treating them as one problem.

Can I blend two sample sources in one study?

Yes, and it improves coverage, but it introduces a systematic difference between subsamples that is easily misread as a real finding. Four controls make it analysable: record source as a variable on every respondent, set quotas within source rather than across it, keep the invitation, screener and instrument identical, and check key measures by source before any substantive analysis. Blending decided mid-field without these is a common route to an uninterpretable study.

How should I describe a non-probability sample in the methodology?

State the population, the frame, the source and how people entered it, the screening criteria and how each was established, the incidence achieved, the quota structure and targets, and the fact that the sample is non-probability. Do not attach a margin of error to it. Reserve "representative" for a stated meaning, such as quota-matched on named variables to a named population statistic, and say which variables.

How do I recruit a low-incidence or hard-to-reach population?

Change the structure rather than trying harder. Options include a short standalone screening survey with consent to re-contact qualifiers, screening against existing profiled variables where they are recent enough to trust, recruiting through organisations or communities where the population concentrates (accepting the coverage limit that creates), or accepting a smaller base and changing the claim from measurement to exploration. Where the population is hidden rather than merely rare, referral-based recruitment may be the only route, and the referral structure should then be recorded as data.

Why do markets look different when the difference is really the sample?

Because the composition of any given source type varies by market with connectivity, urbanisation, language and how research is regarded locally. The result is that cross-market differences partly reflect who each source reached, and that is indistinguishable from a real market difference unless the design separates it. A market that is an outlier on several unrelated measures at once should be checked for source composition before it is interpreted.

Does weighting fix a biased sample?

It corrects composition on the variables used to weight and nothing else. Where the selection mechanism relates to the outcome independently of those variables, weighting to demographic targets leaves the bias intact while making the sample look corrected. Weight where composition is off on variables known to relate to the outcome, state the scheme and the effective base, and do not present it as a repair for coverage error.

What if no adequate source exists?

Then the professionally correct output is that the study as scoped cannot be fielded. Name the claim that fails, offer the reduced claim the available sourcing does support, and say what would be needed for the original. Reducing what a study claims is a legitimate and frequently correct response; manufacturing representativeness in the write-up is not.