Weighting and Base Management

What can survey weighting actually fix?

It adjusts the composition of the sample you achieved so that it matches known targets on the variables you weight by. That is the whole of it. It does not fix coverage error, because people the sample frame never reached cannot be up-weighted from zero. It does not fix non-response bias on characteristics that were not measured, because the adjustment can only act on what it can see. And matching a population on age, gender and region says nothing about whether the people recruited resemble the population on the attitudes being measured.

Does weighting make a non-probability sample representative?

No. Weighting a non-probability sample corrects its demographic composition and leaves everything else where it was. The sample type still governs what claims are permissible, a margin of error is still not appropriate, and the disclosure should state the sample type and what the weighting adjusted as two separate facts so they are not read as one.

What is effective base size and why does it matter?

Weighting increases the variance of estimates, so a weighted sample of 1,000 does not carry the precision of 1,000 interviews. Effective base expresses what it does carry, derived from how much the weights vary, and it falls as they spread. It is the base that should govern precision claims, small-base thresholds and significance testing. Reporting a nominal n on weighted data overstates precision, sometimes substantially, and usually most in the subgroup that was hardest to recruit.

What is a good weight efficiency?

As a rough calibration rather than a standard: above roughly 90% is unremarkable, 70 to 90% is normal for a study correcting a real skew, 50 to 70% indicates a substantial correction that should be visible in the disclosure and prompts a look at recruitment, and below 50% means half the sample's statistical value has gone on the correction and the sample should be questioned rather than weighted. Efficiency is a fieldwork diagnostic before it is a technical statistic.

Should I cap extreme survey weights?

Capping reduces variance and raises the effective base, at the cost of leaving the sample not fully matched to its targets. It buys precision with bias, and neither capping nor not capping is automatically right. Decide by comparing the two on effective base, residual deviation from target and movement in the key measures, set the cap level with a stated rationale before seeing the effect on the headline where possible, and publish the comparison. Capping does not remove the reason the weights were extreme; it hides it, so keep the uncapped diagnostics.

What is the difference between rim and cell weighting?

Cell weighting matches the joint distribution of the weighting variables together, so each combination hits its own target. It preserves the interlock and needs populated cells, which is a problem when three variables produce forty-odd cells on a sample of a thousand. Rim weighting adjusts to each variable's marginal distribution in turn until all converge, needs only the marginals, tolerates more variables and produces more moderate weights, but does not control the joint distribution: every margin can be correct while an important interaction is wrong. If you use rim, inspect the interactions that matter.

When should a sample be rejected rather than weighted?

When the effective base falls to a fraction of the nominal, when the weight range spans an order of magnitude, when a handful of respondents are being asked to represent a large population group, when the sample cannot reach the target even with extreme weights, or when a subgroup with an adequate nominal base falls below reportable thresholds once weighted. The options then are to top up the deficient cells, restrict the claims, report unweighted with the composition disclosed, or decline to report the affected estimates. A heavily weighted sample is evidence of a recruitment problem, not a solution to one.

How should weighted survey results be reported?

Weighted percentages, unweighted counts alongside, and effective base wherever precision or a small-base rule is in play. A weighted count is never presented as a number of interviews, because it is not one. Where a grossed-up figure is genuinely needed for a volume estimate, label it a population estimate and never place it in a base row. Weighted and unweighted figures should never share a table without labels or a deck without a stated convention.

Where do the weighting targets come from, and does the source matter?

It matters more than the technique. A weight is an assertion that the population looks like the target, so a target needs a citable source, a date, a stated population, and category boundaries that match the questionnaire exactly. An age band that differs by a year at the boundary misweights everyone near it. Targets from a client's own records are usable and should be labelled client-supplied rather than presented as an independent statistic. If no defensible target exists, report unweighted and describe the achieved composition.

How do I weight tracker data consistently?

Treat the scheme as a versioned specification covering variables, targets, target source and vintage, approach, capping and the rule for updating targets, and record the version in the data itself. Apply the same specification every wave. When a change is unavoidable, make it at a declared break, run that wave under both schemes, and publish the difference. A weighting change that coincides with a real market movement cannot be separated from it afterwards.

Do I need to check weighting for subgroups separately?

Yes. A scheme built to correct the total does not necessarily correct any given subgroup, and a subgroup can be internally skewed inside a perfectly balanced total. Effective base within a subgroup is also frequently far below its nominal base, because the weight variance concentrates exactly where recruitment was weakest. Report nominal base, effective base, efficiency and a composition check for every subgroup you intend to report.