Cross-Tabulation

How do I decide what goes in a cross-tab banner?

Start from the questions the study exists to answer and write, against each, the comparison that would answer it. A banner point earns its column by appearing in that list. Adding every available demographic looks thorough and is the opposite: it dilutes the analysis, multiplies the comparisons and guarantees false positives. Keep the excluded variables in a second-tier list to be run only if a first-tier result needs explaining.

What is the difference between column percentages and row percentages?

Column percentages answer "of the people in this group, what proportion gave this answer" and each column sums to 100. Row percentages answer "of the people who gave this answer, what proportion are in this group" and each row sums to 100. A product bought by 4% of under-35s and 1% of over-55s is four times more popular among the young, and most of its buyers may still be older, because there are more older people. Both statements are true and they come from different directions of the same table. Reporting one while describing the other is the commonest false statement in commercial research.

What base size is too small to report in a cross-tab?

Below 30 in a cell, no percentages at all: report counts, or report the cell as too small. Between 30 and 99, show the base, flag it and read directionally only. On weighted data the threshold applies to the effective base, not the nominal one, so a column of 120 with a design effect of 1.7 behaves like a column of 70. The important rule is what happens below the threshold: a cell too small to report is reported as too small, with its base, rather than left out. A blank tells the reader nothing was found there, which is a claim, and usually a false one.

Why are there so many significant differences in my cross-tab?

Because a banner runs an enormous number of comparisons. Thirteen columns produce 78 pairwise comparisons per row; thirty columns produce 435. A thirty-column banner across forty questions is well over a thousand comparisons even on a disciplined reading, and at a 5% threshold roughly one in twenty flags with nothing true anywhere. The false positives are arithmetic, not bad luck. Count the comparisons before running them, report the count alongside the findings, and label anything that was not predicted in advance as exploratory.

How do I read a cross-tab properly?

Read the base row first, then read across the row rather than down onto whichever number is furthest from the total. Ask what shape the row has. A steady gradient across ordered columns is credible, because noise does not usually arrange itself in order. A single column standing away from a flat row is a hypothesis, and it is very often the cell with the smallest base, since small bases produce large swings. Only after the shape of the row is described should any individual cell be singled out.

Can a subgroup difference in a cross-tab be caused by something else?

Routinely. Banner points are correlated with each other, so a regional difference may be an age difference wearing a regional label. The diagnostic is to nest one banner point inside the other and see whether the gap survives. In the extreme case a relationship can hold inside every subgroup and reverse in the combined table, because the subgroups differ in size and in base rate. A cross-tab cannot separate correlated variables, so either run the nested check, or say the difference is confounded and move to a model.

Can I compare two columns in my banner directly?

Only if a respondent cannot appear in both. Age bands are fine. "Users of product A" and "users of product B" usually are not, because people use both, and comparing overlapping groups with a standard two-sample test inflates significance. Test the exclusivity in the data rather than assuming it from the labels, and mark any overlapping column in the header so nobody compares it later.

Does a cross-tab tell me whether a difference is real?

No. It shows you the difference. Whether the difference is larger than sampling variation is a separate question requiring a test, and that boundary is where cross-tabulation ends. Describing a gap arithmetically with both bases shown is legitimate. Words like "notably", "clearly" and "significantly" are claims about reliability and need a test behind them, whatever flags the tabulation software has printed.

Can AI produce and read cross-tabs reliably?

It applies conventions consistently, which is a real advantage on a large tabulation. The risks are specific: building a banner from whatever variables exist rather than from the objectives, narrating every gap in the table as though each were a finding, describing a percentage in language that matches the other direction of percentaging, and inferring what a banner point means from its label. Used properly, AI should be required to state the direction of percentaging and the base with every figure, report the count of comparisons its findings were selected from, and label anything unpredicted as a hypothesis.