Trend and Tracker Analysis

How do I know whether a change between two waves is real?

Four things in order. Confirm the two readings are comparable at all, because a rewording or a sample source change produces movement that has nothing to do with the world. Place the movement against how much the metric normally moves when nothing has happened. Test it, and report the confidence interval on the change rather than a flag. Then check whether the calendar or the sample explains it. A movement that clears all four is reportable; one that fails the first is not reportable at any size.

What has to stay the same between tracker waves?

Seven things: question wording including preamble and response order, the scale and its labels, the routing that brings respondents to the question, the base definition including don't know treatment, the sample source and mode mix, the weighting scheme and its targets, and the fieldwork conditions. Record each as holding, changed or unknown, every wave. Unknown is not the same as holding, and an unverifiable precondition is disclosed rather than assumed.

We have to change a tracked question. What are the options?

Run both versions in the same wave on split samples. That measures the size of the step on the same population at the same moment, and it is the difference between a series that continues and a series that restarts. With the step measured you can either present the series as continuous with the break marked, or restate the history on the new basis with the adjustment documented. Without a parallel run, do not construct a bridge from judgement: state the break, start a new series, and keep the old one available.

How much movement is normal in a tracker?

It depends on the metric and the base, and the only way to know is to look at the history. Take every wave-on-wave change the metric has ever shown and look at the spread. A measure that has moved between minus 4 and plus 5 points for sixteen waves has a noise floor of about 5 points, and a 4-point movement this wave is not news whatever a test says, because the same test would have flagged half the previous waves. Put that band on the chart so everyone can see what an ordinary wave looks like.

Why does my tracker show a smooth trend that nobody can explain?

Very often because the sample changed rather than the world. Panels age and refresh unevenly, recruitment routes shift, device and mode mixes change, and weighting only corrects the variables it targets, so drift on anything untargeted passes straight into the series. The diagnostic is to track the unweighted sample profile wave by wave, watch weighting efficiency, and carry a control question that should not move in the population. If that control has moved steadily, the trend is in the sample.

Should I compare with the last wave or the same wave last year?

Both, for different purposes. Wave-on-wave is the noisiest comparison and the most over-read. Year-on-year removes seasonality and is usually the more reliable single comparison, at the cost of being slower to detect a genuine shift. Neither is as informative as the trend line across all waves, which is what a tracker was built to provide. Read the line before the latest two points.

Can I say the campaign caused the movement?

Not from a tracker alone. A before-and-after reading establishes that a number differs at two times; everything else also changed between those dates. State the coincidence of timing as a candidate explanation, name what would test it, and run the test where the data allows. Where fieldwork spans the event, splitting the wave by interview date is often the strongest evidence a single tracker can produce. A causal claim needs a design that licenses it, and the licensing design must be named alongside the claim.

Is "no change" an acceptable finding?

Yes, and frequently it is the correct one. A stable measure says the market did not shift, or the campaign has not landed, or a decline has stopped, and each of those is actionable. The requirement is honesty about what stability means here: state the minimum movement the wave could have detected, because a null on a small base is not evidence of stability. Reports that cannot say "nothing moved" usually end up saying something false instead.

Can I put a rolling average and a discrete wave figure on the same chart?

No. They describe different things and move differently: a rolling figure smooths noise and lags real change by roughly half its window. State which convention a chart uses, keep one per chart, and never test two overlapping rolling windows against each other, because consecutive points share respondents and are not independent.

Can AI analyse a tracker reliably?

It applies conventions consistently across waves, which is exactly what a tracker needs. The risks are specific and serious: it will compare two figures that appear in the same table without knowing the questionnaire changed between them, it will write a fluent narrative over a completely flat series, and it will slide a causal connective into a sentence about timing. Used properly, AI should be required to complete the comparability record before comparing anything, place every movement against the metric's normal range, report the interval on the change, and state the detectable movement beside every claim of stability.