AI-Moderated Interview Design

Can an AI-moderated interview replace a human-moderated depth interview?

Not straightforwardly, and the honest answer depends on which comparison you are making. Against a skilled human depth, AI moderation loses the ability to build enough rapport to ask a harder question, to notice that an answer is defended rather than complete, and to follow a genuinely unanticipated line. Against the realistic alternative in most large studies, which is an unprobed survey open end, it gains a great deal. State in the methodology which comparison your study is making.

How do I stop an AI moderator from leading the respondent?

Four rules do most of the work. Reflect only the respondent's own words rather than a cleaner paraphrase. Ask for expansion, never for confirmation. Never offer a candidate reason, even as an example. Never evaluate an answer. Then apply the inversion test to every probe: if the opposite of the implied thing were true, would this probe make it as easy to say? The structural fix is to force an explicit classification of the answer before a probe is selected, so the probe attaches to a missing component rather than to whatever the model found salient.

How many follow-up questions should an AI interview ask?

Set a bounded probe budget per section, decided against what that section must produce. Two to four follow-ups is a common working range, but treat that as a design convention rather than a validated optimum and calibrate it in pilot. What matters more than the number is that probing stops on a defined condition: sufficiency met, budget exhausted, two consecutive non-responsive answers, or any refusal, which is honoured immediately.

What is a sufficiency definition in an AI interview?

The written condition under which the interview may stop probing a section and move on. It names the components an answer must contain, not a length or a probe count. For an episode, that might be a specific occasion identifiable in time, what the respondent was trying to do, what happened, and what they did next. If two researchers cannot agree whether a given answer meets the definition, it is not written tightly enough for a machine to apply.

How should an AI interview handle distress or a safeguarding disclosure?

Safeguarding overrides every other rule. Specify the trigger categories, and on any of them the moderator stops probing entirely, acknowledges briefly without therapeutic language, offers a route to a named human who is genuinely available during fieldwork, and signposts appropriate external support. It never continues probing the distressing material, never offers advice, and never judges severity in order to decide whether to escalate. If no confirmed human route exists, the study should not field.

Do respondents have to be told they are talking to an AI?

Yes, before consent, in plain language, and in the mode the interview uses. They should also be told what is recorded, whether humans read transcripts, whether verbatim quotes may be reported, that they can stop or skip at any time, and how to reach a person. Beyond the ethics, the disclosure is a research variable: its wording plausibly affects what people say, so hold it constant across the study and reproduce it in the methodology.

Do people say more or less to an AI interviewer than to a person?

Both effects are plausible and probably both occur in the same study on different topics. Older work on computer-administered self-interviewing supports the idea that some socially regulated or embarrassing material is disclosed more readily without a person present. Against that, the effort of articulating something difficult is partly motivated by a person visibly receiving it. Treat either direction as a hypothesis in your own study rather than a settled finding, and design to look for evidence of it.

Where is AI moderation the wrong method?

Where distress or clinical material is foreseeable, where the value depends on rapport built over time, where the analytic point is what is not said, where group interaction is the data, where the objective is prevalence, and where the population cannot be assumed comfortable with an extended exchange with a system. Volume of verbatim is not depth, so a study designed mainly to produce quotes at scale is also the wrong use.

What should I check in the transcripts before analysing them?

Score every transcript, not a sample: whether each section met its sufficiency definition, how many probes actually fired, whether the respondent adopted vocabulary introduced by a probe (the leading signal), length and elapsed time, where any abandonment happened, and whether an escalation triggered. Set the inclusion threshold before fieldwork, log every exclusion with its reason, and carry the per-section sufficiency record into analysis so a weakly evidenced theme can be recognised as one.

How do I pilot an AI-moderated interview design?

Run a small number of interviews and read every transcript in full, not in extracts, against four questions: did each section meet sufficiency and which component was missing; did any probe lead; were there answer states the design does not cover; and where did probing stop too early or run on. Then rebuild the tree and run again. A pilot that produces no changes means nobody read the transcripts closely enough.