The interviewer is the one variable human research can never pin down. Two moderators are never the same person. An AI moderator is: one script, 495 times. So we asked the question, does the moderator change what you learn?
Short answer: a bad interview and a great one land the same conclusions. Only the great one hands you a quote worth using. Moderator quality is the lighting, not the scene.
You can't clone a moderator. So "did the interviewer shape the finding?" has always been a worry you carry. A fixed AI script changes that. Same interviewer, 495 times, and every way it varies is right there in the transcript.
The corpus: US airline travelers on booking and disruption under pressure. We sorted the interviews by how well the AI did its job, then checked whether the conclusions moved.
Every interview was quality-scored before we read it for content. Nine came in dead (no participant speech) and were cut, leaving 486. Names hashed to P-codes. The moderator score sits in a tight band (3.3 / 5, SD 0.55), one competent script, so we compare competent to more-competent, not to a genuinely bad moderator. Full vetting lives in the methods paper.
Two things came back. One reassuring, one not.
Same answer
Trust in airlines, anger at fees, the rest of what travelers care about, all the same whether the AI ran hot or cold. The numbers you'd report don't budge. The biggest gap was a rounding error.
Better stories
The vivid, specific quotes, the ones that actually move a stakeholder, show up far more when the moderator works. Same room. Better photo.
Neither is obvious. So we tested both. Below: one insight card at three quality levels, beside all 486 interviews. Watch the finding hold while the quote sharpens.
Turn the dial. Left: the insight card a researcher would write at that quality level. Right: all 486 interviews, your group lit up, the card is one of those dots. Now switch the outcome. The conclusions stay the same, the texture is what differs.
One caveat: "quality" here is the holistic score, which shares a halo with richness, so treat the texture gaps as an upper bound. Slice quality by talk-time, probe count, or mirroring instead, and the conclusions still overlap every way.
Interviewer effects were always an unmeasurable confound. One script run hundreds of times makes them a parameter, for any scripted AI study.
If conclusions are safe and quotes are fragile, build the thing that flags which transcripts hold the good ones. Read those.
Stop hand-checking headline numbers that don't move. Spend that time on the stories, the layer that does.
Ordinary travelers, the kind of sample you'd actually field. If moderator quality barely moved the findings here, it won't move them in the interviews you run. Fair warning on the shape: it skews younger than the flying public (nearly three-quarters under 45) and leans leisure (about nine in ten). Broad and varied, but not a census. Names hashed; every chart is the real sample.
Multi-select; % of the 495 who answered. Most have been burned.
Does moderator quality change what you learn? We took a 495-person airline study and checked.
One AI script, 495 interviews, so for once the moderator was a variable we could hold still. The AI moderator didn't run identically every time. And that was just enough variation to explore the question.
This shouldn't end with us. If you lean on AI-moderated research, or human, run the same check on your own data. See what your findings are made of.
Want to dig in together? Reach out to Nicole Zeng.