Nobody abandons question one.
They abandon the sight of question twenty-five. Here is what the research says about asking the same things differently — and what it does not say.
- 22% of people who abandon a checkout say they left because it was too long or too complicated — not because of the price, and not because they changed their mind.
- Baymard Institute et al., Baymard Institute, 2024
- People answering through a chat interface gave more differentiated answers and were less likely to satisfice — to pick whatever ends the question fastest — than people answering the same survey on the web.
- Soomin Kim et al., CHI '19, 2019
- An AI chatbot that reads an open-ended answer and probes when it is thin drew significantly higher engagement and significantly better answers — more informative, more relevant, more specific and clearer — than the same questions asked as an ordinary web survey.
- Ziang Xiao et al., ACM Transactions on Computer-Human Interaction 27(3), 2020
A form shows you the bill before you order.
Twenty-five fields announce twenty-five fields. Twenty-five turns announce one.
This is the part that needs no study to see. A form renders its full length on arrival, and the decision a visitor makes is not about any single question — it is about the whole visible stack. Baymard Institute’s checkout research finds 22% of people who abandon a checkout say they left because it was too long or too complicated — ahead of the several other reasons people usually assume come first.
What the research actually found.
Four papers, none of them ours, none of them about a form builder.
- 01
People answering through a chat interface gave more differentiated answers and were less likely to satisfice — to pick whatever ends the question fastest — than people answering the same survey on the web.
A 2×2 experiment crossing platform (web against chatbot) with conversational style (formal against casual).
- 02
An AI chatbot that reads an open-ended answer and probes when it is thin drew significantly higher engagement and significantly better answers — more informative, more relevant, more specific and clearer — than the same questions asked as an ordinary web survey.
Around 600 participants split between a Qualtrics survey and a chatbot, and more than 5,200 free-text responses scored against Grice's maxims.
- 03
People disclosed more, and managed the impression they were making less, when they believed they were talking to a computer rather than to a person operating it.
Participants were told the same virtual interviewer was either automated or human-controlled, and were rated by observers on willingness to disclose.
- 04
Letting the interviewer explain what a question means, rather than reading fixed wording and leaving interpretation to the respondent, sharply reduced error in the answers.
Standardised interviewing compared against conversational interviewing, with answers checked against known facts.
So this is what it does.
Every behaviour below exists because of a finding above, not because it demoed well.
- It shows one question, not twenty-five
- The visible cost of answering is whatever is on screen. Nothing is hidden — the progress indicator is still there — but the decision to continue gets made in small pieces rather than once, at the door, about the whole thing.
- It follows up when an answer is thin
- Xiao et al. found probing is what turns a short open-ended answer into a usable one. A low-confidence read becomes a follow-up question rather than a recorded guess.
- It can explain what a question means
- Schober and Conrad found that letting an interviewer clarify sharply reduces error. Give it up to twenty knowledge entries and it answers from them, quoting you, then carries on exactly where it was.
- It is a machine, and says so
- Lucas et al. found people disclose more when they know they are talking to software. There is no invented persona pretending to be a colleague called Sarah.
- It never loses its place
- A state machine owns the flow, not the model. The model can record an answer, answer from knowledge, clarify, or skip — and every one of those is checked against the flow before it takes effect, so nothing gets reordered or dropped while it is being helpful.
- It shows you where people left
- Completion rate, per-question answer rate and drop-off, and median time to complete — for your form. Not an industry benchmark, which would be a number we made up.
What none of this means.
Four things the research above does not say.
That every form should be a conversation
Three fields and a submit button is already the right interface. A conversation there adds turns and collects nothing extra. Use a form.
That there is a completion-rate number
None of these papers measured completion rate on a commercial web form, and chatform has no cross-customer data to offer one. Anyone quoting you a single percentage for this is quoting marketing.
That chat is faster
Conrad and Schober found conversational interviewing takes longer. It trades time for accuracy. That is a good trade for an intake form and a bad one for a newsletter signup.
That the model should be trusted with the answer
It is not. Choice, scale and consent answers are matched exactly and never sent to a model, and everything the model does extract is re-validated against the same rules a typed answer would face.
References
- Ziang Xiao, Michelle X. Zhou, Q. Vera Liao, Gloria Mark, Changyan Chi, Wenxi Chen and Huahai Yang. Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. ACM Transactions on Computer-Human Interaction 27(3), 2020. doi.org/10.1145/3381804
- Soomin Kim, Joonhwan Lee and Gahgene Gweon. Comparing Data from Chatbot and Web Surveys: Effects of Platform and Conversational Style on Survey Response Quality. CHI '19, 2019. doi.org/10.1145/3290605.3300316
- Gale M. Lucas, Jonathan Gratch, Aisha King and Louis-Philippe Morency. It's only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior 37, 94–100, 2014. doi.org/10.1016/j.chb.2014.04.043
- Michael F. Schober and Frederick G. Conrad. Does Conversational Interviewing Reduce Survey Measurement Error?. Public Opinion Quarterly 61(4), 576–602, 1997. doi.org/10.1086/297818
- Frederick G. Conrad and Michael F. Schober. Clarifying Question Meaning in a Household Telephone Survey. Public Opinion Quarterly 64(1), 1–28, 2000. doi.org/10.1093/poq/64.1.1
- Baymard Institute. Checkout Optimization: Minimize Form Fields. Baymard Institute, 2024. baymard.com/blog/checkout-flow-average-form-fields
Comparing tools rather than formats? The comparisons, including where we lose.