What Happens When We Ask AI Instead?
When someone asks AI a question, we can usually see what they get back from it in response. It is harder to see what would have happened if they had asked a person, worked through the problem themselves, or put the question somewhere other people could learn from it.
That doesn't mean the other way would have been better. Sometimes nobody was available to help. Sometimes the person would have stayed stuck or got the wrong information. But if we only look at the answer, we can miss some of what changes outside of just the end result.
This week's research gives us a few extra places to look. A large analysis of Stack Overflow examines the decline of public questions after generative AI became part of the norm. An experiment with future teachers tests whether making a decision before seeing AI advice helps them catch mistakes. In a classroom writing study, children produced better stories with AI, but reported feeling less absorbed in the work. And interviews with university students help explain why a chatbot can become the easier place to talk.
There are useful results in both directions here. As always, I want us to keep paying attention to the help people are getting, as well as the things they may stop doing because that help is so good.
01 / THE RESEARCH
The questions that no longer become public
Preprint • Observational analysis of Stack Overflow • Not yet peer reviewed
|
2M+
questions
Researchers examined questions posted from 2020 through 2025. Easier questions became a smaller part of what remained.
|
Myokyung Han and colleagues studied how questions on Stack Overflow, the programming question-and-answer site, changed after generative AI became widely available. Posting fell sharply. The decline was uneven, with basic questions and topics well represented in existing online material losing ground relative to more advanced ones.
This does not establish that every missing question went to a chatbot. The researchers could not follow each person who stopped posting, and other changes to the platform and its users could have contributed. More difficult questions making up a larger share also does not mean their total number grew.
What interests me is the difference between getting an answer privately and leaving an answer somewhere. Imagine somebody fixing a small coding problem after reading a public exchange from three years ago. They didn't need to know either of the people who had that conversation. They benefited because one person was willing to ask and another was willing to explain it where others could see.
AI can make that original exchange unnecessary for the person with the problem. They may get a faster answer without having to worry that their question is too basic. That is a real benefit, especially for a beginner. But if the question and answer stay inside a private chat, the next beginner cannot find that exchange in the same way.
I would like to know how much useful material stops being created as a result. We tend to notice when something is removed from the internet. It is harder to notice an explanation that nobody had a reason to write publicly in the first place.
Read the Stack Overflow preprint →
02 / THE RESEARCH
Deciding first helped people resist bad advice
Peer-reviewed randomized experiment • 218 future teachers in the main analysis
|
53.5%
Independent judgment first
|
37.3%
AI advice first
|
Correct choices of both plan and reason on scenarios containing misleading AI advice, comparing independent judgment first with AI advice first.
|
Dong Yan asked preservice teachers to work through classroom scenarios. Some made their own decision before seeing an AI recommendation. Others saw the recommendation before deciding. A third group worked without AI advice.
Half of the recommendations were deliberately misleading. On those scenarios, the people who decided first were more likely to choose both the right plan and the right reason. When the advice was correct, the two AI groups performed similarly.
This was a scenario experiment at one university, not a test of teaching in an actual classroom. The researchers also did not establish a clear advantage on a later test without AI. And because they inserted the bad advice themselves, the study tells us nothing about how often an ordinary chatbot gives a teacher the wrong answer.
Still, there is something practical here. Reviewing a recommendation sounds like an independent act, but the recommendation has already given you a place to begin. You may find yourself asking whether its explanation makes sense without considering another explanation at all.
Having a first answer gives you something to compare it with. If you thought a student needed more time and the system recommends a penalty, you have to deal with that disagreement. You may change your mind, and sometimes you should. But you at least know there was a choice to make.
I wouldn't turn this into a rule that people must always struggle alone before getting help. There are plenty of situations where they need guidance just to understand the problem. When they do have enough information to make a judgment, however, I like the idea of asking them what they think before showing them what the machine thinks.
Read the independent-judgment experiment →
03 / THE RESEARCH
The writing improved, but the children felt less absorbed
Peer-reviewed classroom study • Publisher's abstract available
|
78
primary-school students
Children wrote with AI, with a classmate, or on their own. The AI group had the highest creative-writing scores and the lowest reported flow.
|
Xin Liu and colleagues compared three ways of writing a story. Twenty children used generative AI, 38 worked in pairs, and 20 wrote individually. The children receiving AI support produced stronger creative writing than either of the other groups.
They were not simply handing in whatever the system produced. The researchers report that they generally adapted ideas selectively, and none copied an entire AI response verbatim. Their writing confidence was also higher than that of children working alone, though not clearly higher than that of children working with a partner.
The AI group nevertheless reported the lowest flow, meaning the feeling of being absorbed in what they were doing. Flow and writing performance were not significantly related in this small sample. The study did not establish a lasting change in creativity or in children's willingness to write.
If a teacher looks only at the finished story, the AI group looks like a success. And in that respect, it was. But a child spending time inventing a character is also finding out whether they enjoy making something up. The story is one reason to do the assignment. Becoming interested enough to want to write another one matters too.
I would want to hear from these children about where they felt involved and where they started waiting for suggestions. That would help us understand how to use the tool without assuming that a better story tells us everything we need to know about the experience of writing it.
Read the creative-writing study →
04 / THE RESEARCH
Why a chatbot can feel like an easier place to talk
Peer-reviewed interview study • One UK university • Not a treatment trial
|
16
students describing their own experiences
Participants reported loneliness and had used conversational AI for emotional or relational support. Their accounts do not establish that AI caused or relieved loneliness.
|
Yanyan Li and Tom Ritchie interviewed university students about why they turned to AI when they wanted support. Some found it easier to say something embarrassing or difficult when they did not have to worry about being judged or burdening another person.
There were practical reasons as well. Friends could be asleep or in another time zone. Counseling could involve a wait or limits on appointments. Sometimes the appeal of the chatbot was that it was there when somebody else wasn't.
The students still valued human relationships. These interviews do not show that AI was a good substitute for those relationships, or that using it improved or worsened their lives. They help explain the appeal, which we need to understand before telling people to make a different choice.
I don't think we can read this and simply say that students should call a friend. If someone has tried to get support and found a waiting list, being told to choose a human does not create an appointment. And if they are worried about overwhelming a friend, that worry needs to be taken seriously too.
At the same time, I wonder about the friend who would have wanted to know. A private conversation with AI can help somebody get through an evening, but if it ends there, the people in their life may never learn that the evening was difficult. That is a possibility these interviews cannot measure, and it is one I would want longer studies to follow.
The question for the rest of us is how to become easier to approach without expecting people to arrive with everything already explained. A friend may need to repeat themselves or take a while to get to the point. If we want people to come to us, we have to leave some room for that.
Read the university-student interview study →
|
THE RECKONING
The Stack Overflow study is the one I kept coming back to this week as I was writing Reckonings. A person can get exactly what they need from AI, leave satisfied, and still change something for people who were never actually part of the exchange. Those people aren't necessarily worse off immediately. There may just be one less explanation for them to find later, and this ties into what I always say about “The Quiet Cost.”
This newsletter has talked before about the person who never gets the call when someone needs help. There is a similar question here about the person who never gets the chance to answer. Helping someone can be how a colleague realizes what they know, or how two people begin to trust each other. We don't need to preserve every interruption at work to admit that some of them lead to something useful beyond solving the original problem.
The difficult part is that we won't always know which ones those were. An awkward question can begin a friendship, and I can tell you that from experience. It can also just be an awkward question, and people are allowed to want a quick answer without turning it into a relationship. I don't want to make asking AI sound like a failure to be a good person.
What I do want is a better account of what happens when a private answer becomes the usual choice. Does somebody still get the chance to think through the problem? Does the useful explanation reach anyone else? If a person is having a hard time, does someone who cares about them eventually find out?
The research cannot answer all of that yet. For now, there are ordinary things we can pay attention to. We can share a solution that helped us, explain how we arrived at a decision, or follow up when someone seemed like they wanted to say more. AI can help with the original question and we can still make a point of doing those things.
|
Also this week
A few more findings from the September 29 through October 5 sweeps. These are worth reading alongside the main stories, including the results that point toward benefits.
The chatbot made people state a view before it would draft
A preprint involving 398 adults tested a system that required a position and a reason before unlocking draft generation. Participants later checked material faster, but overall accuracy was not higher. The order of the work changed more clearly than the final result.
Read the Engage-to-Unlock preprint →
Writing the first draft yourself was not always the best arrangement
In a peer-reviewed study of 101 university students learning English, the largest writing gains came when AI drafted and students evaluated and revised. This was a quasi-experiment, not a universal rule, but it is a useful reminder that revising can involve substantial thought when students actually do that work.
Read the university's study abstract →
AI search broadened the news people encountered
A randomized preprint involving 37,561 Washington Post readers found that AI answers with citations broadened topic exposure while increasing what readers encountered in common. Both groups searched the same newspaper archive, so this does not establish what happens across the wider internet or how much people understood.
Read the news-search preprint →
The AI math companion helped with anxiety more than scores
In a randomized study of 174 fifth graders, AI-supported math games reduced anxiety and improved motivation relative to games without AI and traditional instruction. Both game groups did better on achievement than traditional instruction, with no clear extra achievement benefit from adding AI.
Read the math-learning study →
Human researchers brought more varied ideas
A preprint comparing 25 language models with 73 human researchers found strong individual AI performance on theory-building tasks, while human theories were more diverse and benefited more from being combined. The comparison concerned one research domain, not a verdict on human and machine creativity in general.
Read the theory-building preprint →
Feeling close to an AI companion was linked to wanting to pay
A peer-reviewed survey of 393 frequent companion-app users in China linked stronger emotional bonds with intentions to keep using the apps and buy features. This was an association in self-reported intentions, not proof that the apps caused loneliness or made people spend more.
Read the companion-app survey →
Sherry Turkle's new book is out
In an MIT News interview about Artificial Intimacy, Turkle argues that the ease of talking to a chatbot can change what people expect from one another. This is an interview about her book, not a new experiment, and it is worth reading alongside the students' accounts above.
Read the MIT interview →
|
PRACTICE
Try Ask a Person, or put your judgment to work in Signal
At The Quiet Cost Practice, the Ask a Person studio starts with a question you were about to give to a machine. It helps you choose somebody you could ask and work out how to begin. Keep it small. Ask a colleague how they learned something, or a friend what they liked about a place they visited. See what they tell you that you wouldn't have known to put in the prompt.
If you feel like a challenge instead, try Signal. You get a claim and material to check it against, then decide whether the claim is supported, contradicted, or still unresolved. The interesting part is catching the gap between what sounds convincing and what the source actually says. You can see how you did after making your choice.
These are chances to practice asking and checking. We haven't established that using the app prevents AI-related skill loss, but we can give you something enjoyable that asks you to use those skills yourself.
Try the studios at The Quiet Cost Practice →
|
|
YOUR TURN
Think of one question you recently asked AI that you might once have asked a person. Who would that person have been? What, if anything, might have happened beyond getting the answer?
Take five minutes to write about that specific example. It is fine if you conclude that AI was the better choice. If the question reminds you of somebody you haven't spoken to in a while, you could ask them something else.
|
Pass it on
If somebody came to mind while you were reading, forward this to them. Reckonings is free each week, and they can subscribe or read the earlier issues at quietcostweekly.com. My longer essays are at At a Cost on Substack.
Subscribe and read earlier issues →
At a Cost →
Michael McNamara
quietcostweekly.com →
Illustrations created with AI. They are editorial images, not depictions of research participants.