One thousand three hundred seventy-one people entered a study with one thing in common. Each had a difficult conversation that they were considering having with someone they knew. The person was not always the same. For some, it was a partner, others a parent. It could have been a friend, or someone at work.
Half of the people in the study spent six to twelve minutes preparing with a chatbot. They planned what they wanted to say, asked for advice, and in many cases practiced how the conversation might go. The other half used a chatbot to answer neutral trivia questions that had nothing to do with the conversation they were about to have.
A week later, 65 percent of the people who prepared had gone through with the real conversation. In the other group, only 59 percent had done it.
The most interesting part to me is that the preparation did not make people feel better in the moment. It made them feel slightly worse. They had more negative emotion after thinking seriously about the conversation. Then more of them went and had the difficult conversation anyway.
I usually write about the things AI allows us to avoid. This is an example of AI helping someone do the opposite.
Another new study this week complicates that result a bit, however. Across three experiments, followed by a 28-day study, people who used AI for emotional support became more willing to choose AI again the next time they needed some support instead of a person. In the month-long study, daily conversations shifted preference away from people when the conversations became personal.
This to me is a perfect example of “both things can be true.” One use of AI ended in a human conversation that had been delayed. The other use made the machine more likely to receive the next personal conversation too.
This week's Reckoning asks what might be the most important question we can ask when AI enters a relationship. Does the help send the person back to another person, or does it quietly close the loop itself over time?
|
65%
prepared with a chatbot and had the real conversation
|
59%
used the chatbot for trivia and had the real conversation
|
THE CONVERSATION THAT HAPPENED
The chatbot helped people do the human part.
Before the experiment began, researchers screened more than 1,800 people. Eighty-one percent could name a difficult conversation they were thinking about having. Most of the conversations involved a partner, family member, or friend. Others involved a boss or coworker.
The people assigned to prepare with the chatbot used it in a few different ways. Sixty-four percent planned what they would say, and most of that group role-played the conversation. Others asked for advice, tried to anticipate a reaction, or worked through how to keep the conversation calm.
The effect was not enormous, but it is noticeable. Preparation moved the rate of having the conversation by six percentage points. The follow-up also depended on people reporting honestly whether the conversation happened. But this was a preregistered randomized study with a real action measured a week later, which makes it stronger than most work in this area.
There was also no evidence that the chatbot produced a polished fantasy that made the human conversation feel disappointing. Among the people who went through with it, satisfaction and perceived quality were similar in both groups.
What I take from the study is fairly specific. AI can support a relationship when the interaction is preparation for the relationship, and the person still has to walk into the room, feel the discomfort, listen to an answer they did not control, and respond in real time.
That is different from replacing the conversation. The study did not measure whether repeated rehearsal could eventually make people less able to speak without it. The researchers say that question remains open. For this week, though, the chatbot acted like a bridge for people thinking about a difficult conversation, and more people who used it reached the other side.
READ THE PEER-REVIEWED STUDY →
THE PREFERENCE THAT MOVED
The support worked, and the next choice changed.
A separate research team tested what happens when people choose between emotional support from a person and support from AI. Across three experiments involving 1,951 participants, people first chose where they wanted to share something emotional. The researchers then sometimes honored that choice and sometimes assigned the other source.
AI support was rated as better than human support only by the people who had chosen AI in the first place. When AI was assigned to someone who had wanted a person initially, the advantage disappeared.
The part that scares me a bit is the finding that came after the interaction with the AI. Regardless of what people had originally chosen, interacting with AI made them more willing to choose AI the next time they needed some support.
28 DAYS
Daily personal conversations gradually shifted preference toward AI and away from human support.
The researchers followed 981 people for 28 days in a study conducted with OpenAI. Daily conversation gradually shifted preferences toward AI and away from human support, but only when those conversations became personal.
This remains a preprint and has not completed peer review. Four weeks does not prove dependency, and it does not tell us whether the preference would last. I also would not use the study to argue that anyone who gets comfort from AI is doing something wrong.
But it does make one common defense more complicated. We often treat preference as the final proof that a product is serving the person. This study suggests that exposure is partly shaping the preference we later use to justify more exposure.
The first conversation can feel like a choice. Each successful conversation makes the same choice easier the next time we have to choose who or what we want to engage with. And people are unlikely to announce that a human conversation has been replaced by AI, so we will not see it happening. The person simply becomes less likely to select the human, and that becomes a habit, and as we know, once a habit is formed it is incredibly tough to break.
READ THE PREPRINT →
THE REQUEST THAT STOPPED ARRIVING
The family member was not rejected. They were just never summoned.
A small qualitative study looked at six older adults and seven younger relatives who had often helped them with technology. Researchers sometimes call that relative a “warm expert.” It is the person you trust enough to ask when the website will not work, the form you need to fill out makes no sense, or the answer does not look right. Most of us have this person in our lives.
The older adults described using chatbots partly so they could rely on family less often. Admittedly, you can say that can be useful. It can preserve independence and remove a small burden from someone who may not be available at the exact moment help is needed.
The quiet cost (wink, wink, Book Plug!) appeared when the chatbot did not visibly fail. There was no frozen screen or error message that told the younger relative a problem existed. Family members had limited signals for when help was needed, so they depended on partial disclosures and general warnings.
Even when a relative stepped in, the researchers found that the immediate problem was often solved without leaving knowledge the older person could reuse the next time.
This was a thirteen-person formative study and a workshop paper, not a population estimate. Its value is in what could be lost, more from the side of the person who usually helps. The family member did not decide to stop helping. The older adult did not necessarily decide to exclude them. The request simply stopped happening, and nothing really signaled its absence.
That is big to me, because being asked is one of the ways a person knows they are needed. The task may be solved privately and efficiently while the relationship loses one of the ordinary transactions that used to keep it active. Being needed is something fundamental to all of us, and AI's effectiveness threatens to take that away more and more every day.
READ THE QUALITATIVE STUDY →
THE WORDS THAT DID NOT FEEL LIKE YOURS
The guilt was about concealment, not assistance.
Five preregistered experiments asked people to imagine using generative AI to write an emotionally important message, including a thank-you note, birthday card, or love letter. People felt more guilty when they used AI than when they wrote the message themselves.
The researchers found that the same guilt appeared when a friend secretly wrote the message. A preprinted greeting card, meanwhile, produced almost no guilt. The difference was not the AI, necessarily. It was whether the recipient would believe the words were felt by the person who sent the note or letter.
The guilt also weakened when the message was never sent and when the recipient was not someone close. The closer the relationship, the more authorship mattered, and the guiltier people felt for using AI.
These were imagined scenarios and self-reported feelings, not messages exchanged inside actual relationships in real life. Not yet, at least. They do not tell us whether AI assistance damages trust. What they do show is that people treat the effort that they put into a personal message as part of the message itself.
That brings this week's studies back to the same line. AI can help someone find what they need to say before a difficult conversation. It becomes a different kind of help when it speaks in their place and allows the other person to believe the reaching for those words and feelings was theirs.
READ THE PEER-REVIEWED STUDY →
THE RECKONING
I do not think there is a line that we should draw that says AI should handle facts and stay away from feelings. This week's strongest study is a real example of AI helping people do something human they had been avoiding.
The better thing to figure out has to do with sequence. What happens after the AI interaction?
The chatbot can help a person prepare, find the point they are afraid to say, or notice what they need from a conversation. Then the person still has to call, sit down, listen, and respond to another person who did not follow the script that they had with some AI agent.
The concern begins when the AI becomes the place the need ends. It receives the next personal conversation too, and the human does not. It solves the question before a family member knows there was one. It writes the words and leaves the recipient believing the effort belonged to the sender, and we do it enough times that eventually the initial guilt subsides.
The end result can look good in every case because the person feels supported. Or the problem that seemed so daunting gets solved. That is exactly why the next step matters more than whether the interaction felt helpful. Too many people fear what will happen if AI “goes wrong.” I continue to worry about what happens to us if it continues to get better.
My advice is that before you use AI for something personal, decide where the interaction is supposed to end. If the answer is with another person, make sure that person still gets the chance to be there for you. Taking that from them could lead to an emotional loss for them that will cause them to reach for AI instead of a person, and that person they would have reached for in the past now loses part of their purpose, and down and down the rabbit hole we go.
ALSO THIS WEEK
Five more findings worth keeping in view.
A fake disease reached 69 percent of the junior radiologists. Researchers planted an invented diagnosis inside an AI assistant. Eighteen of twenty-six radiologists with six months or less of specialty training accepted it, while none of the fifteen more experienced radiologists did. This was a small preprint experiment in one specialty, but it shows why judgment has to exist before the person can supervise the tool. Read the preprint.
Half the gaps between AI tutor questions contained no independent work. TutorTrace followed 480 programming students and more than 180,000 actions. When the tutor began using behavior between questions as context, intervals containing no independent work fell from 50.0 percent to 20.7 percent. The early classroom result is preliminary, but it shows that effort can be measured and restored by design. Read the paper.
Writing variation fell between 21 and 50 percent. A peer-reviewed Nature Human Behaviour study examined more than 880,000 texts across seven datasets. LLM polishing preserved the content while making styles more alike, reducing variation in writing complexity and muting clues that language carries about identity and social context. The study identifies a broad trend, not the cause of any one person's writing change. Read the study.
Six deliberate obstacles raised ownership every time. Memory improved only sometimes. Two researchers tested six forms of friction in a chatbot with twenty-four participants. Every version made the work slower and harder, and every version increased the feeling that the result belonged to the person. Memory gains depended on the obstacle and the task. Friction is not automatically useful just because it is friction. Read the preprint.
Four leading models talked more than they listened. In sustained mental-health support conversations with carefully designed synthetic help-seekers, all four systems moved toward problem-solving before the situation had been explored and performed similarly on calming. No real person in distress took part, so the paper tests a design pattern rather than a human outcome. Read the preprint.
PRACTICE
Prepare yourself, then talk to the person.
This week, open the Stay Raw studio at The Quiet Cost Practice and use the Stop-Line.
Choose one conversation you have been postponing. Write only two things: what you are feeling and what you need from the conversation.
If you use AI after that, use it to help you find one opening sentence or one point you do not want to forget. Stop before it scripts the other person's reactions or maps the entire conversation.
Then close the tool and contact the person. The preparation has done its job only when the real conversation begins.
OPEN THE QUIET COST PRACTICE →
YOUR TURN
Where is the conversation supposed to end?
Think of one personal question, worry, or difficult conversation you took to AI recently.
Did the interaction help you reach another person, or did it make the other person less necessary?
What would it look like to use the same help as a bridge this week?
Write for five minutes before asking a machine to answer. Then make one call, send one message, or ask one person directly.
PASS IT ON
If this Reckoning made you think of someone, forward it to them. If it was forwarded to you, join us at quietcostweekly.com.
Michael McNamara