Can an AI Summary Change Your Memory?
People in one of this week's experiments watched a short video of a traffic accident. Then, a day or two later, they read a summary that described a detail incorrectly. When asked about the video, many remembered the detail from the summary instead of the one they had actually seen.
This interests me, because it is a different problem from getting an answer wrong. We tend to assume that if we were there, or if we read the original, we will be able to notice when the summary makes a mistake. In this experiment, reading the mistake made that harder.
This week's Reckonings looks at that study, another on how AI drafts change the emails people write, and a small experiment that helped students get more out of asking for help. We also have a study of teenagers who used AI for a personal problem and still involved another person as well. That last one is significant because we should be looking closely at the cases where AI helps someone connect with a human being, along with the cases where the conversation stops with the machine, and we lose connection with other humans.
01 / THE RESEARCH
The summary changed what people remembered seeing
Preprint posted September 23 • Randomized memory experiment • Not yet peer reviewed
|
83.6%
Accurate summary
|
44.8%
Misleading summary
|
Accuracy on the traffic-sign question after an accurate summary versus a misleading one. These figures concern one detail, not overall memory.
|
Mattea Sim, Yael Eiger, and Tadayoshi Kohno analyzed responses from 328 adults who watched an animated traffic accident. After a delay of 24 to 48 hours, participants read a summary. Some summaries correctly identified the traffic sign. Others substituted a stop sign for a yield sign, or the other way around.
The researchers inserted the false detail deliberately. They were testing what misinformation did to memory, rather than measuring how often an ordinary AI summary would make that mistake. Labeling the summary as AI-written or human-written made no clear difference to the effect.
People are using AI to record meetings, and we should think about the person who reads an AI recap of a meeting they attended. If something looks wrong, they can check the recording or their notes. But they may also assume they would remember if the meeting had gone differently. The concern this study raises is that the recap could become part of what they think they remember, so the reason to check becomes a little less obvious.
The study did not test workplace meetings, and it does not show that every summary damages memory. People have always been able to change one another's recollections, so this is nothing new. What AI adds is the possibility of receiving a written account of almost everything, including things we previously would have had to recall ourselves.
If that account later becomes the basis for telling a colleague what they agreed to, somebody else is affected by the mistake too. They may be arguing with a version of the meeting that feels quite real to the person who read it. I would want the original record available before either person has to rely on being more certain than the other, because we all know how faulty our memories are.
Read the memory experiment →
02 / THE RESEARCH
The draft can change how we speak to someone
Preprint posted September 22 • Preregistered writing experiment • Not yet peer reviewed
|
175
workers in Japan and the United States
The main analysis included people who generated AI drafts in both assisted tasks. Their final emails shifted toward the style of those drafts.
|
Shintaro Sakai and colleagues asked people to write workplace emails in their own language, first without AI and then with AI drafts. The drafts varied in how directly they made a request or disagreement and how much politeness, apology, and surrounding context they included.
Japanese participants initially used more of those contextual features, on average, than American participants. Both groups eventually moved toward the style of the AI draft, however. The shift was larger when the draft differed from their group's usual pattern. By the researchers' measure, participants retained more than 85 percent of the supplied draft on average in every group and condition.
These were fictional emails in one session. Nobody sent them to an actual boss, and the researchers did not follow people long enough to establish that their own writing changed permanently.
Still, asking someone for a favor is a good example of why this is more than a question of whether the sentences sound nice. You know whether you have already asked that person for too much. You know whether they appreciate a direct request or will read it as you taking their help for granted. Those details can change what you decide to say, including whether you ask at all.
AI can be useful here, especially if someone is writing in a language they are still learning. I don't think preserving a voice should mean denying people that help. But once there is a finished message on the screen, it is easy to spend your attention fixing the wording and never ask whether this is how you meant to approach the person. The recipient will respond to the message you send, even if some of the choices inside it were never really yours to begin with.
Read the email experiment →
03 / THE RESEARCH
Students practiced deciding what help they needed
Preprint posted September 24 • Randomized programming experiment • Not yet peer reviewed
|
40
Students in the analysis
|
30
Students at follow-up
|
Students using HelpCoach retained more knowledge a week later. The small sample and missing follow-up participants limit how much we can conclude.
|
Hyoungwook Jin and colleagues built HelpCoach to help students be more specific when asking an AI for assistance. It prompted them to identify the part of the programming task they needed help with and the kind of help they wanted, such as an explanation or an example.
Both groups received training in how to ask for help before starting. The difference was that HelpCoach also offered support while students were working. Those students asked more specific questions, and the group retained more of its knowledge gains on a test one week later. The immediate difference in learning gains was inconclusive.
Only 30 of the 40 students in the analysis returned for that later test. The researchers also did not establish that the students would keep asking better questions once the guidance was removed. We have an encouraging result from a small study, with more work needed to understand what lasted and why.
What I personally like about this is that it asks the student to locate the confusion. If a piece of code doesn't work, there is a big difference between wanting somebody to fix it and knowing which part you don't understand. Getting to that second description takes some thought, and the student can still use the AI once they have done it.
For a parent or teacher, that seems like a useful habit to encourage without turning every request for help into an ordeal. Sometimes a child needs the explanation, and making them struggle longer serves no purpose. Other times, asking them to show where they got lost gives both of you a better chance of helping with the actual problem. This study gives us a reason to test whether AI can make that easier to do, instead of leaving it entirely to the adult beside the student.
Read the HelpCoach experiment →
04 / THE RESEARCH
Many teenagers brought a person into the same problem
Peer-reviewed study published September 25 • Survey and interviews • Public abstract available
|
124
of 192 young AI users also involved a person
That is 64.6 percent of the adolescents who reported using AI for a personal or emotional problem. It is not a rate for all teenagers.
|
Eko Sujadi and colleagues surveyed 432 Indonesian students ages 15 to 18. Of those, 192 had used generative AI for a personal or emotional problem in the previous six months. Most of those AI users also involved another person in that same episode.
Some went from AI to a school counselor or somebody outside school. Others used AI and a counselor alongside one another, or spoke to the counselor first. In interviews with ten students, AI could make the first disclosure feel less risky. Involving a counselor depended on things such as confidentiality and feeling heard.
These were students' accounts of what happened, not an experiment assigning them to different kinds of support. The study cannot tell us whether AI caused them to seek human help or whether one sequence led to better outcomes. The accessible abstract gives us the pathways, not proof that a chatbot improved the young person's life.
We have spent a lot of time in this newsletter asking who stops getting the call when AI becomes easier to talk to. This is a useful reminder to ask who still gets it, and what made that possible. A teenager may need a way to begin saying something before they feel ready to tell an adult. I would want to know whether the next conversation becomes easier, rather than assume that using the chatbot has already replaced it.
There is also something here for the adult. If a child doesn't expect to be listened to, telling them to talk to a person may not do much. The counselor, parent, or teacher has a part in making that a reasonable choice. We can't ask a young person to choose human support and give no attention to how the human responds when they finally do.
Read the adolescent help-seeking study →
|
THE RECKONING
This week made me come back to this idea of the point at which we decide what we think happened, what we mean to say, or what we need help with. AI can get involved before we have worked any of that out, and that can be a good thing or a bad thing. Sometimes that is exactly why we asked for it. We are tired, we are behind, or we don't know enough to make a useful start.
The memory study makes me less comfortable with the assumption that we can always put our own judgment back in at the end. If the summary has changed what someone remembers, asking them to review it may not be the safeguard it sounds like. With an email, there may be no factual error to catch. The message can be perfectly clear and still say something about our relationship with the recipient that we never intended.
I don't think the answer is to require everyone to do everything alone before accepting help. A student who cannot figure out where they are stuck may need somebody to help them find that point. HelpCoach is interesting for that reason. It makes room for the student to become clearer about the question while assistance is still available.
And the adolescent study leaves me thinking about the people on the other side of these tools. If AI helps someone find the words to tell a counselor what has been happening, I want to take that seriously. I also want to know whether the counselor has the time and support to listen. Making the first disclosure easier will only get us so far if the person it leads to is unavailable or dismissive.
These are things I would like researchers to keep following after the immediate task is finished. What did the person come to believe? What did they say to someone else? Did they get the help they actually needed? Those questions will take longer to answer than whether the draft was good or the chatbot was pleasant, but they tell us more about what we are making part of everyday life.
|
Also this week
A few more from the seven daily sweeps, with the original papers linked if you want to read further.
A higher score did not make confidence more reliable
In a preprint involving 366 adults, AI-assisted participants answered more reasoning questions correctly, but their confidence did a worse job of distinguishing right answers from wrong ones across the combined task set, while the difference within individual task types remained uncertain.
Read the reasoning preprint →
AI tutoring matched human tutoring on an immediate test
A randomized preprint with 2,383 adults found equivalent immediate GRE learning gains from pooled AI tutoring and expert human tutoring within its chosen statistical bounds, without testing delayed retention or comparing tutoring with simply doing more practice problems.
Read the StudentBench preprint →
A voice made the tutor more engaging without improving learning
In a preregistered experiment with 86 MBA students, a structured AI tutor improved assessed reasoning, while using voice rather than text increased conversation without a measured learning advantage, according to the new preprint.
Read the tutoring preprint →
The task was delegated but the worry remained
A peer-reviewed two-wave survey of 642 hybrid and remote workers linked greater reliance on AI delegation with more monitoring, more after-hours rumination, and poorer recovery, although the self-reported associations cannot establish that delegation caused those problems.
Read the workplace study →
Most generated audit reports went through unchanged
In a preprint on 71 novice auditors, AI assistance helped find more failures in a shopping agent, while 150 of 160 reports produced with a report-writing agent were submitted without edits, a sign of acceptance that does not by itself prove nobody checked them.
Read the auditing preprint →
An AI teammate had trouble with the unwritten rules
An exploratory preprint following an AI agent inside one technology company found that helping with shared work also raised problems around unwritten practices, trust, and who should be allowed to act on whose behalf, without establishing long-term effects on teams.
Read the workplace-agent preprint →
The chatbot followed its persona more reliably than its teaching purpose
In workshops with 27 middle school teachers, chatbot responses matched the intended persona more often than the instructional purpose, according to a preprint that tested teacher configurations rather than student learning.
Read the teacher-configuration preprint →
A thinking partner briefly shifted what people said mattered
A preregistered preprint with 200 adults found that discussing dilemmas with chatbots shifted reported priorities toward personal-focus values even without a recommendation, but the modest shift had receded by the next task and did not establish a lasting change in values or behavior.
Read the values preprint →
|
PRACTICE
Try writing the message before seeing a draft
This week, open The Unpolished Message at The Quiet Cost Practice app. Choose somebody you actually want to thank or something you have been meaning to tell a person. Write what happened between you and what you want them to know before asking AI for wording.
You can still edit it. You can still ask for help with grammar or translation. Keep your first attempt beside the revision and check whether the new version adds a feeling you don't have, a promise you didn't mean, or an apology you would not have made.
If you'd rather start with a game, try one of the daily challenges. You can remember an order, spot a change, or work out a route. Have a go and come back for a new challenge the next day. These are opportunities to use the skills we have been discussing, not evidence that a better game score prevents skill loss elsewhere in life.
Try The Unpolished Message or play today's games →
|
|
YOUR TURN
Think of a recent message, summary, or explanation that AI helped you put into words. Before you read its version, what had you already decided for yourself? What did you accept once you saw it written down?
Take five minutes to write about that specific example. If another person received the result, include what you wanted them to understand and whether the finished version actually said it.
|
Pass it on
If someone came to mind while you were reading, forward this to them. Reckonings is free each week. They can subscribe and read the earlier issues at quietcostweekly.com. My longer essays are at At a Cost on Substack.
Subscribe and read earlier issues →
At a Cost →
Michael McNamara
quietcostweekly.com →
Illustrations created with AI. They are editorial images, not depictions of research participants.