The Human Cost of AI's Math Breakthroughs
The biggest AI story this week may have been one that most of us cannot evaluate for ourselves. OpenAI released hundreds of mathematical manuscripts produced with an internal AI model, and mathematicians have been trying to work out what the results mean for their field. The response has included excitement, anger, and grief.
We don't have to be able to check the proofs to understand why this would be personal. These are people who have spent years learning how to approach a problem, often working side by side with other people doing the same thing. I talk often in this newsletter about the output, but the quiet cost is the human processes we stand to lose as AI keeps performing better and better.
There are smaller versions of that in this week’s studies as well. What do designers do while an image generator works? Does practicing with AI make students more willing to speak in class? And when somebody feels better after talking to a chatbot, what happens to their conversations with real people?
As always, I want to look at what the help makes possible, as well as what we may stop doing because we have that help now. This week gives us reasons to pay attention to both.
01 / THE RESEARCH
What mathematicians are worried about
Company research release • Verification is ongoing • Mathematicians' responses
|
719
manuscripts
|
372
groups of related results
|
OpenAI's catalogue as checked on October 12. The count includes related arguments and alternative proofs, not 719 separate, independently verified breakthroughs.
|
On October 6, OpenAI published a large collection of mathematical work generated with an unreleased model. The subjects range from how closely fractions can approximate pi to mathematical models of magnetism. The company says about 42 percent of the main results have been formalized in Lean, software used to check mathematical proofs. It also warns that some of the work without those formal checks could contain errors. There is real work here for mathematicians to assess, and the number of documents alone cannot tell us how much has been established.
The reaction is understandable, and frankly something I expect to see in more fields as AI continues to improve. In an October 8 discussion reported by Le Monde, Fields Medalist Hugo Duminil-Copin said, “They have annihilated my field of research.” The report also describes his plans to study the results with other mathematicians and help people understand them. His response includes wanting to keep doing useful work, even as that work continues to change with each passing day.
Other objections concern who gets to decide how this happens. The Association for Human Mathematics criticized the use of an internal model unavailable to the community and urged mathematicians to stop working with OpenAI. That is the association's position, not a statement on behalf of every mathematician. Its statement appeared as a guest post on Terence Tao's blog.
Tao's own recent lecture offers a way of understanding what is at stake. Working on an unsolved problem can teach mathematicians things beyond the eventual answer, including things they discover when an approach fails. He argues for using AI in ways that support that understanding and the people developing it. He also points out that many difficult problems remain unsolved. Mathematics has not run out of things to do.
I don't think we should dismiss this reaction as people being upset that a machine can do their job. Imagine spending years getting good enough to contribute to a difficult project, then being told that the project has been completed and your role is to read through the result. A result produced by a machine, not even a colleague. There may be valuable work left to do. You may even find that the result opens up something more interesting. But that doesn't make it the same work you were preparing to do, or give you back the experience of doing it with your colleagues.
We should want mathematical advances. We should also ask how the next group of mathematicians will learn enough to understand them, question them, and take the field somewhere new. Having more answers does not settle what kind of education or working life will make that possible.
OpenAI's release and verification status →
Duminil-Copin's response in Le Monde →
The Association for Human Mathematics statement →
Terence Tao's Math 2.0 lecture and slides →
02 / THE RESEARCH
What designers did while the image generator worked
Peer-reviewed experiment • 60 graduate design students • One university
|
3.77
Watching the generator
|
4.94
Taking a break
|
Average creativity scores on a seven-point scale. Sketching scored 4.98 and reviewing the brief scored 5.07.
|
Researchers at Ningbo University asked design students to work through a task that included a two-minute wait for an AI-generated image. They ranked the students by prior design ability, then randomly assigned students within those groups to spend the wait sketching, rereading the design brief, watching the generation progress, or resting.
Five experts who did not know the assignments rated the resulting designs. All three alternatives to watching the generator produced significantly higher creativity scores. Even taking a break did better than staying focused on the system's progress.
This was a small experiment with a fixed wait that the researchers imposed. It does not tell us that every pause makes us more creative, or establish why the groups differed necessarily. It also says nothing about lasting changes in the students' abilities.
What interests me is how easy it would be to consider watching the screen part of doing the work. You are waiting for something relevant. You haven't wandered away from the assignment. But you are also waiting for the machine to give you the next thing to think about.
Sketching gives you a chance to try an idea while the tool is working. Rereading the brief can remind you what you were trying to make before you see what it offers. And a break leaves you some time without another image asking for your attention. We can use the generator and still think about how much of our own working time we want to organize around waiting for it.
Read the design experiment →
03 / THE RESEARCH
AI practice helped students join the conversation
Preregistered field experiment • Preprint, not yet peer reviewed • 759 MBA students
|
31%
more voluntary contributions
|
The estimated increase after two AI practice sessions, measured relative to the study's average contribution count in a class session.
|
Dan Wang and colleagues studied a voice-based AI discussion partner designed to help MBA students prepare for class. Each student was randomly assigned two of ten class sessions for which they could use it. Because those assignments happened at different times, the researchers could compare students who had already had the practice with students who had not yet had it.
After two uses, students contributed more voluntarily in later class discussions. The estimated increase was roughly one extra contribution across four or five classes. That is the same result expressed as the 31 percent figure above, and it gives us a more concrete sense of its size.
Everyone eventually had access. There was no comparison with students practicing with a human partner, and the experiment did not establish better end-of-course learning. This was also a tool built for the course, so the result should not be treated as a finding about every chatbot.
I like that the researchers looked at what happened when students returned to a room with other people. If preparation makes somebody more willing to offer an idea, the benefit can extend beyond that student's confidence. Other students get something to respond to. The instructor gets a better chance to find out what the student understands or has misunderstood.
There are still questions about the quality of those contributions and what the students learned over time. But this is a useful example of how AI might help someone take part in an experience they would otherwise have avoided. The conversation with the machine was followed by more participation with people, which is something I would like to see more tools designed to encourage.
Read the classroom-participation preprint →
04 / THE RESEARCH
Feeling better and spending less time with people
Peer-reviewed diary study • 97 social-chatbot users • Publisher's abstract available
|
Emotional engagement with the chatbot was associated with better daily well-being and shorter human interaction. The study does not establish what caused what.
|
Yun Chen and colleagues asked social-chatbot users to report on their experiences each day for two weeks. On days when people felt lonelier than usual, they reported spending longer with the chatbot and feeling more emotionally engaged with it.
The researchers separated time spent from emotional engagement. Stronger emotional engagement was associated with feeling better that day, but also with less time in interpersonal interaction. Longer chatbot use was associated with less enjoyment of human interaction.
These are associations in people's daily reports, not evidence that the chatbot caused them to withdraw. The study does not establish whether the shorter contact was a problem, a choice, or part of something that was already happening. Someone having a difficult day may both use AI more and find ordinary conversation less enjoyable.
Still, I think it matters that the researchers looked beyond whether the user felt better. That relief can be valuable. It can also leave us with questions about the rest of the person's day that a satisfaction rating will never answer.
Last week, we looked at interviews about why students found it easier to talk to AI. This study follows people day by day, but we still don't hear from the friend or partner on the other side of the shorter conversation. Did they notice anything? Had the user been withdrawing before they began using the chatbot? Or did the relief help them manage a difficult period while keeping the relationships that mattered to them?
I would want longer research to include those people too. Otherwise, we could learn a great deal about how somebody feels after a chat and very little about what is changing between them and the people who care about them.
Read the chatbot diary-study abstract →
|
THE RECKONING
The mathematics story was one I thought about all week, and continues to linger as I put this issue together. As I have said for years, we often fear what is going to happen if AI messes up, while I spend my time worrying about what happens to us as AI continues to improve. We can spend a lot of time discussing AI's mistakes, and there are still mistakes to check in this release. But the correct answer could be more frightening because a correct answer can change the reason people have to keep working together on a problem.
We have organized a great deal of learning around work that needed to be done. A newer colleague learns by taking on part of a project. A student develops an idea because they have to explain it in class. People get to know one another while they are trying to figure something out. Those relationships lead to the next journey of discovery together. If AI takes care of more of that work, the opportunities to learn and make those connections don't automatically reappear somewhere else. They might be lost forever.
I don't think the answer is to preserve every slow process. Some work is just exhausting, and finishing it sooner can give people time for something they actually want to do. There is also no reason to assume that a person who struggled for years had to struggle for exactly that long. Better help can make a field available to people who would otherwise have been excluded from it.
But we do have to ask what happens next. If a junior employee no longer needs to prepare the analysis, how will they learn enough to question the next one? If a student has a good rehearsal with AI, do they then get a chance to try their thinking with classmates? Who is making room for those experiences once they are no longer necessary to get the immediate task finished?
That is what I want us to keep watching and paying attention to. We can welcome a better result and still care about whether people remain able to understand it, contribute to the next one, and have a reason to include someone else along the way.
|
Also this week
A few more findings from the week's recaps. Each link goes to the research rather than a summary of it.
AI can take on decisions we never asked it to make
In a preprint involving 24 users, participants reviewed 992 reasoning steps and considered 48.6 percent to be work the AI had initiated without being asked, a small study that raises a useful question about how much of a task we actually meant to hand over.
Read the cognitive-delegation preprint →
Solving first and checking AI helped future math teachers
A peer-reviewed, 12-week experiment with 86 mathematics-education students found better algebra and error-checking scores after a course built around attempting problems, comparing AI solutions, and explaining mistakes, though the study tested that whole teaching approach rather than isolating the AI's contribution.
Read the algebra teaching study →
Detailed AI feedback can still miss the misunderstanding
A preprint comparing teachers and three AI systems across 3,041 computer-science responses found that the systems' polished feedback identified student misconceptions less often, suggesting that a thorough-looking assessment may still leave a teacher needing to find out what a student doesn't understand.
Read the assessment preprint →
Letting AI choose can feel different depending on the purchase
Across four peer-reviewed experiments with 1,427 participants, delegating a choice to AI improved reported well-being in scenarios involving experiences but reduced it for material purchases, a result about immediate responses to imagined choices rather than long-term happiness or decision-making ability.
Read the consumer-choice study →
Asking AI to anticipate criticism can weaken the writing
A preprint found that asking models to prepare scientific prose for review encouraged defensive wording and sometimes unsupported qualifications, a reminder to check whether an AI edit has made a claim more accurate or merely made its author sound less willing to stand behind it.
Read the defensive-writing preprint →
A person still has to explain the automated decision
A qualitative conference paper on a ride-pooling service describes how drivers handled the friction between automated decisions and passengers' expectations, pointing to human work that can remain necessary without showing up in the system's measures of how well it is doing.
Read the frontline-worker paper →
|
PRACTICE
See if you can work out the rule in Black Box
This week, try Black Box at The Quiet Cost Practice. You get observations and a hidden rule to figure out. Make a prediction, choose something to test, and see whether the result fits your explanation. If it doesn't, you have something specific to rethink before trying again.
The fun is in having a theory that seems right, then finding the example that makes you change it. There is no timer pushing you toward a quick guess. You can stay with the problem long enough to get interested in why your first explanation didn't work.
Then try taking that habit into something you were already going to do. Before asking AI why a result looks strange or why a plan didn't work, write down your best explanation and one thing you could check. You can still ask for help afterward. You will have a thought of your own to compare with it.
Black Box offers practice in forming and testing explanations. We haven't established that the game prevents AI-related skill loss or improves professional judgment. The point is to give you an enjoyable reason to do some of that thinking yourself.
Try Black Box at The Quiet Cost Practice →
|
|
YOUR TURN
Choose one task you would be happy to have AI finish for you. Apart from the finished result, what have you gotten from doing that task yourself? Have you learned something, found that you enjoyed it, or had a reason to work with someone?
Take five minutes to write about that particular task before asking AI to help. If the answer is that it was mostly a chore, that is useful too. If there was something you want to keep, how could you make room for it once the task becomes easier to hand over?
|
Pass it on
If someone came to mind while you were reading, send this to them. Reckonings is free each week, with research on what AI is changing in us and things we can do to keep using our skills. They can subscribe and read earlier issues at quietcostweekly.com. My longer essays are at At a Cost on Substack.
Subscribe and read earlier issues →
At a Cost →
Michael McNamara
quietcostweekly.com →
Illustrations created with AI. They are editorial images, not depictions of research participants.