New study suggests that human-machine interaction is better than its reputation
Collaboration between humans and AI can be beneficial, provided certain conditions are met.
© MPI for Human Development
At a glance
Earlier analyses of human-machine interaction mostly showed negative synergy effects. A review of 74 studies suggested that human-AI teams perform less well than the better of the two working alone.
These 74 studies have been reanalyzed by researchers from the Center for Adaptive Rationality at the Max Planck Institute for Human Development. Their results paint a much more positive picture.
The reason for the difference: The studies systematically underestimated human learning ability; most did not examine it a all. Yet when participants were able to learn from feedback and explanations, human-AI teams tended to outperform the better partner alone.
Need for a paradigm shift: The researchers call for future work on human-AI interaction to explicitly factor human learning ability into studies and experiments.
Human-AI collaboration is widely seen as the work model of the future: The expectation is that together, humans and AI will be more productive than either working alone. Yet the research to date seemed to tell a different story: On average, human-AI teams did less well than the better of the two alone. A reanalysis led by researchers at the Max Planck Institute for Human Development now suggests this verdict was overly pessimistic.
“The potential of human-AI collaboration is underestimated.” That is the central conclusion of the new study published in the Proceedings of the National Academy of Sciences, and conducted by an international team led by Julian Berger and Stefan Herzog from the Center for Adaptive Rationality at the Max Planck Institute for Human Development. The research team took a closer look at a review study that made waves in 2024. The review analyzed 74 studies on human-AI interaction, spanning a wide range of tasks from classifying images and predicting house prices to writing code.
Its findings were sobering: On average, the synergy effect was negative. In other words, the human-AI team performed worse together than the better of the two working alone. This fueled doubts about whether human-AI collaboration was worth pursuing at all.
Learning effects underestimated
Is the situation really that bleak? That is the question the researchers set out to answer in their new review. They revisited the 74 studies, examining the nature of the human-AI interaction in more detail. They wanted to know whether the picture would look any different if AIs explained their results or if humans were given feedback on what the right answer would have been after each decision, giving them the chance to learn and adapt to the algorithm.
The first surprising result was that most studies did not offer learning opportunities. Only 10 of the 74 gave participants any feedback at all, and none varied it experimentally. “Learning effects in human-machine interaction remain virtually unexplored,” says Julian Berger, who explored this topic as part of his doctoral dissertation at the Max Planck Institute for Human Development.
To tease out these effects, the researchers dug deeper. Using a statistical method known as meta-regression, they compared studies with and without feedback, factoring in whether the AI explained its recommendations. And here came the second surprising result: When participants had access to both feedback and explanations, the human-AI tandem tended to outperform the better partner working alone. On the other hand, explanations alone-without feedback-were associated with a distinctly negative synergy.
What appears to be the key is a combination of explanation and feedback. Feedback alone-without explanations-made no clearly discernible difference. This result suggests that “explanations improve synergy mainly when humans can learn to verify the AI’s reliability through feedback,” the researchers write.
Open questions: causality and generative AI
Because none of the 74 studies varied the learning opportunities provided within an experiment (e.g., feedback vs. no feedback), the studies were compared based on their design features. The researchers stress that experiments directly manipulating learning opportunities are needed for stronger, causal conclusions to be drawn. Another open question is how the findings translate to generative AI and large language models (LLMs). “Verbal interaction with chatbots could make it easier to provide people with personalized feedback and explanations, thereby stimulating learning. But for that to happen, people also need to find out what is actually correct; a chatbot’s answers can be wrong and do not constitute independent feedback,” says Stefan Herzog, Senior Research Scientist in the Center for Adaptive Rationality at the Max Planck Institute for Human Development. “At the same time, new risks arise from cognitive ‘offloading’-that is, delegating mental work to AI. In the medium and long term, this could jeopardize learning success or even lead to loss of skills.”
Factoring human learning into AI implementation
To get a clearer picture, the researchers call not only for further experiments but, above all, for “a paradigm shift in human-AI interaction research that explicitly addresses human learning.” They emphasize that “supporting human learning-particularly by providing outcome feedback and AI explanations-is an underappreciated and understudied but promising lever for increasing human-AI synergy. Our findings paint a more optimistic picture and highlight the need to explicitly and experimentally study when and how fostering human learning boosts human-AI synergy.”
The research already supports a clear recommendation: AI is a tool, and like any tool it must be designed with humans in mind from day one. “When designing AI systems and interactions with them, we need to consider humans as learning beings from the outset,” Herzog stresses. Only then can the full potential of the interaction be realized.
