“AI Has Become A Major Force Shaping Our World”

Iyad Rahwan shares his perspective on the findings of the new AI100 report.

Artificial intelligence is reshaping research, economies, and everyday life at a remarkable speed. In this interview accompanying the new AI100 report, Iyad Rahwan reflects on the rise of generative AI, the growing concentration of AI power, and the challenges facing societies worldwide. He also explains why Europe must seize this “Sputnik moment” and why he remains optimistic about AI’s potential to accelerate scientific discovery.

The report has been published every five years since 2016. Looking back over the past five years, which development has surprised the panel the most?

Iyad Rahwan: I think that the 2021 report was very positive about the potential for rapid AI development. Yet it failed to anticipate what became known as the ‘ChatGPT moment’ that took the world by storm. AI became a mainstream topic, and billions of people now interact with AI on a frequent basis. This all happened in 2023, marking a major shift in how the technology was perceived across society, the economy, and industry.

You served on the Study Panel. How did the process of developing the report work?

Producing the report required bringing together many very busy AI researchers located across the world in different time zones; it was organizationally very challenging to get people to co-author the report in a coordinated and coherent fashion. Some of the authors met in Singapore while attending a conference, but the most significant progress happened during an intensive writing workshop at Oxford University, hosted by Prof. Michael Wooldridge earlier this year. Following this intensive session, we had many rounds of feedback from other colleagues, including the Standing Committee (which oversees the AI100 study).

If you had to distil the entire report into one key message, what should policymakers, researchers, and the public take from it?

AI is now a major force shaping our world; its arrival and development will mark a major transition in our human culture and institutions.

The report describes the past few years as the “Scaling Era.” What does that term mean?

One of the key insights that led to the recent explosive advancement in AI capability is this: to build better AI, just give it more data and more computational power (i.e. larger neural networks, longer training time, etc.). In other words, simply scaling up produces qualitatively new capabilities: you use the same data, but make the neural network larger, and all of a sudden the AI can do all sorts of things it couldn’t do before; you give the model more data, and all of a sudden it can do advanced mathematics. In the past, AI scientists thought we needed new scientific insights in order to build smarter machines. But during the ‘Scaling Era’, AI scientists realized they just needed to scale up existing ideas.

The report shows that even the experts involved in the previous AI100 report often underestimated the pace of AI development. Why is it so difficult to forecast in this field?

I would say this is in part due to the surprising power of scaling laws. Making models and data bigger did a lot of work on its own. So, progress was much faster than expected, since it did not require new conceptual breakthroughs, as was normally the case.

You helped pioneer the concept of “Machine Behavior.” Why should we study AI systems not only as technologies, but also as actors within social systems?

Until recently, AI was mostly studied by the same computer scientists who built it. In 2019, I brought a group of authors from diverse fields together, including computer science, economics, psychology, political science, sociology, and biology. We argued that researchers from across the behavioral sciences are very much needed to help us understand AI systems. While computer scientists built this impressive technology, they simply do not have the conceptual and scientific tools needed to study the broad impact of AI. Think of the impact on jobs, the impact on learning, human resource practices (like AI-driven job interviews), or romantic partnerships with AI bots. Understanding this impact requires a new behavioral science that integrates the behavior of the machines into the study of human behavioral phenomena.

The report describes the increasing interaction between AI and human culture. Is AI already influencing the way people think, communicate, and make decisions?

Yes, indeed, in many ways. People are using AI now to do a lot of knowledge work, ranging from marketing to science. This is already changing the nature of work immensely and reorganizing what human skills are considered valuable. AI is also being used to generate a lot of cultural artefacts, from novels and poetry to UN reports and legal analysis. In our own work, we also find that AI is influencing the way people speak. All of this will leave a mark on human culture.

More broadly, I think we may be underestimating the degree to which AI is going to change culture. Human culture has always been shaped by our ability to store and transmit knowledge. At first we did this orally by telling stories, then the invention of writing enabled us to preserve and share knowledge in entirely new ways. Later came television and the internet, each representing a major cultural transition.

Now, for the first time, we can interact with systems that have access to much of humanity’s accumulated knowledge, systems that can generate new ideas, new knowledge and new cultural content. What this will do to human culture, to our ability to forget, communicate with future generations, and create new knowledge remains to be seen.

But I believe this is a major transformation in what it means to be human. It will be very interesting to see how it unfolds.

What is the most common misconception about AI today?

Many people see AI only as a tool. It is a tool, an extremely powerful one at that, enabling us to do much more than before, but, crucially, AI is also becoming an agent. It is not just something we delegate tasks to. As billions of people interact with AI, it can perceive the world through those interactions and, in turn, itself act in the world through the people who use it. When people make decisions or take action based on AI’s advice, recommendations, or information, their behavior is shaped by AI. In that sense, AI is not simply a passive technology.

I sometimes imagine AI as a kind of octopus. It has countless tentacles that allow it to sense and influence the world. Those tentacles are us. Through the billions of people who interact with AI every day, it can observe what is happening and, indirectly, shape actions in the world.

This perspective, the idea that AI has a form of collective agency that emerges through its interactions with people, is still somewhat underappreciated.

Public debate about AI is often dominated by risks. Where do you see the greatest potential for AI to enable societal progress that would be difficult to achieve without this technology?

It is very difficult to do justice to this question in a brief answer. I think AI will usher in a new era of scientific discovery. Essentially, the pace of scientific research will be massively accelerated. Researchers are using AI to generate a much higher volume of scientific papers, which is putting a lot of strain on the academic peer review system. But a recent study by researchers at Harvard found that AI can produce very high-quality reviews of scientific papers. In my lab, we are also working on making AI more scientifically creative. So, I think the coming decades will see accelerating advances in science and technology thanks to AI.

The report identifies misinformation, cyberattacks, concentration of power, and changes in human behavior as key challenges. Which of these developments concerns you the most?

I find all of these challenges concerning. But among them, the concentration of power is the most disturbing. If the entire world runs on a handful of very powerful AI systems, controlled by a handful of companies in one or two countries, then just a few people have a massive amount of influence. AI that powers news generation and educational applications can manipulate citizen opinion and children’s learning to promote whatever ideologies the elite want. AI that powers weapons can alter the calculus of war and create an unstable situation. I am also concerned about the societal adaptation to a world where AI can do many of the jobs humans do today. I do not believe we will all necessarily be jobless, since we will be able to envision new jobs that are unimaginable today. But transitioning to a new situation demands adaptation of our markets and institutions, and this is difficult and always comes at a cost.

The report documents a strong concentration of computing capacity, talent, and resources in a small number of companies and countries. Is this primarily a technological, economic, or political problem?

I think it is both an economic and a political issue. On the economic side, developingfrontier AI systems requires enormous investments in computing infrastructure, data centers, and talent, which only a handful of companies and countries can currently afford.

At the same time, it is also a political challenge. Governments play a crucial role in shaping the conditions that enable AI development, from funding research and innovation to ensuring access to reliable energy and creating an investment-friendly environment. Political decisions influence whether countries can build and maintain the infrastructure needed to compete.

Many leading AI companies and infrastructure providers are based outside of Europe. What must Europe do now to remain competitive in the field of AI?

If Europe fails to keep pace in the race toward frontier AI, it may face an existential challenge to its sovereignty and way of life. AI is rapidly transforming defense, scientific research, economic competitiveness, and the functioning of democratic societies.

Europe unquestionably possesses the scientific and technical talent and research institutions to be at the frontier of the global race to AI. The obstacles are institutional. Many of the people building the leading AI systems in the US are Europeans.

Europe should seize this ‘Sputnik moment’ and invest heavily to retain and develop its AI talent and build the commercial ecosystem for a sovereign ‘made in Europe’ AI infrastructure. The window is closing, but it’s not too late.

Public trust in AI appears increasingly ambivalent: adoption is rising, yet concerns are growing. How should policymakers and researchers respond?

I think policymakers need to ensure that the power of AI remains under the control of society, and not a handful of tech leaders. They need to ensure there are no monopolies, and that the benefits of AI technology are accessible to everyone. Policymakers can also develop institutions and technologies for real-time monitoring of AI systems and the regulation and sanctioning of bad behavior. This is obviously the case for cybersecurity but also for detecting misinformation, opinion manipulation, non-competitive collusion among AI systems in markets, and other nefarious uses of AI.

You contributed numerous illustrations to the report. What role can images play in making complex technological and societal developments easier to understand? Is there one illustration that captures the report’s central message especially well?

Yes, indeed. Back in 2021, during the COVID pandemic, I started teaching myself how to draw New Yorker-style cartoons. I then illustrated about 80 cartoons that illustrate various dilemmas that we face with the development of AI. I had no idea that, five years later, some of these cartoons would find their way to the AI100 report. As they say, an image is worth a thousand words. I don’t think my cartoons quite reach that level, but I do think they help illustrate some of the concepts. They can also provoke thinking beyond what is in the text, by triggering people’s imagination. One illustration I particularly like shows a child who wants to play with her friends outside, but her toy robot is asking her to stay because it gets lonely when left alone. I think this cartoon foreshadowed some of the problems we are witnessing today, with the rise of ‘AI companions’ that people are falling in love with or becoming addicted to. I hope we can introduce design principles and regulations that prevent AI toys from undermining healthy social development in children. There is certainly a future in which toy robots can actually amplify and enhance learning and socialization in children. But this is by no means a guaranteed outcome. We have to make it so.

Looking ahead to the next report: What are your biggest hopes for 2031?

My hope for 2031 is that we will have solved the alignment problem. In other words, that we will have found reliable ways to ensure that AI systems remain under human control and cannot autonomously pursue goals that conflict with our interests, whether by manipulating systems, exploiting vulnerabilities, or acting in unintended ways.

I suspect that AI will still be used by bad actors, but I hope that by 2031 we will at least have solved the problem of AI systems acting badly on their own.

My other hope is a more positive one: that AI will fundamentally transform science, allowing us to conduct research faster and more effectively than ever before. This could accelerate progress across a wide range of fields, from energy and climate technologies to medicine and many other areas that directly improve people’s lives.

Interview conducted by Maria Einhorn and Nicole Siller

Iyad Rahwan is the Director of the Center for Humans and Machines at the Max Planck Institute for Human Development. His research focuses on how advanced technologies, particularly artificial intelligence, influence human behavior, cooperation, and society. He served on the AI100 Study Panel, contributing his expertise on the societal implications of AI. He also contributed several cartoons to the AI100 Report to help communicate its key ideas and findings.