AI And The End Of Progress? Why Innovation May Be More Fragile Than We Think

For two centuries, technology has transformed how we live and work, from the steam engine to the internet. It is tempting to assume that progress will always continue. Yet history offers a sobering lesson: great leaps forward often grind to a halt when institutions fail to adapt. Economist Carl Benedikt Frey, author of "How Progress Ends," warns that artificial intelligence could either power a new economic boom or bring innovation to a standstill, depending on how we respond today.

Why Progress Can End

Frey argues that the idea of inevitable technological progress is deeply flawed. “If progress was inevitable, the first industrial revolution would have happened a lot earlier,” he explained in our recent conversation. “And if progress was inevitable, most countries around the world would be rich and prosperous today.” Many societies have seen periods of intense innovation followed by stagnation or collapse. Ancient cities such as Ephesus once thrived and then disappeared. The Soviet Union industrialized rapidly but failed to keep up when the computer era began.

The heart of the problem lies in how societies transition between two distinct phases of innovation. Frey describes an early "exploration" phase, during which new ideas emerge in a highly decentralized environment where experimentation thrives. Successful economies later shift into “exploitation,” scaling technologies, cutting costs, and consolidating power. “Exploration thrives on decentralization,” he said. “Exploitation thrives on concentration and consolidation. But then, to get onto the next cycle, you need decentralization again. And that transition is very hard to make.”

Lessons For The AI Age

Artificial intelligence sits squarely at the center of this fragile transition. Early breakthroughs, from transformers to generative AI, came from open experimentation in universities and small labs. Yet market power is concentrating fast. Frey notes that “OpenAI together with Microsoft has something like 70 percent of the market,” and many large tech incumbents invest in the very startups that might otherwise challenge them. A return to consolidation could slow the next wave of breakthroughs and turn AI into a tool for incremental efficiency rather than real transformation.

Frey is also critical of assuming that scaling existing models will deliver human-level intelligence. “A lot of people still thought that you could just take existing models and scale them,” he said. “That might work in a static world, but the world is changing all the time.” He points out how even superhuman systems can fail when faced with unfamiliar situations, citing how amateur players using laptops recently beat top Go programs by presenting novel strategies. For AI to keep advancing, we need fresh approaches and a competitive, decentralized innovation environment.

Regulation And The Risk Of Strangling Progress

One of Frey’s strongest warnings concerns regulation. He acknowledges that some guardrails are necessary, for instance, around training data and harmful applications. Yet he believes that overly complex rules can unintentionally crush smaller innovators and entrench the largest firms. “The more barriers to entry, the more compliance costs we add, the higher the risk of a few firms or even one firm monopolizing the technology,” he said.

He points to Europe’s experience with data privacy laws as an example. “Take a well-intended piece of regulation like the GDPR,” Frey explained. “Good reasons to care about data privacy, yes, but we have to look at the unintended consequences. Larger tech companies were essentially able to offset compliance costs by capturing a larger share of the market, where some smaller firms struggled to compete.” Europe’s proposed AI Act risks repeating this mistake. If AI development becomes as costly and bureaucratic as pharmaceuticals, innovation will slow, and power will consolidate further.

Centralization, Decentralization, And Global Competition

The tension between centralization and decentralization also shapes the geopolitical AI race. Historically, the United States led because it encouraged competition and flexible institutions. Government funding supported exploration while antitrust actions opened markets for new entrants. This allowed companies like Microsoft and a generation of internet innovators to emerge. Japan excelled at refining and scaling existing technologies; yet, its highly centralized corporate system left it poorly positioned for the software revolution.

Frey is concerned that America is losing this advantage. “The autonomy of American universities is being reduced, funding through the National Science Foundation cut, clampdown on access to talent and immigration,” he said. “I am much more worried about America than I was when I wrote the book.” Meanwhile, China is centralising power and prioritising national security over growth, which also risks stagnation. The result is an uneasy race where neither side may maintain the conditions for sustained innovation.

The Real Opportunity: Beyond Automation

Many organizations are using AI primarily for process automation and cost-cutting. Frey believes this will not deliver transformative growth. “If AI means we do email and spreadsheets a bit more efficiently and ease the way we book travel, the transformation is not going to be on par with electricity or the internal combustion engine,” he said. True prosperity comes from creating new industries and doing previously inconceivable things. Leaders should encourage their teams to experiment with AI to create new products and services, not just streamline existing ones.

This requires giving employees at all levels the autonomy to test and implement AI solutions. “The people that do the experimentation understand best themselves what the technology can and cannot be used for,” Frey explained. Companies that reward experimentation and decentralize decision-making are better placed to ride the AI wave.

Inclusion And The Future Innovation Pipeline

Frey also highlights the importance of inclusion. Half of humanity’s inventive potential remains underused when large groups are excluded from innovation. “If you take the fraction of female versus male inventors in the U.S., the gap is closing, but at the rate it is closing, it would take a hundred to a hundred and twenty years for it to close,” he noted. Role models and diverse networks are crucial in attracting more people to science and technology. Broader participation makes innovation more resilient and adaptable.

Leading Through Uncertainty

AI’s future impact is unpredictable, but Frey believes leadership decisions today will shape whether it accelerates growth or traps us in stagnation. He warns against overconfidence that technology will simply work things out. Progress has ended before when societies failed to adapt their institutions and mindset.

He remains cautiously optimistic that we can foster open competition, avoid heavy-handed regulation, and empower people to experiment. “If you want to thrive as a business in the AI revolution, you need to give people at low levels of the organization more decision-making autonomy to actually implement the improvements they are finding for themselves,” he said.

The message is clear: AI could be the most powerful growth engine in generations, yet that outcome is far from assured. History shows that technological waves can stall when power centralizes, innovation narrows, and regulation smothers competition. Leaders who embrace experimentation, support diversity, and guard against monopolization have the best chance to ensure that the AI era fuels a new cycle of progress rather than bringing it to an early halt.