Artificial intelligence leaders like Anthropic and OpenAI are racing toward blockbuster IPOs that could value them at over US$1 trillion, driven by the rapid progress of their systems.
Against this backdrop, some are arguing that the world should at least have the option to slow down or temporarily pause frontier AI development. The idea is simple: give society and safety research time to catch up with the pace of the technology.
One of the biggest concerns is what researchers call “recursive self-improvement,” the idea that AI systems might eventually improve themselves without human help. As models become more capable and AI agents begin operating more independently, there is a belief that they could eventually outpace human oversight, becoming able to outthink, outmaneuver and outcompete human decision-making across various domains.
But this focus is misplaced. The most serious risk from artificial intelligence is not an imminent loss of control to super-intelligent systems, but a slower and more immediate shift: the concentration of informational power in a small number of companies and states, and the fragmentation of shared reality through highly personalized AI systems.
AI systems today are already highly effective at identifying patterns in large volumes of human-generated data. In many cases, they can surface correlations or insights that were previously overlooked, even if those insights were, in principle, discoverable by humans. This capability can accelerate research and improve decision-making, particularly in fields where complexity or scale makes human analysis difficult.
However, this is different from the kind of creativity that drives major conceptual breakthroughs. Human innovation is not only a product of pattern recognition, but of lived experience, curiosity and the ability to connect ideas across domains that are not obviously related. People explore the world through work, hobbies, conversation and trial and error, building intuition that cannot be fully reduced to data alone.
Some of the most important innovations emerge from this kind of synthesis. Steve Jobs, for example, famously drew connections between phones, cameras, music players and computers to envision the smartphone as a unified device. That kind of insight depends not just on information, but on perspective shaped by connecting the dots through lived experience.
Another source of innovation comes from feedback loops. Products and technologies improve as users interact with them, revealing flaws, limits and unexpected uses. While each change is often small, these incremental improvements build up over time into major advances.
As AI systems are deployed more widely across industries, these feedback loops are likely to speed up, producing more data on performance and limitations and enabling faster refinement.
This dynamic has important geopolitical implications. As China scholar Philip Fei-Ling Wang and other researchers have noted, countries like China that can deploy AI systems at scale may gain a significant strategic advantage. Large economies with strong state capacity may be especially well positioned to integrate these systems rapidly across sectors, turning deployment speed into a source of power.
However, the risks go beyond capability alone. They also involve perception. As AI becomes embedded in everyday decision-making, there is a growing tendency to treat its outputs as objective or authoritative, even though they remain probabilistic and context-dependent. Over time, this could weaken the public’s ability to critically evaluate reasoning and the assumptions behind conclusions.
More concerning is the possibility that either the systems themselves or the data they rely on could be deliberately influenced or compromised by foreign actors. Even subtle manipulation could introduce distortions that favor certain narratives, policies or interpretations over others.
Unlike traditional media, which sends the same message to everyone, AI systems can produce different answers for different people even when they ask the same question. The result is not always obvious manipulation, but something more subtle: each person can end up in a personalized “information world,” where they see a slightly different version of reality shaped by hidden algorithmic choices.
The answer is not to slow artificial intelligence, but to prevent its control from becoming too concentrated and opaque. That requires fostering real competition among AI systems so that no single group of companies dominates how information is produced and interpreted.
It also requires greater transparency, along with training in schools, universities and workplaces on how these systems are built and deployed, especially as they become embedded in search, work, education and decision-making. Without that understanding, it will become increasingly difficult to distinguish between genuine knowledge and statistically-patterned output.
Finally, according to Bruce Hogan of the Oxford Internet Institute, in a recent interview, we will need better tools to detect and verify AI-generated content as these systems grow more capable of producing convincing, deceptive material.
The debate over AI is often presented as a choice between runaway superintelligence and strict limits. But the more immediate issue is structural: whether shared reality stays stable, transparent and accountable as information is increasingly produced by a small number of powerful systems.
Derek Levine’s commentaries on technology, education and US-China relations have appeared in The Hill, National Review, The Diplomat, RealClear Media and Asia Times. He is the author of “China’s Path to Dominance: Preparing for Confrontation with the United States”, which can be purchased on Amazon here.
