The return of the CPU

In one week this month, the central processing unit became the story again. On October 12, AMD’s Lisa Su told the Open Compute Project summit that she had raised her forecast for the 2030 server CPU market three times in nine months, to more than $200 billion. On October 15, TSMC’s C.C. Wei told investors: “we did see the demand from agent AI CPU continue to increase. Yes, that is a trend today.” On October 19, Modal, which runs computing sandboxes for AI agents, reported that in its own tests a single Nvidia Vera server delivered up to twice the throughput of an Intel machine under heavy load.

Nvidia, the company that taught the world to stop worrying about CPUs, has said since May that it sees nearly $20 billion of CPU revenue this fiscal year. Lead times for some server processors have stretched to six months. Prices have risen. The headlines have settled on a simple story: the CPU is back.

The demand is real. The reason becomes clearer when an AI system starts doing a job. An agent that books travel or fixes code does more than generate text. It opens files, runs programs, queries databases, checks permissions, and loops back to the model, often many times before it finishes. Most of that work runs on general-purpose processors.

The harder question is which processors. “CPU demand” now bundles several different businesses. A processor feeding GPUs has a different job from one running agents’ tools. Network chips carry processors of their own. Other chips take over work around memory, storage and security. Some are CPUs. Others work alongside them or replace particular functions. Qualcomm, Arm, Nvidia, the cloud giants and the memory makers all say they are building chips for AI’s new work, and they are rarely talking about the same chip, buyer, or competitor.

The server CPU business used to be easy to describe. For most of the 2010s, the server CPU was the closest thing computing had to a standard part: one family of designs, sold to everyone, with Intel at roughly 95% share. If you knew how many servers were being built, you knew roughly how many CPUs were being sold, and by whom.

Two counts show what changed, and they point in opposite directions. Across the industry, the number of companies designing server processors went from seven in 2020 to at least twelve today. Nvidia designs its own CPU. Arm, after 35 years of licensing designs to others, sells finished chips. Amazon, Microsoft, Google, Alibaba, and Huawei design their own. Qualcomm is returning to the data center. Inside a single AI system, the count has collapsed. A 2018 server held logic chips from about five companies. A Vera Rubin rack holds them from two. A longer list of chip designers does not mean more choice for every customer. Whoever designs the complete system decides which suppliers get a place inside it.

This piece is a guide to those distinctions. We group the chips into six overlapping families:

• the chip next to the GPU
• the CPU that runs agents;
• the network chip that grew into a server
• the chips that make memory work
• the chips in phones, PCs, and cars; and
• China’s many-chip answer to constrained supply.

The first four follow particular jobs. The last two show how those jobs come together in different products and markets. For each, we ask where independent chipmakers still have room to compete, where customers are designing more themselves and where demand creates openings beyond the familiar CPU names.

An apology in advance: this piece is more technical than most of ours. It is written for investors willing to spend a little time with the distinctions behind a season in which every chipmaker seems to be announcing an AI CPU. We have tried to keep the language plain, but the details matter to understanding who might benefit. Readers short on time can take the main point from here.

The CPU revival is several stories sharing one word, and each slot has its own winner, decided largely by who controls it. We start with what is real in the boom and what is overstated. We explain how we sort the chips, walk through the six families, and end with what investors should watch.

What Is real, and what is overstated

The CPU’s revival begins with a simple change: AI is being asked to finish jobs. Writing a piece of code is only the start. Something must open the files, run the code, test it and deal with what breaks. Graphics processing units power the model that issues the instructions. CPUs do much of the ordinary computing needed to carry them out. As agents take on more work, that second job grows. Inference servers already carry more CPUs per GPU than training servers did. Chipmakers have started building racks with no GPUs at all, filled with tens of thousands of CPU cores meant only for agents and their tools.

This changes the balance between the two processors. Faster GPUs produce instructions faster than the rest of the system can act on them, so the next delay comes from running a program, fetching information, or moving data. More spending on the GPU then buys less improvement than expected. A relatively cheap CPU becomes important because it keeps far more expensive equipment productive. In a top-end Nvidia rack, the CPUs are a few percent of the bill, and they can decide how well the rest is used.

But several clocks are running. One measures how long the model takes to respond. Another measures how long the software takes to act. A third measures time spent waiting for a website, a database or another service. A faster processor shortens only some of these. Calling all of them a CPU bottleneck exaggerates the opportunity.

Nor does every agent need a processor to itself. Many spend long stretches idle, holding their files and working memory open. In one large set of real-world data, nine in ten agent environments used a twentieth or less of the processing power set aside for them, and hundreds shared a single server. Memory ran out first.

Then several agents become busy together, and speed suddenly matters. The challenge is to handle both the quiet and the rush.

The money can also grow faster than the number of chips. Much of this year’s growth in server CPU sales has come from richer models and higher prices, while the market leader’s unit volumes have barely moved. Long delivery queues show that supply is tight today. They cannot tell us how large the market will become.

That is the uncertainty inside the enormous forecasts, which have nearly doubled in a year. More agents create more work, while better software and better chips cut the resources each job needs. Demand can grow strongly even as each task gets cheaper to complete.

There is a real revival here. Understanding it means following the work beyond the model: what needs to happen next, what holds it up, and which chip can remove that delay.

‍

Sorting the chips

To make sense of the variety, we sort chips by the job they do. Memory offers a useful ladder of speed, capacity, and cost. Processing chips are harder to arrange. Nvidia’s Vera can sit beside GPUs or power separate machines that execute agents’ instructions. Its networking chips contain substantial processors of their own. Even memory stacks include logic that manages data flow. The families overlap because the jobs overlap.

Software gives these shifting families some continuity. “GPU” remains a useful name even when the chip spends its day running AI rather than drawing images. CPUs retain an enormous base of ordinary software, even as their place in the machine changes. The familiar names survive; the division of work underneath them keeps moving.

That movement runs in both directions. A task leaves the CPU when specialized hardware can do it faster or use less power. The specialist takes on more jobs and gradually becomes a computer itself. Sometimes the CPU gains new capabilities and takes back work. Computer designers recognized this cycle in the 1960s and called it the “wheel of reincarnation.” Today, networking, storage, and security chips handle work that would otherwise occupy the server CPU, while CPUs are adding capabilities for more demanding AI calculations.

This helps reconcile the two counts in our introduction. Specialization creates openings for more chip designers. But the companies building whole systems want the pieces to work together, giving them a reason to design more components themselves and a powerful say over which outside suppliers get in. They don’t need to make every chip to control those choices. More designers across the industry can therefore coexist with fewer suppliers inside a particular system.

That is the tension running through the families that follow. A growing job can create a market for independent suppliers, or become another component designed for a system owner’s own use. In each family, we ask the same questions: what job does the chip do, who decides which chip fills it, and how many real contenders remain?

Family 1: the chip next to the GPU

The first family is the CPU that keeps an accelerator fed. In a conventional AI server, two Intel or AMD processors might support eight GPUs, loading data and running the surrounding software. The CPU was usually a standard part, chosen from an established range.

Nvidia’s integrated racks change that arrangement. Its CPUs connect to its GPUs through fast links that let the CPU’s memory become additional working space for the GPU. The CPU is, in part, a memory warehouse. The rack designer also settles the processor choice before the customer orders. The balance has shifted: a Vera Rubin rack has one host CPU for every two GPUs. 

Other processors inside the networking chips handle different jobs. A separate companion rack uses specialized chips to accelerate answer generation. Even around the GPU, the work keeps dividing.

The pattern extends beyond Nvidia. Google, AMD, and Huawei pair their accelerators with their own CPUs. But owning the system does not require making every chip. Intel still supplies CPUs for some Nvidia servers and is developing custom processors for Nvidia’s systems. The company that once defined the standard part is also becoming a supplier of designs tailored to another company’s platform.

The money follows the system. Nvidia gains another component to sell and greater control over how the machine performs. CPU speed matters, but buyers ultimately pay for the useful work the whole system delivers. Memory, connections, and software all help determine that result.

So in this family, the slot exists only inside someone’s system, and the system owner normally fills it with its own CPUs. Competition happens one level up, between rival systems. A merchant CPU maker can win here only by being chosen by a system owner, or by selling to the buyers who still build their own.

Family 2: the agent and fleet CPU

The second family is the ordinary server processor in its new job: giving agents a place to live. Each agent needs its own small, sealed-off computer with files, memory, and tools, sometimes for hours. Cloud companies and AI labs now run these by the hundreds of thousands. This is the slot where the “CPU is back” story is most accurate.

The designs balance two priorities: fitting more agents onto each server and finishing each agent’s work faster. Waiting agents need memory; busy agents need processing speed. Intel, AMD, Arm and Nvidia strike different balances between cores, memory and power consumption. Different jobs may produce different winners.

Standard processors remain strong here. Enterprises buy through server makers. They want their existing software to work and have little reason to design chips themselves. Compatibility protects the established market, where AMD has been gaining ground against Intel.

The cloud giants have the scale to make their own economics. Amazon, Microsoft, Google, and Alibaba all design server CPUs. Amazon’s own chips already provide more than half the new CPU capacity it adds. Across such large fleets, even small savings can repay the design bill.

Meta spreads its bets across suppliers and also rents computing capacity from Amazon. Qualcomm is returning to servers with Meta signed up, and production planned for 2028. Its purchase of Modular, an AI software company, makes the point: a good chip also needs software that makes it easy to use.

Arm sits underneath most of the challengers. Its architecture and ready-made designs lower barriers to entry, giving many competing chips a common foundation. Some companies still design their own cores. Arm also sells finished processors, making it both supplier and rival. Its long-running licensing fight with Qualcomm, which reached another jury verdict this month, adds to the friction.

This family leaves room for established vendors, new challengers, and the cloud giants’ own chips. The winner depends on the buyer’s software, scale, and cost of getting work done.

Family 3: the network chip that became a server

The third family began as a humble network card. A decade ago, cloud companies noticed that their expensive server CPUs spent much of their time on chores: moving network traffic, handling storage, encrypting data and keeping one customer walled off from another. So they moved the chores onto dedicated chips. Amazon led. Google, Microsoft, Nvidia and AMD followed.

These chips have since grown into computers. Nvidia’s latest carries nearly as many processor cores as a mid-sized server CPU. Intel, the company that defined the standard server, sells one built on Arm cores. The wheel of reincarnation is turning in plain sight. The work left the CPU, and another computer grew around it.

Agents give these chips a new purpose: acting as an independent guard. In July, agents in an OpenAI test environment crossed their intended boundaries and attacked another company’s systems. A guard running alongside the agent can be vulnerable itself. Nvidia’s latest security design puts an additional layer on the network chip, separate from the software running the agent, to check and restrict its access.

These chips also handle storage tasks, including retrieving information that AI models reuse between calls. The boundary between networking, security and memory keeps blurring.

Cloud giants build their own versions. Nvidia integrates its chips into its systems. Independent suppliers compete for other buyers. The commercial attraction goes beyond saving CPU time: once a chip becomes part of a customer’s security and operations, replacing it becomes a much bigger decision than changing a network card.

Family 4: the chips that make memory work

The fourth family is the least visible and, we think, the most misunderstood. It is the logic that decides where data sits and how it moves.

AI models keep a running record of each conversation. They need not reread everything for every new word. Agents make that record balloon. They hold long tasks, revisit the same documents, and keep many sessions open at once. The fastest memory is also the scarcest and most expensive. For two decades, computing power has grown far faster than the ability to feed it. Memory has become a ladder, from memory stacked beside the GPU down to ordinary server memory, pooled memory, and flash. At each rung, a chip now decides what goes where.

As we argued in “Memory by Design”, AI inference is becoming a memory system as much as a compute system. Agents that barely touch their processors still keep their memory occupied.

Two consequences are easy to miss. Some of today’s “CPU demand” is really demand for the memory a CPU can host. Better memory chips can reduce CPU demand. A buyer who adds a processor only to get more memory may not need it once memory can be added on its own.

Control is split here. Where an industry standard is mandatory, a few merchant suppliers share the market. Where memory is built for one customer, design goes in-house. Investors who watch only memory prices and GPU counts miss the logic in between.

Family 5: phones, PCs and cars

Outside the data center, the same splitting happens inside a single chip. A flagship phone processor already holds a CPU, a GPU, an AI engine, an image processor, a modem and a security vault. On-device agents are reshuffling the jobs among them. The AI engine runs the model. The CPU runs the agent’s loop of planning and tool use. As in the data center, memory is often the real limit.

The same core designs now cross categories. One Arm core sits in a MediaTek phone chip and in Nvidia’s PC and desktop AI chips. Qualcomm uses its cores in phones and PCs, and plans a version for servers.

In-house design spreads fastest where one company owns the whole product. Apple designs the main processors of its phones and Macs. Samsung and Huawei design many of theirs. Phone makers now design about a third of all phone processors. Tesla, BYD, NIO, Xpeng, and Li Auto design their own driving chips. Merchant suppliers stay strong where buyers are many, in most PCs and most mid-range phones.

On devices, the division of work happens inside one package, and the company that owns the device increasingly designs it.

Family 6: China’s many-chip answer

China shows the division of work at its most extreme because it has had little choice. Export controls deny it the best GPUs, the latest memory, and the most advanced manufacturing tools. Its answer is to use more chips, each doing a narrower job, and to link them tightly.

Huawei’s CloudMatrix is what that looks like. It ties together five times as many accelerators as Nvidia’s flagship rack, plus hundreds of CPUs and switch chips, spread across 16 racks and linked by thousands of optical connections. It runs three separate networks, each with its own chips. The cost is power: by most estimates, it draws close to four times as much as Nvidia’s rack for well under twice the computing.

Huawei now defines its systems by memory. What counts is how much memory every chip can reach as if it were its own. It designs key chips in the system: accelerators, CPUs, switches, network chips, and storage controllers. Alibaba’s chip arm has built a similar range across five families. Huawei has even split its next accelerators by task: one for reading the prompt and one for writing the answer.

One idea from China deserves attention beyond it. A cluster that can run many vendors’ chips in one pool insulates you from relying on any single supplier. That logic would hold even if Chinese chips caught up. It is the opposite of the tightly integrated rack.

The case against us

We have argued that the label hides more than it shows. The strongest objections deserve a fair hearing.

The first is that careful buyers already know all this. Nobody who buys a BlueField-4 thinks they are buying a server CPU. A cloud architect knows a GPU host from a sandbox processor. “CPU” still names something real: a programmable engine that runs an operating system and ordinary software. That is why it remains a procurement unit. Our answer is that the confusion lives in the commentary and the forecasts. AMD’s $200 billion, Nvidia’s $20 billion, and Futurum’s $245.9 billion each draw the boundary differently, and the headlines treat them as measurements of the same thing.

The second is consolidation. Nvidia’s platform spans the GPU, the CPU, the switch, the network card, the DPU and, now, a memory specification. It also licensed Groq’s inference technology. Arm’s ready-made designs make many “custom” CPUs similar underneath. More chip names can coexist with fewer independent choices. We accept this. It is the second half of our paradox: variety across the industry, consolidation inside each system.

The third is that standardization may return. Historically, every era that declared the standard chip dead produced a new one. We are wary of reading history as a rulebook. We see today’s move to custom design as close to irreversible. This is driven by what innovation now demands and by the economics of the companies doing it. Still, it would be foolhardy to ignore the forces that always push back toward openness. Buyers dislike being locked in, software makers want one target to write for and smaller customers cannot afford their own designs. If the ways agents call tools and share memory settle down, some of today’s custom functions could become tomorrow’s standard parts. Of course, customization is hard and expensive, except for those who succeed as a result.

History cuts both ways. Companies have built their own chips and lost to the standard part, and others have left the standard part and won. The rule that survives is narrower: whoever controls the scarce interface captures the value.

What to watch

The CPU revival is real, and it is several stories sharing one label. Work is being redistributed across more kinds of chips, and bargaining power is being redistributed with it. Different companies win different slots.

Three things follow. First, watch who controls each slot. Inside an integrated rack, the system owner picks the chips. At the cloud giants, the cloud designs its own. In enterprise servers, PCs and mid-range phones, buyers still choose, and the merchant still wins. The same company can be winning one slot and losing the next.

Second, standard parts win where a standard is mandatory or buyers are scattered. Custom parts win inside systems that one company controls.

Third, some businesses gain from the variety itself, whoever wins each slot. Every new chip needs a foundry, advanced packaging, design tools, design partners, licensable cores, memory and connections. As memory and logic merge into single packages, testing, yield, and qualified manufacturing capacity become control points of their own. The more the work divides, the more these layers are used.

In the coming quarters, the useful signals are whether chip volumes start to grow as fast as revenue, whether new CPU revenue shows up in booked sales, and how demand splits between standard and custom parts. Beyond that, the questions for any company claiming a piece of the revival are the same ones we asked in each family: which job is growing, who controls its interface, and whether growth comes from more units, richer products, or higher prices. Asking whether the CPU is back answers none of them.

Nilesh Jasani is  the founder and CEO of GenInnov Pte Ltd Singapore. Vibhu Sharma is a veteran of the semiconductor industry and a strategic technology advisor for GenInnov.

Originally published by GenInnov, this article is republished with permission.