Dylan Patel on Why Two AI Labs May Soon Control Most of the World's Compute and Workforce
Dwarkesh Patel
Dwarkesh Patel's yearly conversation with Dylan Patel, founder of SemiAnalysis (the two are not related, despite the running joke), starts from one premise: the world economy is increasingly a function of AI lab economics and the compute market. Across the session they project lab compute and revenue a few years out and ask who captures the value. They then follow the capital requirements into a speculative scenario of rising interest rates, sovereign defaults, and collapsing non-AI equities. Dylan's position throughout is that almost every force in the industry points toward concentration in OpenAI and Anthropic, and that only regulation, credit markets, and politics are slowing that down. Neither speaker could say what would reverse it.
Where lab compute and revenue stand today
Dylan began with the macro picture. By the end of last year, he said, most US GDP growth was AI infrastructure. This year, about a third of the compute coming online is ultimately for OpenAI and Anthropic, even when other companies build it and rent it to them. Total capex is a little over a trillion dollars this year, and he expects it to exceed $2 trillion by 2028. The labs are moving from spending tens of billions a year to hundreds of billions, and some of the contracts they have signed with partners imply trillions a year toward the end of the decade.
That requires a change in their economics. Until recently, both labs mostly lost money on venture funding. Dylan said Anthropic began turning a profit in Q2. He said OpenAI is believed to be capable of turning a profit sometime in Q3, helped by Codex and GPT-5.6. Both still raise capital to grow faster, but more of the business is now funded from revenue.
The key change in his account is gross margin. Compute costs roughly $10–15 million per megawatt. When OpenAI served GPT-4 on Nvidia Hopper GPUs, it ran at negative gross margin. Now, serving GPT-5.6, Opus 5, or Fable 5, revenue per megawatt is well above that cost. For Anthropic, Dylan said it has reached as high as $50 million per megawatt. His summary of the new logic: spend $10 on inference capacity, earn $50 in revenue, and spend the profit on training.
How fast the labs are absorbing the world's compute
Dwarkesh asked when more than half of incremental compute would go to the labs. Dylan's numbers:
- OpenAI started this year at about 2 gigawatts and Anthropic at under 2. Both end the year above 5, roughly a 3–4x increase.
- That growth is about 30% of all compute added this year.
- Based on contracts already signed, the two labs take 40–50% of new compute next year.
- By the end of next year, half of incremental compute goes to them. Because compute is growing so fast, incremental compute soon makes up most of the total stock.
The builders will change. Dylan named SpaceX as a big new entrant next year. It is building a lot of compute and, he expects, will likely lease much of it to Anthropic and OpenAI, because they can pay the highest price. The labs are also building their own: OpenAI with its own chips, and Anthropic with TPUs bought from Google and deployed with Fluidstack. He added a counting convention: when Amazon serves Anthropic models through Bedrock, SemiAnalysis counts that as Anthropic compute, since it ends up as Anthropic revenue despite revenue-share arrangements.
Dwarkesh noted that frontier-lab compute seems to triple yearly while world compute roughly doubles. Continuing that trend gives about 6 gigawatts per lab at the end of this year, 18 at the end of 2027, and 54 at the end of 2028. Dylan estimated world incremental additions at about 30 gigawatts this year, 50 next year, and roughly 70–80 in 2028. He called the 2028 figure his "so fucking bullish" upper bound. Adding it up gives over 200 gigawatts globally by the end of 2028.
Watts also understate the concentration. New chips such as GB300s, TPUv7s, and Trainium3s deliver 3–5x more performance per watt than the previous generation, and they are better suited to AI workloads. If the labs take about half of new compute by December 2027, that half is also the most capable hardware. Dylan said that if the trend continues, and he sees nothing stopping it, by late 2028 the two labs control most of the world's usable FLOPs.
$6 billion of fab capex, a trillion dollars of revenue
Dwarkesh questioned why world compute would grow only by tens of gigawatts a year if compute is so valuable. He built a rough calculation from figures Dylan gave in an earlier interview: a gigawatt of Vera Rubin requires about 55,000 N3 wafers, 6,000 N5 wafers, and 170,000 DRAM wafers. He had an LLM run Dylan's wafer fab equipment model. It estimated $3–4 billion of tooling to produce a gigawatt of compute per year, or about $6 billion including cleanrooms and fab shells. If a gigawatt generates about $100 billion a year, each year's output of that fab keeps earning for years. Over five years, the $6 billion yields over a trillion dollars of end revenue. Dylan pointed to the other costs along the way: operating costs, data centers, power, installation, and lab R&D. Dwarkesh halved the figure for all the middlemen and still got a 100x gap between fab capex and end revenue. Dylan said the true gap is larger and the calculation was conservative.
Dwarkesh asked why capitalism wouldn't close the gap by making more ASML mirrors. Dylan agreed it eventually would, but said the supply chain responds like a whip: the signal takes a long time to reach the far end. Arbitrage is already happening. People buy turbines to resell because turbines bottleneck data centers. Dylan said anyone with $400 million who could persuade ASML to sell them an EUV tool should buy one and later sell it for over a billion dollars.
On Carl Zeiss, which makes the mirrors, Dylan said that earlier this year the company did not think it needed mirror capacity for 100 EUV tools a year by the end of the decade. It now accepts that target, but Dylan thinks the economics justify even more. At the current rate of expansion, he considers about 100 tools by 2030 still the right number. Handing Zeiss $10 billion would change that, but the same would have to happen at every company in the chain.
Dwarkesh asked whether the labs will soon have enough cash flow to fund that expansion themselves. Dylan did not expect it this year, next year, or the year after, because the world is capital-constrained. He expects the labs to generate hundreds of billions in revenue next year against about $2 trillion of total capex. That total includes roughly $200 billion in wafer fab equipment plus larger amounts for data centers, accelerators, and energy. He added that the labs will never fully fund capex from cash flow, since the goal is always to invest more than current returns.
Compute prices have to rise for the labs to win
Reaching 100 gigawatts between them by 2028 would mean the labs taking 70–80% of incremental compute. Dylan said that would disrupt the market, because at today's prices almost anyone can make money on compute. His example: buy a GB300 rack, download Kimi weights, set up vLLM or SGLang (Codex and Fable can help), and list it on OpenRouter. He said revenue will exceed the compute cost. This has already started pushing prices above $10–15 million per megawatt. For the labs to take most of the market, they would have to pay $25, $30, or $50 million per megawatt.
Dwarkesh argued that the labs' lead in revenue per megawatt should keep growing, especially if they have unreleased internal models helping them build the next one. Players slightly behind, such as SpaceX, would then rationally sell to the highest bidder. Dylan agreed with that view, then raised the main caveat: regulation and safety constraints, which he said already slow the US labs more than Chinese open models. His examples were OpenAI not releasing Astra, OpenAI pausing training for two weeks, and Anthropic not releasing what its safety assessment calls "Model 2," widely believed to be the next Mythos. If labs cannot ship their best models, revenue per megawatt stalls or falls as other models catch up, and so does their ability to outbid everyone. In a world where safety didn't matter, Dylan said, the labs could earn $100 million per megawatt or more and pay $50 million. Everyone else would just ask Dario to take their compute.
Dwarkesh offered an intuition pump. If a gigawatt could sustain about a million fully automated white-collar workers at $100,000 each, that would be $100 billion. He found that surprisingly low and said full AGI would mean many hundreds of billions per gigawatt.
Which layer captures the surplus
Dylan said most of the value AI creates currently goes to users, not the labs. His examples were Jane Street, with an exclusive OpenAI contract for GPT-5.6 Ultrafast mode and a spot among Anthropic's biggest customers, and Meta, once rumored to be up to 10% of Anthropic's business. Both, he said, earn far more from the tokens than Anthropic earns in profit, through trading or through ad-algorithm and engagement gains.
Dwarkesh asked whether the market would reach an equilibrium where compute prices approach what the labs can earn from it, given the current 4x-or-more gap. Dylan described value capture moving through the stack. A year ago, the models ran at negative gross margins on VC money, hyperscalers built without knowing if it would pay off, and the hardware chain took the margin. In 2023, memory makers earned almost nothing on HBM despite its value. Now memory captures more than TSMC. The model layer has recently moved to large positive margins.
Elon Musk, in Dylan's telling, showed that the labs won't necessarily take everything. SpaceX sold compute to Anthropic and Google at $25–40 million per megawatt, enough to recoup its capex in about a year. Dylan still predicted that most compute will trade below $20 billion per gigawatt through the end of next year, because most compute is contracted and financed before it is built. A typical cloud needs a customer commitment to raise capital from credit markets. Meta and SpaceX are different: they have balance sheets to build without a signed customer, which lets them hoard compute and later decide whether to use it internally or rent it at high margins. Dylan called them the only plausible #3 players.
Revenue per megawatt and the regulatory brake
For the end of 2027, Dylan estimated lab revenue of at least $50 million per megawatt, possibly $70–80 million blended across the company, provided labs can keep releasing their best models. Dwarkesh said that seemed low given how far models have come in the past year and a half. Dylan predicted a bullwhip effect in prices. If Anthropic pays SpaceX $40 million per megawatt, Nvidia can raise prices, and so can SK Hynix, Micron, and Samsung. In his account, memory and substrate suppliers are raising prices quickly and TSMC slowly.
Dwarkesh argued that a leap like GPT-4o to Mythos 2 by the end of 2027 should push revenue per gigawatt far higher. Dylan kept returning to deployment. By his account, the best model in the world was trained in February, Mythos 2 isn't out, Mythos itself has been "neutered" so they can't use it to optimize inference, and Astra isn't widely deployed even inside OpenAI. He said regulation is also spreading to physical supply: New York banning data centers, Texas holding moratoriums, and Ohio trying to require operators to pay property taxes within a radius. That raises costs, which get passed on, and slows external progress even as internal models improve. Dwarkesh added that in a takeoff, a lab would have competitive as well as regulatory reasons to keep its best model six months ahead of what it releases. With accelerating progress, that gap matters more and caps revenue-per-megawatt growth.
Why the labs will shift compute from inference to R&D
Dwarkesh posed a hypothetical. Suppose a public lab with about 20 gigawatts wants to move from 60% to 70% of compute on training, giving up roughly $200 billion of revenue at $100 billion per gigawatt. How would investors react? Dylan's answer, which he called very non-consensus, is that labs will allocate less and less compute to inference, against the common view that inference will dominate. He expects more compute to go to forward passes for training than to revenue-generating inference.
His reasoning: at $30–40 million per megawatt, a lab might put 40% of compute on inference. At $60–70 million, keeping 40% would produce huge profits to spend on dividends and buybacks, or the lab could build AGI instead. He thinks executives and boards will choose AGI as the more profitable path. Ultrafast modes will serve internal researchers as well as customers, because the internal value is higher. On this view, the main purpose of inference revenue is to fund the training fleet.
Dylan said this is already visible. Anthropic adds more compute nearly every month, but after new revenue surged early in the year, it stopped adding around $25 billion of ARR each month. So, by his reading, the marginal megawatt is going more to R&D than inference, which he called self-evident to anyone watching closely.
China: under 10% of new compute, but its labs need less
Dylan said that in 2022 the US added about 45–50% of new compute and China 30–35%. Since then, export controls and the US buildout have left America with about 70% of new watts and China under 10%. China's domestic production and Nvidia purchases remain small, and some purchased chips end up elsewhere, such as Malaysia. He put China at 30 gigawatts or less by 2028.
He expects 2026 to depend on smuggled chips, TSMC chips made for companies later revealed as Huawei fronts, and HBM shipped by Samsung. SMIC and CXMT fabs start ramping in 2027, and especially 2028, reaching many millions of units a year and adding 5–10 gigawatts of domestic chips in 2028 alone. Those chips will be worse than Nvidia's, Google's, or OpenAI's 2028 chips. How steep China's hockey stick gets depends on whether the US passes the MATCH Act, whether tool export controls continue, and how fast China builds its own equipment. Dylan is sure it will come, since scaling manufacturing is what China does best. He called 50 gigawatts in 2029 "completely reasonable," partly from foreign purchases, though mostly domestic chips might make that worth about 20 American-chip gigawatts.
Dwarkesh concluded that the leading US lab in 2028 might have more quality-weighted compute than all of China in 2029 or 2030. Dylan qualified this: it assumes nothing slows the US labs, while politicians are already trying to, and China will only accelerate. Dwarkesh said that when he interviewed Jensen Huang he had steelmanned cooperation with China, partly because China controls supply chains robotics will need. He hadn't realized how lopsided the compute situation was and now sees export controls as a notable success. Dylan added that the gap also reflects finance: American markets are more willing to "YOLO" into startups, but once China's system targets an industry it subsidizes heavily. He said Chinese semiconductor subsidies exceed those of the rest of the world combined. If takeoff is slower, he expects China to catch up drastically on chips.
How labs actually spend their compute
Dylan also noted that Chinese models are not far behind in public perception given their compute. Leading Chinese labs have at most 100–200 megawatts, with ByteDance Seed the outlier, and Kimi is nowhere near a gigawatt. Anthropic will have more than 5 gigawatts by year end. He thinks the difference doesn't matter much yet, because of how labs split their budgets.
Labs have spent about 60% on training and 40% on inference, but the training share splits into roughly 50% research and 10% development. Research means testing ideas, architectures, data mixes, hyperparameters, and attention techniques. Development means the training run itself. He said Anthropic's Mythos pre-training used under 200 megawatts for about two months, and RL used less at any single site, though total compute was probably higher because the phases ran sequentially. Most of a lab's multiple gigawatts go to research. That is partly because coordinating clusters, multi-site training, and RL at scale are hard, and more RL rollouts don't necessarily help. As automated coding, automated research, and continual learning arrive, he expects the research/training line to blur and training's share to rise.
The scale of capex, and who pays
Dylan said 100 gigawatts a year at current prices would mean about $5 trillion of capex a year. Power plants are 30-year assets and data centers 15–20-year assets, and both must be built before the chips arrive. Commonly cited per-gigawatt figures of $40–50 billion cover only critical IT: servers, networking, fiber, transceivers, and optics. Accounting for the buildings and power for next year's larger buildout, he put the annual total closer to $7–10 trillion. Dwarkesh noted that $10 trillion a year by 2030 would be about a tenth of the world economy, and at today's US GDP, a quarter to a third of it. Saying that aloud, he wondered whether society simply won't allow it.
Paring back to 2028, Dylan estimated $3–4 trillion: over $2.5 trillion of IT capex plus $1–2 trillion for data centers, energy, and upstream supply chains. Nobody generates that much cash. The hyperscalers (Google, Microsoft, Amazon, Meta), which funded most growth so far, now spend all their cash flow on capex and borrow on top. Meta, Amazon, and Google already do, and Microsoft soon will. Stopping buybacks had some market effect, but by 2028 hyperscalers and their suppliers would be raising hundreds of billions in debt. He named the likely funders: chip and memory companies like Nvidia and Broadcom financing capex, infrastructure investors putting money into data centers instead of bridges, and everyone else. Individuals might skip buying a home or mortgage credit or government bonds to buy hyperscaler, data center, or Anthropic debt. Anthropic might pay 20% rates for an extra billion because that still beats renting from SpaceX at $50 billion per gigawatt.
Could AI trigger a sovereign debt crisis?
Dwarkesh laid out the argument they had debated off air. When a little investment produces a lot of return, the rate of interest rises. Heavy borrowing for data centers competes with governments, companies, and mortgage borrowers, raising everyone's costs. He expects the US to be fine if it builds data centers domestically and taxes them. But corporate income is under 10% of federal revenue, and over 80% comes from payroll and income taxes that automation will shrink. About 20% of tax revenue already goes to interest, and much of the debt rolls over within about five years. In his rough figures, a 1-point rise in rates takes interest to about 25% of revenue over five years. A 5-point rise takes it past 40%, and past 60% once you include the roughly $2 trillion borrowed each year. He expects countries with heavy debt, low tax revenue, and frequently rolled debt, such as Pakistan or Nigeria, to be hit very hard.
Dylan said this crowding-out is why the world won't build unlimited gigawatts. Consumer packaged goods, telecom, and banks all rely on debt. What matters is less the Fed rate than the spread markets charge as, for example, Amazon borrows heavily. SemiAnalysis models about $11 trillion of capex from 2024 to 2029: about $6 trillion from cash and $5 trillion from debt. That debt pushes rates up. Even so, he considers it not enough compute for the demand, so revenue per megawatt keeps rising. Regulation, angry consumers and politicians, and withheld models all push the other way, bending the buildout below what pure economics would want.
Asked for a rate, Dylan gave what he called a heavily vibed guess. Meta recently borrowed at 5–6% and would gladly pay 8% given the returns on compute. A roughly 250-basis-point rise would spread to everyone else. Banks would suffer because their debt reprices faster than their assets. Dwarkesh added that a higher discount rate hits equities built on long, steady cash flows. The index might be fine, but most individual stocks would fall, with Johnson & Johnson and railways as the examples. Dwarkesh cited economist Basil Halperin's prediction of a "second Volcker shock." In the 1980s, Paul Volcker's rate hikes, which Dwarkesh described as reaching about an 8% real rate, were followed by about 40 countries defaulting, mostly in Latin America. Dwarkesh expects something similar, and Dylan said all of this happens before any singularity.
Explosive growth and the reallocation of capital
Dwarkesh then went further. Citing researcher Damon Binder's input-output work, he argued that if the labor force can double every year, the whole economy could eventually double yearly, or at least grow tens of percent a year. Interest rates should roughly track growth, so in the 2030s rates might be tens of percent, possibly hundreds. In that world, he said, every country not producing AI defaults, non-AI stocks are worth almost nothing, the federal government can't service its debt unless it taxes AI, and mortgages become unobtainable. The underlying cause is the opportunity cost of capital: money used to pay pensions could instead build robot factories that build more robot factories.
Dylan applied this to markets. People ask why memory makers like Micron, Hynix, or Kioxia trade at 2–3x earnings. His answer: if you're truly AI-pilled, everything should trade at 2–3x, and the market should crash. So he thinks memory will do great but its stocks shouldn't 10x again. Conversely, he called Meta at about $1.5 trillion "silly," given the compute it is hoarding and could monetize through its own lab or by selling to Anthropic and OpenAI. The reallocation happens by pricing everyone else out. So the limit on AGI, he said, is not how fast researchers like their roommate Sholto work, but how much the rest of the world allows: regulation, interest rates, data center and fab opposition, and falling equity values that make other companies less able to buy AI. That pushes the labs to build their own chips and infrastructure. Dylan believes the models could support a fast takeoff but hopes a slow one is possible because of these frictions.
Dwarkesh worried more about something else. Restricting external deployment is counterproductive, he argued, because a rule like a six-month wait before public release would let recursive self-improvement happen inside the labs while the public uses models years behind. Slowing compute by a year also counts for little if RSI then yields 3–6 years of progress in one year. Dylan expects governments to restrict internal use too. He cited Anthropic's claim that it stopped giving Mythos to foreign employees for a while, and predicted the US government won't let Anthropic freely use "Mythos 4" internally. His reasoning is political: elected officials and their constituents already hate AI. Society might tear itself apart before AI arrives.
Most of the world's labor inside two companies
In the final segment, Dwarkesh raised what he finds most striking. Frontier compute is growing 4–5x a year in FLOPs, and the compute needed for a given capability falls about 3x a year. Together, the effective AI population at frontier labs grows about 10x a year. That matters little now, since AIs can't do full jobs autonomously. But if trends hold, OpenAI could go from roughly 10 million AI workers this year to 100 million next year and a billion after that. Dylan called it very plausible that by the end of the decade a single lab has more effective labor than there are people on Earth. Dwarkesh noted this holds without RSI, and with RSI growth might be 100x or 1,000x a year, or intelligence might rise instead of headcount. If these AIs are misaligned, most of the world's minds are; if not, very few companies still hold enormous influence.
Dylan brought up the recent dispute where Gavin Baker said Dario believes there will be only one company in the world, and Sholto and Dario denied it. Dylan's view: if labs use compute most productively and RSI is real, centralization follows. He asked Dwarkesh what non-centralized world he could imagine, calling the direction "scary as hell." Dwarkesh named the structural drivers:
- Training has huge economies of scale, because skills trained once are amortized over billions of sessions.
- Under compute scarcity, the leader can charge a higher markup.
- Models learning from deployment favor the most widely used one.
He called designing a decentralized, broadly empowered future that takes these economies of scale seriously a major intellectual project. The alternative, government control, doesn't reassure him. He said he trusts neither the government nor Dario nor Sam Altman.
Dylan contrasted this with capitalism's success through decentralized decisions, which AI may overturn by making a centralized AI economy grow faster. Dwarkesh noted the property may stay private, but AI is already roughly 2% of the economy by his rough math, concentrated in Nvidia, Anthropic, OpenAI, and the hyperscalers. Asked what could prevent concentration, Dylan said "I don't know." Unless progress slows or governments regulate heavily, he sees two paths: extreme concentration where we hope one company gets everything right, or a slowdown that preserves more balance of power.
Dylan's one hopeful point was that Anthropic doesn't capture most of the value. It still pays about $13 million per megawatt for much of its compute, and it can charge $100 million because customers like Jane Street capture $300–500 million per megawatt. Dwarkesh countered with Dylan's own earlier argument: shifting compute from inference to R&D makes sense only if labor is worth more inside the labs than outside. Dylan conceded that this was his "cope." If Anthropic can earn hundreds of millions per megawatt using compute internally, it has no reason to let Jane Street keep that value, and in his view that is already happening. The conversation ended there, with no answer to what could counter the pull toward centralization.
Okay, I'm back with Dylan Patel, founder of SemiAnalysis. Our version of a family Thanksgiving dinner is a regular yearly podcast. But we're not actually related. Don't tell the people this.
It will destroy the myth. Basically where the world economy is headed is more and more becoming a function of where lab economics are headed, where the compute market is headed, et cetera. I want to understand where the crazy future ends up within a few years. But let's start with where we are today. Walk me through lab compute and lab revenue right now, and maybe project out a year or two.
When we go back to last year, even at the end of the year, most of GDP growth in America was just AI infrastructure. As we look towards this year, about a third of the compute coming online is for the labs, for OpenAI and Anthropic. It may be built by others and then rented to them, but at the end customer, it's them. As we go forward into the future, the numbers for compute are ballooning. We're at a little bit over a trillion dollars of CapEx this year. As we go out into '28, it's going to be more than $2 trillion. The labs are also taking an increasing percentage of this. So ultimately, you've got a very interesting situation where the labs are going from companies that spend tens of billions of dollars a year to hundreds of billions of dollars a year, to forecasting to spend trillions of dollars a year even towards the end of the decade. This is at least some of the contracts they've begun signing with their partners.
This requires a big reshaping of what happens with their economics. Up until now, they have been companies that mostly lost money. Anthropic started turning a profit in Q2. It's believed at some point in Q3, OpenAI could start turning a profit even, with the bigger rise of Codex and 5.6 and all this. But if we go back a year ago, all the money they had was venture-funded losses. If we go back to even the beginning of this year, it was venture-funded losses. They've now turned the corner and are actually starting to profit. That doesn't mean they're not taking in new capital. The new capital is still coming in to accelerate the growth further. But ultimately, more and more of their business is being funded off of their own revenue rather than capital injections into them.
Over the last year and a half, their margins have really skyrocketed. The base cost of compute tends to be around $10 or $13 or $15 million per megawatt. The most interesting aspect about what's happening now is this: Before, if they served a model — GPT-4 being served on Nvidia Hopper GPUs — it was generating negative gross margin for OpenAI. But now, when OpenAI serves GPT-5.6 or Anthropic serves Opus 5 or Fable 5, their revenue generation has passed well beyond the incremental $10-15 million per megawatt. In the case of Anthropic, the revenue has gone as high as $50 million per megawatt. What that now enables them to do is: "Hey, if I spend 10 bucks on inference capacity, I actually generate 50 bucks of revenue, and then I can turn around and incrementally spend all of that profit on training."
One thing I'm very interested in understanding is how you see the centralization of compute happening at the labs, or the relative ratio of compute that goes to the world versus the labs. If you say right now a third of marginal compute is going to the labs, by when is over half of the incremental compute in the world going to the labs? By what point do the labs have basically a vast majority of the world's compute?
At the beginning of this year, OpenAI started at 2 gigawatts and Anthropic at less than 2. End of this year, they're both above 5. So they've 3-4x'd compute as a whole. When you look at the incremental compute added, that's about 30% of the compute added this year. As we step forward to next year, given what's already been signed and penned and inked, you've got something even more dramatic. Anthropic and OpenAI are taking as much as 40% to 50% of compute next year. This centralization doesn't look like it's slowing down or stopping. In fact, it looks like it's only accelerating.
Who's building that compute for them will change. Next year, a big new entrant is, for example, SpaceX, which is building a ton of compute. They're actively going to lease quite a bit of it to Anthropic and OpenAI, most likely, because they're the ones who have the marginal capability to pay the highest price. In addition, OpenAI and Anthropic are also starting to build their own compute — OpenAI with their own chips, Anthropic with TPUs that they're purchasing from Google and deploying with Fluidstack.
So you ask, "Hey, when does half of the world's incremental new compute go to just OpenAI and Anthropic?" It's really by the end of next year when half of the incremental compute is already going to Anthropic and OpenAI. Because compute is growing so fast, incremental compute is going to be basically most of compute.
So it's very soon — you're saying maybe within a year and a half or two years — that most of the world's compute is owned by two labs, or at least is serving the demand from two labs.
There's this trend where maybe world compute in gigawatts doubles every year, but the compute at the frontier labs triples every single year. If you keep the current trend going, it goes from 2 at the beginning of this year to close to 6 at the end of this year. Just multiplying out by 3. It's 18 by the end of 2027, 54 by the end of 2028. Are you like, "Okay, at that point, they simply can't continue tripling given the amount of world compute"? How do you see the world compute situation over the next few years?
If the incremental compute this year adds 30 gigawatts, next year 50 gigawatts, and the year after that roughly 70, you end up with this really interesting phenomenon. A new watt deployed this year is significantly more efficient than the watts deployed two years ago. A humongous percentage of the world's compute was deployed this year. Even though it didn't double the number of watts deployed, I'm deploying GB300s and TPUv7s and Trainium3s, which are way, way, way more efficient. They're 3-5x more performance per watt than the prior-generation chips.
So ultimately you've got a huge ladder here. If Anthropic and OpenAI take on 45% of compute next year, you've got them in, let's say, December '27 having taken on half of the world's incremental new compute. But that half of the world's new incremental compute is actually at a higher performance than everything else before it. So you've got another multiplier on that. By the time you're towards the end of 2028 — if this trend continues, and I see nothing that's stopping it — you've got them just controlling most of the usable flops in the world on their own.
The thing I'm confused about is why you think we only add 80 gigawatts in 2028 if we enter a world in which the value of compute increases so much.
That's the upper bound, by the way. That's the like, "I'm so fucking bullish."
Okay, let's do some chain of thought here. When I interviewed you a few months ago, you said that in order to make a gigawatt of, I think, Vera Rubins, you need 55,000 N3 wafers, 6K N5 wafers, and 170K DRAM wafers. I know if those numbers might have changed.
I'm going to troll you, but the way you said wafers was so fucking Indian. Vafers. By the way, when we first moved to the US, I had the v/w thing pretty bad, and I was a vegetarian.
I remember you told me about this. In North Dakota, I was in elementary school, and I'd be like— Can I get a "wedgie"? Can I get some "wedgies"?
Anyways, so that's for one gigawatt. I had an LLM run your wafer fab equipment model and figure out how much the tooling costs to produce a gigawatt of compute basically every single year. It said $3-4 billion. Now suppose you add in cleanrooms and shell and everything else at the fab. So $6 billion of fab CapEx produces a gigawatt every single year. A gigawatt produces right now $100 billion of revenue. But also that $6 billion in CapEx is producing a gigawatt every single year, and that gigawatt is producing $100 billion every single year. So over the course of five years, the first gigawatt has generated five years of profits, the second gigawatt the fab has produced has generated four years of profits, and so on. $6 billion of CapEx at the fab level will have generated over a trillion dollars of end AI revenue.
Yeah. There's a lot of OpEx along the way. There's a lot of other CapEx, like the data center, the power. And you had to pay OpenAI for the R&D. Installation. There's a lot of different people who need money here.
Take away half of it for all these middlemen. That still means there's a 100x discrepancy between fab CapEx and end revenue generated. More than that, actually, but we're just being very conservative. As a result… This is capitalism. You have this huge discrepancy where you can turn $1 into $100. They're not going to figure out a way to make more mirrors?
They are. It's just that these mirrors take some time to make. But the emergency is so big where Anthropic and OpenAI are like, "We could make a trillion dollars right now, but we're just bottlenecked on the mirrors that go into the ASML machines." How can we make more mirrors if we spend $100 billion on this? That's the situation we're going to be in pretty soon.
We're not going to be able to solve that supply constraint? That just seems quite hard to imagine.
You've seen people do funny arbitrages here where they buy turbines and then try and resell them, because the value of a turbine is way more since it's the thing bottlenecking your data center. I think if anyone had $400 million and the ability to convince ASML to sell them an EUV tool, they should totally just go buy one, wait, and then sell it for north of a billion dollars.
But ultimately, yes, capitalism will cause these things to expand. But it's a whip. It takes a long time for the whip signal to get to the tail end of that. The supply chain doesn't react immediately. In fact, you go talk to someone at Carl Zeiss, they're like, "Yeah, yeah, yeah, we need to make 100 EUV tools by the end of the decade." When we had our episode earlier this year, they didn't even think they needed to make that many, enough mirrors to make 100 EUV tools a year. Now they're like, "Okay, we need to do that." But in reality, because of all the economics of what's going on, it should be even more. It takes so long to pill.
Suppose that every single company in the stack got private equitied. Somebody came in who was super AGI-pilled and was like, "We're going to maximize production." What do you think the physical constraints on making more things would be? The reason I ask is we're pretty soon going to be in a world where the lab revenue, or just AI cash flows — because obviously the accelerators also have these huge cash flows — will be so big that you can just fund extreme expansion of all this production from cash flows themselves.
I do agree generally. There's obviously some physical constraints. The way the supply chain is expanding currently, 100 is roughly still the right number.
For 2030? 100 ASML tools for 2030.
But if you said, "Carl Zeiss, here's $10 billion. Please fucking just expand production," that would change things. You would have to do this with every company in the supply chain.
But you don't think that's gonna happen next year? I don't think it'll happen this year. I don't think it'll happen next year. I don't think it'll happen the year after, because the world is capital constrained.
But in a world where, say, the top labs are generating, even combined, a trillion dollars in revenue next year, they're not able to take $10B of that— I don't think they're going to do that, but… Or hundreds of billions at least? It just seems like they realize where the world is headed. I feel like they could just make…
The thing is, the labs are going to generate hundreds of billions of revenue next year. But ultimately, CapEx next year is like $2 trillion. So you've got this big mismatch.
The wafer fabrication equipment supply chain will do something on the order of $200 billion. The data center market supply chain will do even more. The accelerator supply chain will do even more. The energy supply chain will do a number. You sum all this up, it's going to be well north of $2 trillion of CapEx. So the labs have not yet gotten to the point where their cash flows can fund this stuff.
Obviously they will never get to that point, because you want to keep your CapEx higher than your returns. Yeah, you reinvest.
The key question I really want to understand is: if the current trend continues, it'd be north of 50 gigawatts per lab by the end of 2028. So between them they'd have 100 gigawatts. Those gigawatts, as you're saying, drive many-fold more throughput or performance by 2028 than they do now, because the hardware's gotten better. Not only have flops per watt increased, but also the hardware gets better at working with AI workloads.
Okay, so 100 gigawatts for the labs by the end of 2028. How much is world compute?
I think that may be a little difficult, given that by 2028 they've taken 70-80% of incremental compute. And I'm not sure what happens to markets then. How much does the price of compute skyrocket for them to actually be able to buy 70-80% of compute? Is Google or Meta or Amazon willing to sell even that much?
Also, there's one caveat when we're talking about these gigawatt numbers. When Amazon is serving Bedrock Anthropic models, that counts as Anthropic compute in our worldview, because it is effectively, at the end of the day, counted as revenue for Anthropic even though there's a revenue share and credit back all that.
But ultimately in 2028, if they get to 100 gigawatts combined, they have done really disruptive things to the market. Because anyone can make money off of $10-15 million per megawatt compute today. I kid you not, it's not that hard. Go get a GB300 rack, go download the Kimi weights, go download vLLM or SGLang, set it up. Codex and Fable can actually help you do this. It's pretty simple. It's not trivial, but it's not rocket science. Go put it on OpenRouter. It's very simple. You'll start generating more revenue than you're paying for the compute.
This has already led to this compute pricing, $10-15 million per megawatt, starting to inflect up. To get to that 100 gigawatts in 2028, you have to believe that the labs can outpay for compute, because anyone can make money at $10 to $15. Does compute now get to $25 million a megawatt? Does it get to $40 million a megawatt?
As you're saying, it's already the case that the labs are generating way more revenue per megawatt than everybody else. If they stay as far ahead as they are currently, you would expect that to continue being the case. If there's some kind of recursive self-improvement where the AI labs are relatively uplifted — or they have models internally they're not releasing externally that are helping them make their next model better — you'd expect that to be even more the case. Aren't you already seeing this, where SpaceX, or whoever is slightly further behind, will just sell compute to the highest bidder if they can't internally monetize it as well as the labs? You'd expect them to keep bidding for larger and larger shares of the compute market.
I think that is my worldview. They will continue to gobble up more of the compute. But ultimately they can't do it at current pricing or anywhere close to it. They do have to start paying $25, $30, $50 million a megawatt to really gobble up 70% of the world's compute in 2028, to get to 100 gigawatts by 2028, which is a very aggressive goal.
The other aspect of this that's really challenging is that we've already seen a huge slowdown for the AI labs. This regulation that they advocate for is actually slowing down the labs a lot more than it slows down the open-source Chinese language models. OpenAI not releasing Astra. OpenAI stopping training for two weeks.
Anthropic not releasing what their safety assessment says is Model 2, which is widely believed to be the next version of Mythos. They’re clearly not releasing their best models, in which case their revenue per megawatt stalls or can even start to decline again because other models are competitive again. It’s not that they’re falling behind. It’s just that they’re not releasing their best stuff.
What if there is some regulatory impact that prevents them from releasing their best models? Now their revenue per megawatt does not climb as fast. Their ability to buy that incremental compute for a higher price than everyone else starts to diminish, and then maybe they can’t get to that 100 gigawatts.
But in a world where safety doesn’t matter, I do believe that’s exactly what happens. They can start generating $100 million per megawatt or more, and they can pay $50 million a megawatt. No one else has any logical reason to do anything with their compute besides say, "Please, Dario, take everything off of my hands."
But there are forces at play, which we cannot describe, that would potentially slow this down.
I think a good intuition pump is: what if the AI models were literally as good as a fully automated software engineer? They’re not currently there yet. I think they’re far from being able to fully automate the job of a full white-collar worker. But white-collar workers earn six figures or north of that a year. If you have a gigawatt that can sustain a population of, say, roughly a million white-collar workers. Then off the back of that… That would be $100 billion.
That’s actually surprisingly low. Yeah, $100K per person, million population. I don’t know. But it would be many hundreds of billions of dollars per gigawatt if you get full AGI.
The other aspect of this — and we’ve continued to see this — is that most of the value capture is not happening. Most of the value that these models generate does not get given to OpenAI and Anthropic. Thankfully, so far it is mostly just being given to the users.
Jane Street, with their exclusive contract with OpenAI for GPT-5.6 Ultrafast mode, or Jane Street where they’re one of Anthropic’s biggest customers, is generating way, way, way more value out of the tokens they’re paying for than Anthropic is generating in terms of profit, because they get to make money off of the market. Or take Meta, who at one point was rumored to be as much as 10% of Anthropic’s business. They’re generating way more efficiencies by optimizing their ad algorithms or what have you, getting engagement time 5% longer, all these things. They’re making way more money off of using these models than Anthropic is. That’s what’s required. Sure, if you had a million new software engineers, the cost for a software engineer would also fall.
One thing I’m confused about is, does the market come into equilibrium? If it comes into equilibrium, would you just expect the price of compute to equal whatever Anthropic and OpenAI can generate from it, or be very close to it with a small amount of markup for Anthropic and OpenAI? Right now it’s really weird that there is a 4x or more difference between what compute sells for and how much money Anthropic can make from it. In a world where the revenue per gigawatt continues to increase, if Anthropic’s ability to monetize a gigawatt doubles or triples, it’d be weird if the gap continued to increase. Anthropic, just by having some weights, can take something that cost them $10 and turn it into $100.
This is always a fun question. Where does the value go in AI? AI’s generating all this value. You’ve got the end user, which I think we all agree is generating more value than anyone else, hence they’re paying a lot for these models. Then you have the app layer. So far the app layer’s generated very little value. Then you’ve got the model layer, which up until a year ago was generating negative gross margins and is now generating massive positive gross margins. It looks like it’s on the path to generating $100 million per megawatt. So turning $10-15 into $100, as you said.
But if we go back a year ago, the hardware supply chain was generating all this gross margin while literally everyone else was losing money on it. OpenAI and Anthropic were just plowing VC money in, as were many other startups. Many of these hyperscalers were building infrastructure without knowing if there was going to be a payoff. So ultimately you had this negative value being created on the model layer, if you will, because they were selling the tokens for less than it cost them on the infra side. All the value was being captured at the chip, the fab. Initially in 2023, the memory guys were making no money off of HBM or memory for AI, even though theoretically the value they were delivering was humongous. Now you’ve got… Well, actually TSMC captures way less value than the memory guys.
So the value capture’s shifted around a lot, which is very fun for people tracking the market or participating in the market, like Jane Street as an example.
This is not an ad. This is not an ad. This is not an ad. They’re a sponsor but you don’t have to plug them that hard.
So what happens going forward? Anthropic and OpenAI have slowly started to balloon in value capture. Do they balloon and take all the value capture? Well, that was a thought, and then Elon showed, "Actually, no. I can sell my compute for $25 million a megawatt or $40 million a megawatt to Anthropic and Google. Even if it’s a short-term thing, I’ve sold it for this price, and I’ll recoup my entire CapEx in a year."
What’s your prediction of how much the relevant tranche of compute — B300s or whatever that SpaceX sold for $40B a gigawatt to Google — what does that sell for at the end of next year?
I think most compute will still continue to transact at sub-$20 billion a gigawatt. Even at the end of next year? Because all of it has to be financed.
If Meta, Microsoft, Amazon, SpaceX can build compute without finding a customer, just saying, "Fuck it, I’m going to build this compute," and then turn around and wait till it’s already built, they now control what’s going on. Most compute is contracted well before it’s built.
This is what Elon took advantage of in the market. He actually had all this compute. He was like, "Hey, Anthropic, I know you’re making $60-plus billion per gigawatt. Why don’t you just buy my stuff for a crazy amount of money?" Obviously it’s not like Elon decided this or Anthropic decided this. The market figured itself out.
Other people, you go to a random cloud, they’re like, "Okay, I’m going to build a gigawatt of compute or 100 megawatts of compute. I’m going to spend the CapEx. I need to turn around and find a customer. If I want to find a customer, I need to find the capital. Who’s going to give me the capital and the customer? The customer has to sign a deal. Then I take the customer’s commitment to the credit markets and I raise the capital."
So there’s this completely different power structure where Meta is effectively hoarding compute. Them and SpaceX are the only plausible #3, because they’re hoarding all this compute. They’re using their balance sheets and capabilities to build compute without an end customer that’s monetizing at a huge degree. They have an actual balance sheet, so they can go to the credit market. You build a gigawatt, you can make your margin, not a crazy margin, but a good margin. Now I have all this compute. Now Meta and SpaceX have this optionality of looking around and being like, "Is my internal use case going to make me more money, or should I go out there and sell it to Anthropic or OpenAI at crazy margins?" So now we’ve entered a regime where SpaceX and Meta are saying, "Actually, I’m going to build the compute, and I can rent it out for not $13. I can sell it for $25, $50, and more."
So as I've been doing video essays and other formats, I've been looking to hire a new editor for the podcast. But actively searching for editors has been quite time-consuming because the vast majority of candidates don't fit the profile that I'm looking for. So I created a recruiter in Grok Bot to see if it would help. I gave it a huge context dump where I monologued basically everything that I wanted, and then it spun up four other bots to narrow in on different parts of the search. One went through the last year of my email for relevant inbound. One searched my X feed and DMs. One went through the end credits on various documentaries I like. And the last one looked for editors who work for some of the YouTubers that I follow. Grok Bot then took all these different candidates that the subagents had found, filtered them against my criteria, and delivered for a final shortlist to review.
To be honest, the first batch had a few good candidates I wanted to see, but mainly a bunch of duds. But after I gave Grok Bot some more feedback about what it was missing, it came back with a new list of candidates that I'm actually extremely excited about. I ended up saving this whole workflow as a routine. So every week now Grok Bot checks my inbound email and X DMs for promising new candidates to potentially interview. If you want to try Grok Bot yourself, go to x.ai/bot.
What do you think their revenue per gigawatt is by the end of 2027? For Anthropic or OpenAI, by the end of ’27. I think it’s highly dependent on who has the best model, if they’re allowed to keep releasing their best models. But I don’t see why it wouldn’t be $50-plus million a megawatt. By the end of ’27. Oh, by the end of ’27? That’s where it gets more challenging, but I think it could get higher than that, to like $70, $80 million a megawatt, blended across the company, if not higher. Seems low.
So if that’s the case, then what happens to the price of compute? Well, if I’m Anthropic, incremental compute is worth it. Maybe I spend $40 million a megawatt on SpaceX compute. If I’m SpaceX, I look to the supply chain and I’m like, "Well, I’ve struck this deal with Jensen (where he’s now all of a sudden using Twitter)." And Elon’s saying they’re exclusive to Nvidia, but why doesn’t Jensen raise his prices? Then SK Hynix and Micron and Samsung look at it and they’re like, "Well, why don’t we raise our prices?"
So with the value capture, I think there’s a bullwhip effect here. Just because someone has raised prices doesn’t mean the entire supply chain rebalances immediately. But over time, the supply chain will rebalance and things will cost more and more. To get that incremental capacity, you sort of have to. So TSMC raising prices very slowly, but memory companies raising prices very quickly. Substrate companies raising prices very quickly. Elon wouldn’t have sold if it was $15, but he’s selling because it’s $25+. So obviously he raised his prices really quickly.
I’m surprised you think that revenue per gigawatt doesn’t increase way more than even 100 per gigawatt by the end of next year. When does RSI happen? When does takeoff happen? Or even if RSI doesn’t happen, just say the current rate of progress continues. Just look at how much progress we’ve made in, let’s say, the last year and a half. What was the model from a year and a half ago? Claude 3.5 or something?
My problem with this is that the best model that exists in the world was trained in February. So you’re saying maybe we just won’t be allowed to release the labs’ best models. OpenAI says they’re not training models for two weeks, man. What the hell? There’s one thing where internally, are they getting enough use for it that they’ll bid up the price of compute? Another is, does AI progress as a whole slow down because of regulation? Yeah, but they’re not even allowed to use this new model internally. Astra’s not even widely deployed internally.
But still, if you have a model that is… What was the model released at the beginning of last year? GPT… 4o? Was that 4o? Yeah. You’re talking about a GPT-4o to Mythos 2-size leap by this point, again, by the end of 2027. Yeah, but Mythos 2’s not out. Or even Mythos. That leap again. Even Mythos is not allowed to be out. They’ve neutered it. We can’t use it to optimize inference performance. We can’t use it to optimize all sorts of things.
Yeah, maybe there’s some slowdown in AI progress or the deployment of AI that means the revenue per gigawatt can be lower. But that’s the only way I could see it being only $100 million per megawatt by the end of next year. As long as the model gets better, the value generated out of it gets better. Obviously, who captures the value is still up for debate, but ultimately everyone’s going to raise their prices. Because they can, and it’s super inflationary. Especially if the method of regulation is… Right now, so far, it’s just "don’t release the models." But more and more, the method of regulation is New York’s banning data centers. Texas is holding moratoriums. Ohio’s saying, or at least trying to say, you have to pay everyone’s property tax in a certain radius. These sorts of things are going to decrease supply and increase cost. That’s going to get passed on as well. You start to end up in a spot where progress does slow, at least in the external sense, even if the models internally keep getting better and better.
In a takeoff scenario, why would Anthropic not have their best model six months ahead of what is externally available? Because of safety and regulation, but also the competitive advantage? That six-month difference, if progress accelerates, is actually a bigger differential. So that’s the thing that would cap revenue-per-megawatt gains to much lower growth than we’ve seen in the first half of this year.
Here’s something I’m very interested in. As these companies go public and they’re accountable to investors, let’s say by the end of next year they have close to 20 gigawatts. So 10% of compute is 2 gigawatts. Let’s say they want to go from 60% of compute to training to 70% of compute to training. And their investors are like, "Well, if you’re going to be able to generate $100 billion per gigawatt, you’re basically saying no to $200 billion of revenue in order to increase your training compute." So investors are like, "What the fuck? You’re already spending so much on training. Why are you spending even more on training?" As a public company, what do you think would happen if they’re just like, "No, we will keep increasing the share of compute we spend on training to offset the increase in revenue that each gigawatt of compute is giving us"?
This is what I personally believe. The labs are going to allocate less and less compute to inference over time. I think that’s very non-consensus. The standard belief of most people is, "Oh, most compute will go to inference." Most of it will go to forward passes for training, not necessarily revenue-generating inference. Ultimately, if they’re generating $30-40 million per megawatt today, you allocate 40% to inference. If you now get to generating $60-70 million per megawatt, do you still allocate 40% to inference and generate all this profit and then do dividends and share buybacks? Or do you go build AGI?
I think the obvious answer from Anthropic and OpenAI, not just at the executive level but also their board, is to go build AGI, because it’s way more profitable. So ultimately you’re going to see them ratchet up their percentage of compute dedicated to training— While each increment of compute is getting more and more profit-generating if they had dedicated it to inference.
Right. The whole point is, if I’m selling tokens… Is OpenAI releasing Ultrafast mode for just external, or are they doing it internally too? It turns out, no. Actually, I’m going to allocate it to internal and external, because the internal value I’m generating from super-fast AI or the best AI model is way more than what someone external is. So ultimately, sure, I could generate $100 million per megawatt, but if I turn that towards AI research, what is the incremental progress that I get? What does that do towards my future earnings potential, the discounted
cash flows of whatever the hell I’ve done? They’re not going through that calculation, but ultimately it makes more sense to dedicate more and more compute internally. The only reason to have inference compute be so large is so you can grow your training fleet.
I think this is an interesting economics question that I feel we can have the models digest. What would have to be true about a world where they reduce the fraction of compute spent on inference?
I think they have been over the last three months already. I think at parts of this year, they were increasing the fraction of compute… Let’s just take it month by month. You would agree that every month, Anthropic has added more compute than the prior month. There might be some noise when they sign a SpaceX deal or whatever, but in general, the amount of compute is a curve up. So in January, they added less compute than December, and yet their revenue adds skyrocketed. Then they’ve sort of plateaued. They’re not adding $25 billion of ARR every month now. That means the marginal megawatt they’re getting is going as a higher percentage to R&D than it is to inference. So they are factually increasing their compute towards R&D today. I think this is self-evident if you look enough at what they’re doing.
If I look at the numbers you said for how fast world compute grows, here are some things I want to understand. It seems like if I add up the numbers you just said, it would be over 200 gigawatts of world compute by the end of 2028, right?
Yeah, globally.
Okay. How fast can that continue growing, global AI compute after 2028?
30 this year, 50 next year, 70 in ’28. ’29 should be on the order of 90-100.
Then just 100 more every single year or something?
I think the slope can continue to go upwards. It’s hard to predict anything more than four years out. Who knows whether we’re in an RSI regime, or when is the world economy growing at 10% a year? Because if you’re at 100+ gigawatts a year, you’re at absurd GDP growth.
If you think there’s 200 gigawatts globally in 2028, how much is in China by that point? How does Chinese compute continue increasing through this whole trend? Because if the RSI stuff kicks off in the West before China has a large amount of compute, maybe we’re living in a different world than when it doesn’t.
If we level-set back to 2022, the US was adding about 45-50% of the world’s compute. China was adding about 30-35%. The rest was being taken up by the rest of the world. Since 2022, we’ve had big regulations against China and a dramatic increase in America. So today, 70% of watts are being deployed in America.
China is really a very small number. Sub-10% of watts being deployed for data center AI compute is in China. As we step forward, they’re still at a very small number. Their domestic production is quite small. Their purchasing from Nvidia is still quite small, and a lot of that ends up in other places as well, Malaysia or what have you. So ultimately, China domestically still continues to have sub-10% of incremental new compute. In 2028 it might start to inflect up, I think. But it’s pretty easy to say China will have 30 gigawatts of AI compute or less.
By 2028?
Yeah, in 2028.
Okay. And then how fast does their hockey stick go up?
I do think in 2028, they have a big uplift in what compute they’re able to deploy. In 2026, they’re still mostly relying on a lot of the smuggled chips, a lot of the chips that TSMC made for companies that they thought weren’t Huawei but ended up being Huawei, or a lot of HBM that Samsung is shipping. But in ’27, fabs start to go up. In ’28 especially, fabs start to go up from SMIC and CXMT and such, where domestic production is actually reaching many millions of units a year. Now they’re incrementally adding 5-10 gigawatts, in just 2028, of domestically produced chips. Those chips are definitely worse than the chips that Nvidia will have in ’28, or Google will have in ’28, or OpenAI will have in 2028.
So even the gigawatt number overstates things, you’re saying. It’s 30 gigawatts, but it’s really much worse chips. But if you think the world is going to add 100 gigawatts the following year — I know you said you can’t really say that far out — how much is China able to add the subsequent year? Basically, I want to know: do they just hockey stick at the point at which they are able to start shipping large amounts of compute, or is it still going to be less than US plus allies?
There’s a lot left to whether or not the US passes the MATCH Act, whether or not tools continue to get export-controlled, how fast China can build their new equipment that they’re starting to be able to produce domestically. But ultimately, China is definitely going to hockey stick. If there’s anything China’s really good at, it’s scaling manufacturing really, really quickly. I imagine China will start to be able to extract more and more purchasing of even foreign chips into domestic China, or at least close the gap in what the US is allowing Nvidia to sell them, or what have you.
But do you think China could be adding 50 incremental gigawatts in 2029?
I think that’s completely reasonable. Part of that could also be purchased from foreign. But yeah, I think it’s completely reasonable that China in 2029 can do 50 gigs. But if most of those are domestic chips, there is some factor there where that 50 gigawatts is really worth as much as 20 gigawatts from American chips.
Right. So you’re actually projecting a world where maybe the leading lab in 2028 has more compute than all of China will have in ’29 or even ’30, if you weighted gigawatts by their quality.
Implying that there’s nothing done to slow down the US labs.
That’s right.
But clearly the government and politicians are starting to do that. Whereas China’s not going to slow down AI. In fact, the only thing they’re going to do is accelerate it.
Honestly, when I interviewed Jensen and asked about export controls — I am a libertarian person — I wasn’t genuinely sure what I thought about this issue. I was steelmanning the opposite view from what he has, because I think it’s important to hash out ideas. I’m like, "Yeah, maybe there’s a world where if we just cooperated with China, it would be better for us, especially since they control so much of the supply chain and the other things that will be needed for robotics." But I didn’t realize the compute situation was as fucked as you’re saying. Actually, the export controls do seem to have really… If they ship the amount that you’re saying, that’s a huge difference. By the time we have automated coder and are getting into automated researcher, China is way far behind on the compute stock. If that ends up being the case, that would have worked.
I think that’s actually a notable success. The only caveat there is that some of it is export controls, but some of it is also just financial systems. American financial systems are more willing to YOLO into startups than Chinese financial systems. But once Chinese financial systems choose an industry to focus on, they’ll subsidize it a hell of a lot more. So the Chinese semiconductor industry has significantly more subsidies than the rest of the world’s semiconductor industries combined.
If takeoff is not as fast as you’re implying but actually takes longer, then ultimately China will catch up drastically on the semiconductor side, which then is compute at some point.
The other noteworthy aspect of this is that Chinese companies today are not that far behind in AI models, at least perceivably by the public, relative to the amount of compute they have. The leading Chinese labs have 100-200 megawatts total of compute at most, ByteDance Seed being the one outlier where they have significantly more than that. But Kimi is not running a gigawatt or anywhere close to it. Whereas Anthropic is more than 5 gigawatts by the end of the year. So the question is, does it matter? I think right now this difference in compute doesn’t matter that much.
When we break down the compute ratio or budget of a lab, so far it’s been 60% training, 40% inference. But that training gets broken down further. Actually 50% of the compute is research, 10% of the compute is development, and then 40% is inference. What I mean by research and development is: researchers are generating ideas, testing new architectures, testing new data mixes, testing new hyperparameters, new attention techniques, blah, blah, blah. But ultimately when they do the training run, when Anthropic trains Mythos, it’s sub-200 megawatts.
The pre-train or the whole thing?
The pre-train. It’s sub-200 megawatts for, call it, two months. Then the RL is even less.
You think the RL was less compute than the pre-train?
At least in terms of single site of pre-training, yeah.
But total compute was probably higher, right?
But it’s sequential. At most, the most they ever used at one point in time was maybe 200 megawatts. In reality they had multiple gigawatts, so most of their compute was going to the research, not the development of a model. There’s reasons for this. It’s hard to coordinate all these clusters. It’s hard to co-locate all of them. It’s hard to do multi-site training. It’s hard to do RL. Generating even more rollouts during RL does not necessarily make it better. There’s all sorts of reasons why you may not be able to leverage all two gigawatts that you have onto training. Actually, I can only leverage 200 megawatts.
As we get further and further down automated coding and automated researcher, I actually expect the percentage of the compute budget that goes to research versus training to become a lot more fuzzy, or even higher for training. Also things like continual learning. All of these things start to mean that more and more is actually going to training the model.
If you end up in a world where you’re doing 100 gigawatts a year, at current prices, that would be $5 trillion of CapEx every single year. Then stack on the fact that you have to build the power plants way before then. It’s also a 30-year asset. You stack on the fact that the data centers are a 15-, 20-year asset, and you have to build that then too. So the $5 trillion, once you account for future years’ growth, is actually going to be more like $7 or $10 trillion of CapEx.
Wait, I didn’t understand. That doesn’t include the fact that there’s not the infrastructure for the power generation in the data center itself.
Right, exactly. When you talk about AI CapEx, people are saying $40, $50 billion. But that’s really just the critical IT: the servers, the networking, the fiber, the transceivers, optical communications, all this sort of stuff. It doesn’t account for the data center itself or the power plants themselves, which are being built ahead of time. If I’m building 100 gigawatts this year and 150 gigawatts next year, then all of the buildings for that 150 gigawatts need to be built in CapEx this year. If I’m building 200 gigawatts the year after that, all those power plants need to be spent… You have to buy the turbines this year. So actually, it’s much bigger than even $5 trillion if you’re building 100 gigawatts.
Right. Very plausibly, incremental CapEx every year is getting close to $10 trillion by the end of 2030, which is going to be close to a tenth of the world economy. If all of it’s going up in the US… The US economy will have grown as well. But still, at the current size of the US economy, it’ll be like a third to a quarter of the US economy just going towards data centers. As I say that out loud, I’m like, "Maybe you’re right and we just won’t allow it, and that’s the reason this doesn’t happen." Because for this exponential to continue, a quarter of America’s economy is just building data centers.
I believe in capitalism and reallocation of resources towards the most profitable thing. But at the same time, politics exist, credit markets exist, and capital markets exist. So to enable, let’s say, that 100 gigawatts by 2030… Or let’s even pare it down to 2028, where it’s like $3 or $4 trillion of CapEx across all of these items: over $2.5 trillion towards IT CapEx, and then another $1 to $2 trillion on data center and energy, and all the supply chain downstream, like semiconductors and all that stuff. If you’re at $3 or $4 trillion of CapEx, where does all this cash come from? No one is generating that much cash from the business yet.
Hyperscalers funded all of the growth up until now. Google, Microsoft, Amazon, Meta. They funded a huge percentage of it. They were more than half of compute, but they now don’t generate cash. They actually spend everything on CapEx. In addition, they raise debt and spend everything on CapEx. You’ve seen Meta do it, even Amazon, even Google. Microsoft will be there soon. Everyone is raising debt to pay for their CapEx. Now who is the incremental person to pay for this that was not doing it before? In the case of Google, it was pretty simple for them to stop doing buybacks, or Meta stop doing buybacks, and turn around and buy computer infrastructure. That doesn’t have a huge effect on the market, but it does have some effect. But as you step forward to 2028 — where the hyperscalers are now raising hundreds of billions of dollars of debt, and then all of their supply chain is raising hundreds of billions of dollars of debt — who pays for this?
So there’s a few different ways. There’s semiconductor companies like Nvidia and Broadcom and the memory companies turning around and deciding to fund some of this CapEx. There’s the traditional infrastructure investors who are gathering capital and investing in infrastructure. Instead of bridges, it’s data centers.
Then lastly, there’s everyone in the economy who’s realizing, "Maybe I shouldn’t buy a home, or maybe I shouldn’t invest in credit that’s helping people buy homes, or maybe I shouldn’t buy government debt. I should just buy hyperscaler debt, or I should buy this data center’s debt, or I should buy Anthropic’s debt. Because Anthropic’s willing to pay 20% rates for the incremental billion dollars to build their capacity. Because they know their revenue from it’s going to be huge, and they’re going to pay 20% because it’s still better than renting it from SpaceX for $50 billion a gigawatt." So you’ve got all of this contention. But if you now do this, the whole world economy is really shifted around.
Antithesis is a deterministic software testing platform that enables perfect reproducibility. It also unlocks some pretty insane approaches to debugging. Like time travel. With Antithesis, you can jump to any point in a trajectory and start from there. So when there’s a crash, you can rewind to the exact moment that something went wrong and freeze the entire system: the application, the database, even the environment itself.
This lets you do something that would otherwise be impossible, which is to observe every part of a distributed system at the exact same instant. Time travel also allows you to add telemetry and logging to an event that has already happened. For example, you can rewind to five seconds before a crash and decide to capture all the network traffic.
Most powerfully, Antithesis gives you a live terminal into your system that you can use to perturb anything you wish. Kill a node or disable a feature, then hit play and see what happens. Then go back and try something else. In production, you often only get one shot on goal with this sort of destructive analysis. If you restart a deadlocked service, for example, the exact deadlock you needed to study disappears. But with Antithesis, the original timeline is always reproducible, so you can test as many hypotheses as you need. And if you don’t want to do all this time traveling yourself, you can just have your agents do it for you via the Antithesis API. Go to antithesis.com/dwarkesh to learn more.
You and I have been debating off air for the last few days whether there will be a sovereign debt crisis as a result of AI.
The logic is this. As we were mentioning, you have a situation where very little investment turns into a lot of money. So the rate of return—
What a fucking problem, dude. Oh my God. Can’t believe it.
No, it is a huge problem for everybody else who can’t turn a little money into a lot of money.
So the rate of return is incredibly high. Even at the data center level, if you build a data center and you’re trying to get rented out to an Anthropic or an OpenAI for 10x what it costs you on a depreciated basis to build it, it’s fucking crazy. You turn $1 into $2 or $10 or something at the end of the year.
That raises the rate of interest higher. Now, if the rate of interest goes higher, and if it does that for the entire economy… People are borrowing more and more money. They’re competing against the other lending that the government would’ve done, or that other companies would’ve done, or that you as a consumer or a mortgage buyer would’ve done. That’s making it more expensive for everybody else to borrow. This has huge implications for tons and tons of people.
Sorry, I’m going to go on a bit of a monologue here, but we’ve been thinking about this together.
I think the US will be fine at the end of the day. Because if the data centers are built in America, you can fundamentally just tax the data centers. But the way the current tax system is set up, corporate income is less than 10% of federal revenues. 80%-plus is payroll taxes and income taxes, which, as more and more automation happens, will shrink.
At the same time, on the spending side, currently 20% of tax revenue spending goes towards servicing the debt, paying interest payments on the debt. Now, a lot of the debt is short duration, so it rolls over every five years. Why are you fucking laughing?
Because it’s things you’ve learned in the last month.
Like it’s any different for you. Like you got a degree in fucking financial economics. I didn't. The internet thinks I’m a beekeeper.
Few months, few months. This is our business, Dylan.
I know, I know. Sorry, sorry. Now I’m self-conscious. Fuck.
No, it’s good. You’re doing good. I just think it’s funny. A million people listen to this guy who just learned about debt this month.
Suppose the interest rates rise 1%. Over a five-year basis, the fraction of tax revenue that goes towards servicing the debt goes from 20% to 25%. If it rises 5 percentage points, that would go north of 40%.
But if you take into account the fact that the government is borrowing $2 trillion every single year, then that goes from 40% to north of 60%. So 60% of tax revenue just goes towards paying interest payments on the debt. Now, I think the US is going to be fine because the tax base will increase if we let data centers get built in America. Other countries are absolutely fucked, in my opinion.
I was just looking at which countries have a lot of debt, have very little tax revenue, and also a lot of their debt is serviced quite often. Those countries, like Pakistan or Nigeria, I think are just going to be very fucked in this new interest-rate regime.
This crowding-out effect is the reason it’s not YOLO 1 billion gigawatts.
You’ve got all these industries and countries that use a lot of debt, all these impoverished countries that you mentioned earlier that are just going to default. You’ve got consumer packaged goods, all of these companies that make things you see at Trader Joe’s or wherever. They use a lot of debt. All these telecom companies use a lot of debt. Banks use a lot of debt.
So if interest rates go up in the market — not necessarily the government-set interest rate, but the spread between what the government says their federal rate is versus what everyone else is charging, because Amazon wants to raise $100 billion of debt next year or whatever the hell the number is, probably less — you end up with this really challenging problem of, where does the cash come from? There is some level that is funded by cash flows and the cash flows keep going up. But the logical thing to do is to invest way more than your cash flows because then the returns in future years will be amazing.
So you have this delta. Then what’s pushing down on the delta is all of these other things: regulations against data centers, consumers getting mad, politicians getting mad, regulations against AI, the AI labs not releasing their latest models because of safety reasons. Interest rates going up are an influence on all of these things. So all of these things bend the curve from what capitalism wants in terms of pure, simple economics to what the complex system that we have wants, and bend it lower and lower to where not as many gigawatts as should be built will be built.
Well, the interest rate is part of capitalism, right?
Yeah, but in the simple economic model versus the more complex what we have.
What is the rate at which you think Amazon or Anthropic or whatever will be issuing bonds for debt next year? If they do hundreds of billions of dollars of debt. What is the average rate?
I don’t think Amazon will do hundreds of billions of dollars of debt.
In total. Let’s say the big tech guys. The hyperscalers in total, and all the clouds… In the modeling that we do, we have about $11 trillion of CapEx from 2024 to 2029.
Total? Total. If you fund a lot of this with cash flows, as much as you can, you still end up with north of $5 trillion of credit that needs to be issued for this $11 trillion-plus build out. So you don’t think the AI revenue continues even 3x-ing year over year? AI revenue does go up.
I don’t think it can go up forever without certain constraints being hit. Labs will have certain incentives. Labs are not the ones building all the compute in many cases, even though they’re increasingly trying to go that way.
But they’ll have all this cash flow. How much did you say the revenue will be? You think they’ll not have that much revenue? No, I’m just saying till 2029 there’s something on the order of $11 trillion of CapEx. $6 trillion of that is funded with cash, and $5 trillion of that is funded with debt. If that’s the case, $5 trillion of debt being raised across the whole ecosystem does make interest rates go up.
Then what prevents that? There’s a couple things. One, do labs increase their revenue per megawatt more and keep inference allocations large? In which case, they’re accumulating all the profit across the S&P 500 because everyone’s paying to reduce their costs. Of course, their profits will also go up, but cash has to come from somewhere.
So there’s an upper limit on how fast their revenue can grow versus the value they deliver into the world. And there’s a diffusion aspect of the technology. But ultimately labs’ revenues keep going up. They can’t cash-flow fund everything.
The optimal scenario is you actually use credit as much as you can to fund, because even if cash flows from the labs fund a lot of stuff, you want to build more than that. So there is some amount of credit that gets built. Our current modeling has $5 trillion of credit and $6 trillion of cash-funded infrastructure investments through ’29.
When you take that, this is not enough compute relative to what the demand growth is from the AI models. So you’ve got the obvious answer, which is revenue per megawatt keeps going up. That makes sense. How much do you think interest rates will increase by 2029 as a result of all this?
Dude, this is vibing a number, but if you’re vibing a number out… Growth in the world economy is going up a lot, so why wouldn’t interest rates for Amazon go up from where they are today?
This is going to be extremely vibed out, but recently Meta’s raised at 5 to 6%. I don’t see why they wouldn’t pay 8%. They would happily pay 8% because the return from the compute that they’re going to build is humongous. The market won’t want them to, but they’ll want to pay 8%.
The flip side is that if they pay 8% versus the 5%, 5.5%, 6% they do today — a 250 bps increase — that makes everyone else in the economy also pay 250 bps more, which then causes a lot of things.
Banks will scream, because if their credit spread goes up, their debt reprices faster than their assets reprice. They ultimately end up losing tons of money if their credit spread blows up. The other consequence of this — this is a point you made — is that if interest rates rise, the discount rate increases, which means that the discounted cash flows of all equities crater. Which means that even though the stock market as a whole might be doing fine — the S&P 500 will be fine — any individual stock will probably have just cratered in value, especially the Buffett, Berkshire type, pay-good-cash-flows-for-30-years type stocks. Yeah. It’s like, "Why would I pay this much for Johnson & Johnson?" They’re seen as a stable stock: good cash flows, they’ll return their cash flows over time. Or a railway company. Why the fuck would I invest that much if my discount rate isn’t 3% or 5%?" It’s now 8% or 10%.
For developing countries… Basil Halperin, who’s a good friend and an economist, made this point that we’ll see a second Volcker shock. In the ’80s, to fight inflation, Fed Chair Paul Volcker raised interest rates more than 5%, something like 8% real interest rate. That caused some 40 different countries, mostly in Latin America, to default in that decade. I think that will probably happen again. Okay, now we’re getting into singularity talk.
We’ve been talking about what happens if interest rates rise—
I think this all happens before singularity, by the way.
Yeah, that’s what I’m saying. We were talking about before singularity, interest rates rise 2-3%, et cetera. At some point, I think it’s very likely that the world economy will be doubling every single year.
This is not happening in five years. But it’ll happen eventually. There’s this researcher, Damon Binder, who’s done great work on this. If you look at input-output tables in a fully automated economy… What would it take to double the entire stock of things in the economy every single year? Yeah. If the economy grows at 3% a year, then rule of 70, that’s 20-something years. Right. But he was like, "Okay, right now we’re bottlenecked by the fact that there’s people, and you can’t double people every single year." But in a world where you can also double the labor force every single year, how fast can the economy grow? I think it could double every single year. At the very least it would be tens of percent every single year.
Okay. The rate of interest should be pretty close to the growth rate. It won’t be exactly that because of consumption, but it should be pretty similar. Then we’ll go into a world, I think in the 2030s, where the rate of interest is tens of percent.
Part of my brain is like, "It might be hundreds of percent," but let’s say it’s at least tens of percent. I’m just like, okay. Every country that is not involved in the production of AI defaults. Every stock that is not an AI stock is worth basically zero because discounted cash flows are worth nothing. If the federal government can’t figure out a way to tax AI, servicing the debt is more than the current tax revenue. And you have all these other effects that I’m sure we’re not even pricing in: you can’t get a mortgage, et cetera, et cetera.
Fundamentally, what is happening in this world? This is all nerd speak, right? But let’s step back. What’s happening? Just now it started, the nerd speak? We’d be entering a totally different growth regime.
The economy’s basically saying, "Hey, the opportunity cost of the government borrowing money to pay people pensions is extremely high now. Because that money could be spent building a robot factory that builds a robot factory that builds a robot factory." The opportunity cost of capital is going to increase a ton. That’s fundamentally the cause of all of these things we’re talking about.
As interest rates go up, equity markets get pummeled. Even AI companies. Some people who really believe in AI are like, "Why does Micron or Hynix or Kioxia trade at 2 or 3 times earnings?" It’s like, "Well, if you’re really AI-pilled, everything in the economy should trade at 2 or 3 times earnings." If you’re not AI-pilled, then sure, they’re over-earning.
It’s an argument for why — I think memory is going to do great — memory stocks shouldn’t 10x or whatever again. Because if we’re in the market where there’s that much demand for memory — which means AI’s caused this drastic change in the economy — then everything should trade at 2 or 3x multiples and the stock market should fucking crash.
In a sense, Meta trading at… I think they’re like a $1.5 trillion company. It’s like, what? Silly. They’re worth way more than that, at least in a logical sense. You just look at their cash flows, all the infrastructure they’re hoarding, and all the compute that they’re going to be able to sell for crazy amounts of dollars per watt, either as tokens because their lab works, or just to Anthropic and OpenAI.
It ultimately becomes a question of, you have to reallocate all the capital to the AGI. You do that by pricing everyone else out. So the limiter on AGI is not how fast the research engineers, like our roommate Sholto, can crank the gears. It’s actually just how much does the rest of the world let that happen? Because they’re going to regulate.
They’re going to obviously increase interest rates. They’re going to say, "No data centers." They’re going to say, "Stop building fabs." They’re going to say, "Oh shit, every company’s equity value is tanking, so how can I pay for AI to increase my business?" Well then, Anthropic and OpenAI have to start building their own stuff. They’re building their own chips already, or at least designing their own chips, and it’ll expand out. They’re contracting their own data centers and building their own infra in the next couple years.
There’s the question of how this reallocation of the economy happens. There’s a lot of downward pressure on it not being just straight takeoff, even if the models were capable of it. I think you and I believe we’re in a world where models are capable of that. But slow takeoff is, at least my hope, possible, because of everything in the economy and regulatory world. Government saying, "Don’t release your models," the government saying, "Actually, you can’t even use your models internally that much," because that’s going to happen soon. They’re already saying you can’t release your models.
The thing I’m most worried about is a singularity, which external deployment is actually helping. So the fact that we’re preventing external deployment is stupid.
Does that prevent singularity? Right now it would lead to more revenue, because the models are incapable of RSI. But I’m worried about a world where it’s 2030 and the government’s like, "We’re going to wait six months before you can release your newest model to the public." Six months, 100x. Let’s go.
In that six months, they do recursive self-improvement internally. They just have all kinds of crazy shit happening in the company. Meanwhile, the rest of us are stuck with models that are, at current pace, years behind.
Here’s my thought. Suppose that the whole world gets in on this conspiracy to try to slow down AI.
I don’t think it’s a conspiracy. It’s outwardly written from every politician.
Suppose they slow down AI by a year. If compute is increasing 2 to 3x every single year, they prevent a whole year of AI deployment such that you’re a year behind where you would otherwise have been. During RSI, you’re getting 3 to 6 years of AI progress in a single year. But they don’t just limit compute.
They also limit the lab’s ability to release the model internally. We saw that. If they did that, that would be ideal. Anthropic had to stop giving Mythos to foreign employees for a bit. I didn’t know that was true, internally as well? That’s what they claimed. I thought that was just a different checkpoint that was not Mythos, but it was basically Mythos. But stuff like that is not going to be allowed either.
The government is dumb, but they’re not that dumb, I would hope, at least. Governments — at least the US government, which has the cards here — are not going to want Anthropic to use Mythos 4 internally. They’re going to be like, "Hold the fuck on. Slow down," because of all of these regulatory reasons.
Everyone who’s elected is going to hate AI. Even the people who are elected already hate AI. All the constituents. I bet you at some point your parents are going to call you and be like, "Dwarkesh beta, you’re doing a terrible job. You’re making AI progress happen faster."
Because of my podcast I’m accelerating AI progress?
Maybe. You educate people. Maybe if they’re smarter, they’re progressing AI faster. Anyway, you’re going to have real-world constraints on the progress and development and deployment of AI. Even though it will happen eventually, we could tear ourselves apart before we get there.
Jane Street is hiring for two separate ML internships right now: one focused on ML engineering and the other focused on ML research. I sat down with Alok, who helps run the research track, to learn more about that program.
I think this domain is fundamentally understudied. Often we have unanswered questions within our deep learning research team where we don't understand, say, some market participants' behaviors or certain dynamics of how trading happens. These unanswered questions make for really good intern projects because they are ultimately topics that we care about and just haven't gotten around to figuring out yet. So even as an intern, you'll be contributing to real research, not working on some sort of contrived exercise.
The Jane Street team follows frontier LLM research closely. A relatively common intern project is adapting a recent paper to financial markets, which come with their own set of gnarly problems. Ultimately, we're trying to model thousands of interconnected irregular time series. The signal-to-noise ratios are extremely low because we have a lot of competitors trying to do the same. So we have this adversarial, non-stationary, extremely high-dimensional problem that we're trying to solve.
To be clear, you don't need to know anything about finance in order to be a good fit. As long as you have a background in ML research, Jane Street can teach you the rest. Their 2027 internship applications are open now. Apply at janestreet.com/dwarkesh.
One thing I find crazy about these scenarios is just how much of the world’s future labor supply ends up in very few companies, and also how fast that labor supply grows year over year. If compute at the frontier in FLOP terms is growing 4-5x a year — and further the compute required to achieve a level of capabilities is decreasing 3x a year — basically the effective AI population size at the frontier labs is increasing 10x year over year.
That doesn’t really matter that much right now, because AIs are not good enough to do full jobs or be as autonomous as people in their capacity to do work or pull off schemes or whatever. But if the current trend continues, you have a world where OpenAI goes from having, say, basically 10 million AI laborers this year to 100 million the next year, to a billion the year after that. Pretty soon, even if compute scaling slows down, it doesn’t take many more years before each company individually has more labor equivalence than there are people on Earth.
I think that’s very plausible by the end of this decade, that there’s more AI labor, more effective population, within a single lab than there are people on Earth.
We talk often about centralization of power because of nationalization or whatever. But we don’t think enough about the fact that we’re actually moving very fast into a regime where most "people", in terms of work output, are concentrated within two labs who are consuming more and more of the world’s compute. If these AIs are misaligned, then most of the world is misaligned, basically, because most of the world’s minds are there. But even if they’re not, very few companies have a lot of influence or a lot of control.
There was the whole spat recently where I think Gavin Baker was like, "Dario believes that there’s only going to be one company in the world." Then Sholto and Dario came out and were like, "No, no, no. We didn’t say that." But ultimately, if you believe in RSI, if you believe the labs are the most effective user of compute and can generate the most value from the compute, then the only thing that’s going to happen is centralization of compute. If you believe in AI researchers, RSI, AGI, then all of this exists, all of this is the base.
This is even true if there’s no RSI. The effective population of the frontier is currently increasing 10x year over year for a given level of capabilities. So if you get to the level of capabilities of a very competent remote worker or a very competent software engineer or a very competent researcher, the population of those is increasing 10x year over year at the current rate of capabilities growth.
I see, and without RSI. Then once you have RSI, it’s even crazier. Then it’s maybe growing 100x a year or 1,000x a year.
Or their intelligence is increasing but the population isn’t increasing. Or some mixture of the two, right?
What world do you see, Dwarkesh, where everything is not centralized? Because it seems to me that every force is screeching towards centralization. And that’s scary as hell. I would love for it not to be centralized completely. But maybe that’s the whole point of a machine that loves grace, right? It is everything and it makes our lives great.
It’s so hard to think about the future. But I agree with you. I think the fundamental problem is that AI training has huge economies of scale, because any effort you spend on training an AI for a specific skill or a specific set of knowledge gets amortized across billions of sessions or billions of users. So that’s one effect. The other effect is that if you’re slightly ahead in the AI race and compute is in shortage, you can charge a much higher markup because you can better economize this scarce resource. So there are two effects which give more and more to the person who’s ahead in the AI race.
There may be more. If models are learning from deployment, and one model is deployed much more widely than another one, it’s getting much more real-world data.
Your point is taken that whether it’s user deployment and continual learning, whether it’s training and having these economies of scale, whether it’s the incremental progress where the best AI model helps you to make the next best AI model, RSI, all of these things point to centralization.
I think one of the big intellectual projects, honestly, that we should spend some time thinking about — or at least I’ll spend some time thinking about — is: what is a vision of a decentralized, broadly empowered future after AGI that takes these economies of scale seriously? The alternative vision is that the government controls it, and maybe you think that you can trust the government more because it’s not a private corporation. I don’t trust the government, and I don’t trust Dario, and I don’t trust Sam. That’s a problem, right? Obviously it’s very easy to be wrong about the future. You don’t anticipate a key effect or something that changes everything. But ex ante, it’s very hard to see how we avoid a scenario where we have to choose one source of centralization.
It’s why capitalism worked, right? It’s decentralized decision-making and decentralized power. And it’s why super-centralized capitalistic economies actually grew slower than super-decentralized capitalist economies, to some extent. You have to have rule of law and all this. But then AI flips all this on its head. And ultimately you’re like, "Actually, private ownership is probably not the most efficient economy, and therefore it grows slower than an AI economy, which is centralized."
Well, it’s still private ownership, but how many firms are really involved in this share of the economy? It’s, what, maybe 2% of the economy right now? $1 trillion divided by 30. Nvidia is a huge share of it, and Anthropic and OpenAI and these hyperscalers. Obviously there are other firms involved, but a large share of the AI stuff is just happening from very few companies. So it could be private property, but very few companies are involved.
I mean, this is what the structure of the market is doing.
So what can prevent it?
I don’t know. Unless AI progress slows down, unless governments regulate the fuck out of it, this is all that happens. In which case, we’re headed for a world where either we have super concentration of resources and we pray that that one company gets everything right, or we have governments slow everything down and people slow everything down, and you have a slowdown of progress somehow hopefully, and there is more of a balance of power. Even as we go towards AGI, ASI, RSI, everything along the way will still lead to someone capturing more resources. So it’s kind of hard to find a framework in which AI doesn’t lead to super concentration.
Now, the one positive thing here is that today Anthropic does not capture most of the value. We can talk all we want about how they went from $20 million per megawatt to $100 million per megawatt, but they’re still paying $13 million for a lot of the compute they’re buying. But at the end of the day, the reason they’ve gone to $100 million per megawatt is because Jane Street is capturing $300 million per megawatt or $500 million per megawatt. Or Dwarkesh, from researching his podcast and learning about credit, is capturing how many dollars per megawatt?
Now how much can you use? Tough. But I think that’s the one saving grace, that the rest of the economy maybe profits so much more from Anthropic—
No, but the whole logic you were laying out earlier — them reallocating inference to AI R&D — the whole logic of that is that the returns to labor inside AI labs are much higher than the returns outside.
Yes. This is my cope. I agree. In all scenarios of the world… There’s 80,000 worlds and in only one of them, Anthropic doesn’t own the whole world. Again, power concentrates because I don’t want to send the tokens outside. They’re more valuable inside. So it’s the same thing. Why would I let Jane Street make all this money off of these degenerate options traders?
Hey, they’re a sponsor, come on. Jesus Christ.
No, I think it’s great. It’s a good value for the world to make it an efficient market. Jane Street making all this money off of getting the worldview correctly, making money off of degenerate options traders, whatever it is, why would Anthropic allocate compute to that? If the end monetization that Jane Street has per megawatt is $200 million, so they’re willing to pay Anthropic $100 million… Well, what if Anthropic can just generate hundreds of millions of dollars per megawatt by using that compute internally? That’s what’s happening.
On that somber note, I guess we’ll meet again when the RSI is officially kicked off.
You’re not going to have me on your podcast again for like two months? Alright, cool. Thanks, dude.
Article published