Why Smarter AI Could Make Compute Up to 10x More Expensive
Dwarkesh Patel
In this narrated version of a blog post, Dwarkesh asks what the compute situation for frontier AI labs will look like over the next few years if current revenue trends continue. The argument is that lab revenue is growing much faster than lab compute. If that continues, the difference has to show up somewhere, and the most plausible place, in the speaker's view, is a sharp rise in the price of compute itself.
The gap between 10x revenue and 3x compute
The starting point is Anthropic's revenue. According to the speaker, it has grown 10x year over year for three consecutive years and is likely to do so again this year. Anthropic ended last year with $9 billion in revenue, and the speaker expects it to end this year somewhere between $100 billion and $150 billion. If the trend held, Anthropic would need $1 trillion in revenue by the end of next year.
Dwarkesh stresses that nothing guarantees this. The conclusion is "very wild," and whether it happens depends on whether AI actually becomes that useful by the end of next year. The essay is a thought experiment: suppose the trend does continue, and work out what follows.
The second trend is that lab compute grows only about 3x per year. For revenue to keep growing 10x while compute grows 3x, the speaker says at least one of three things has to happen:
- Lab margins increase.
- The price of compute increases.
- Labs spend a larger share of their compute on inference rather than training.
All three are already happening
The speaker believes all three are already underway and gives an example for each.
- Margins: Anthropic's inference margins reportedly rose from about 40% in the middle of last year to upwards of 80% now for Fable.
- Compute prices: spot prices are more than 40% higher than at the trough in February of this year.
- Inference share: according to Epoch, OpenAI spent only about a quarter of its compute on inference in 2024. The speaker thinks that figure is now likely closer to 50%, if not higher.
Why labs resist shifting compute to inference
Dwarkesh argues that labs would rather not pull the third lever. As the labs see it, inference revenue exists mainly to convince investors to fund training of the next, bigger model. A lab that spends most of its compute on inference is effectively declaring that AI progress has stalled and that it has become a cloud provider. That is a less compelling business than building AGI.
The labs also don't believe they are in that world. They expect that within a year they will have models that make today's models look "extremely shitty." Building those models takes the majority of their compute for training and experiments. The speaker therefore sets this option aside, which leaves two ways to close the gap. Either lab margins rise and the labs capture the surplus, or compute prices rise and everyone in the stack below the labs captures it.
Why margins above 90% seem implausible
The speaker says it is unclear which world we end up in. If the margin effect dominates, top models would go from roughly 80% margins to above 90%. Dwarkesh doubts this. In a market economy, margins exist because what you sell is much better than what a customer could buy from someone else. Margins above 90% would require the leading model to be very far ahead of the competition. The speaker finds it "really wild" to imagine margins on something like intelligence staying above 90% without being competed away.
That leaves rising compute prices as the main escape valve. Dwarkesh notes that this is already happening, and that the effect is stronger for the kind of compute frontier labs actually need. They can't just buy spot instances. They need enough scale for good efficiency and flexibility, and they need compute that meets the security requirements for their own model weights and their customers' data.
Case study: the SpaceX compute deals
The speaker points to compute that Google and Anthropic are renting from SpaceX. According to the essay, Google pays $900 million a month for about 110,000 GPUs, a mix of GB200s and GB300s. Dwarkesh says this works out to roughly 2x the spot price per GPU-hour. That spot price is itself more than 40% above its February level.
The core claim: smarter models monetize the same compute better
Dwarkesh calls this the key conclusion: as AI models get smarter, they can extract more revenue from the same amount of compute. The illustration is a truly human-level software engineer running on one H100-equivalent. At today's software engineer salaries, that H100 should rent for over $250,000 a year. The speaker says that is more than 15x the current H100 spot price, before counting the fact that an AI can work nights and weekends.
The obvious objection is that ten million extra software engineers appearing in the economy would lower the marginal value of a software engineer, so the H100's earnings would not really be 15x higher. Dwarkesh isn't sure this holds. Applied to people, the argument would be the classic lump-of-labor fallacy. The speaker notes that economists generally believe high-skill immigration does not lower wages in the long run, because innovation and specialization raise the value of labor. This supply shock might be so big and so fast that the heuristic breaks down. But if standard economics holds, the speaker argues, the marginal value of labor, and therefore of compute, should stay "astonishingly high."
What changes in that world
The speaker draws three implications.
Competition gets harder. If the top labs keep getting better at monetizing compute while compute gets more expensive, other players have to bid for the same resource against someone who can use it more productively. That makes it harder to compete with the leaders.
Efficient models command a premium. Dwarkesh calls this the most interesting implication of the exercise and ties it to the Alchian-Allen effect. If renting an H100 costs $20 an hour, using a weaker, less efficient model would be "extremely stupid," because it burns more tokens on expensive compute to reach the same result. A lab that trains a model that uses this scarce input more sparingly can charge a much larger premium. Getting the same result with less compute is, in a sense, creating more compute, and that compute is becoming more valuable.
Cheap uses get priced out. The speaker suggests many popular AI applications will likely become unaffordable. AI is relatively cheap today because it can't yet do much of what top humans can do. Once that changes, Google, Anthropic, or OpenAI will pay more for tokens to automate AI research than ordinary users will pay to produce what Dwarkesh calls "AI slop."
Is this just another wrong scarcity prediction?
Dwarkesh admits the analysis resembles past predictions of scarcity that turned out wrong. The example is the Simon–Ehrlich bet. Paul Ehrlich, a well-known pessimist about population growth, bet that a basket of commodities would rise in price over the decade before 1990. The bet is usually cited to show that Ehrlich's Malthusian view underestimated how market signals and human ingenuity find ways to economize on scarce inputs.
The speaker thinks the analogy probably doesn't hold, for two reasons. First, other analysis has shown that Ehrlich might have won if the bet had covered a different decade. Second, and more generally, the speaker argues that compute supply is much less elastic than metal extraction. It is less able to absorb large demand shocks and less able to be replaced with substitutes.
Why 3x compute growth is hard to speed up, or even sustain
To explain why the 3x annual growth in compute is hard to raise, Dwarkesh breaks it into three components and argues none can be much accelerated:
- About 1.4x from Moore's Law. Far from speeding up, the speaker says it will be "a miracle" if it keeps going for a few more years.
- About 1.2x from new fabs. This is bottlenecked, until 2030 and possibly beyond, by how fast ASML can build new EUV machines. The speaker points to a podcast conversation with Dylan a few months earlier that covered this in detail.
- About 1.8x from AI taking wafer allocation from smartphones and PCs. The speaker expects this to hit a wall by the end of next year. At TSMC's leading-edge N3 nodes, AI's share will have gone from 60% to 86%. Once AI has absorbed essentially all leading-edge wafer capacity, this number can't keep rising.
Given these limits, Dwarkesh says it is unclear how compute scaling can even continue at 3x per year for the next few years, let alone go faster.
A pre-singularity regime
The speaker clarifies that compute will eventually get cheap again. At some point, robots may be able to turn silica sand and copper directly into chips, and the price of compute will then be roughly the cost of raw materials and processing tools. The argument applies only to the current "pre-singularity regime," in which AI compute grows merely 3x per year. In the speaker's view, that is not enough to offset how quickly AI is becoming more valuable.
Economies of scale and a worry about concentration
Dwarkesh ends with one more observation. The fact that Anthropic's revenue has grown 10x a year while its compute has grown only 3x shows, in the speaker's view, how strong the economies of scale are in the model business. That makes sense to the speaker: training a model is a one-time cost of learning many skills, which are then shared across all users. Human labor is different, because each person has to be trained from scratch.
The speaker says they wish intelligence did not have such strong economies of scale, because of concerns about power concentration, but concludes that it seems to.
Today I want to talk about what the compute situation for the labs will look like over the next few years.
For the last three consecutive years, Anthropic's revenue has 10x'd year over year, and it's likely to do so again this year. They ended last year with nine billion in revenue. I think they'll probably end this year with somewhere between one hundred billion to one hundred and fifty billion dollars in revenue. Now, for this trend to continue, Anthropic would need to make one trillion dollars in revenue by the end of next year. Of course, there's no deep reason why this has to be true. It's a very wild conclusion, and it's ultimately a question of AI capabilities. Does AI get that useful by the end of next year? But suppose the trend does continue. I want to think through what happens in that world.
The other big trend in AI is that lab compute only 3x's year over year. For a lab to keep 10x'ing revenue year over year while compute only 3x's, one of the following three things needs to happen, or some combination of the three: One, lab margins have to increase. Two, the price of compute has to increase. Or three, the percentage of compute that labs spend on inference rather than training has to increase.
My understanding is that basically all three of these things are already happening. With regards to the margins, Anthropic's inference margins reportedly went from forty percent in the middle of last year to upwards of eighty percent now for Fable. With regards to compute, the spot prices for compute are more than forty percent higher than they were in the February trough that we had earlier this year. And with regards to the share of compute that goes to training versus inference, in 2024, according to Epoch, OpenAI was spending just a quarter of its compute on inference, and that number is likely closer to fifty percent, if not higher, now.
Now, labs would prefer not to do this final thing of increasing the share of compute they spend on inference. The way the labs see the world, the whole point of inference revenue is to help convince investors to give you more money in order to train the next bigger, better model. And if you're spending most of your compute on inference, then you're basically declaring that AI progress has stalled and you're just now in the business of being a cloud provider. This is a less compelling business than building AGI, so the labs do not want to be in this business, nor do they think they are in this world. They think that within a year, they'll have built models that make the current ones look extremely shitty. But they need to invest a lot of their compute — the majority of their compute — into doing the training and experiments that are necessary to build the next model.
So that leaves only two options for how you can get out of this gap between the fact that lab compute only increases 3x year over year, but revenue increases 10x. Either the lab's margins have to increase so that they get the surplus, or the price of compute has to increase so that everybody in the stack below the lab gets the surplus.
It's not clear to me which world we end up in. Do we end up in a world where we go from 80% for some of the top models to greater than 90% margins if the lab margin effect dominates? That would require the leading model to be so far ahead of the competition, because the nature of margins — why they exist in a market economy — is that the thing you are serving is so much better than what somebody else could go get and replace you on the market. But it's just really wild for me to consider that the margins for something like intelligence will be greater than 90% and they don't get competed away at that level.
So that leaves only one other possibility of this escape valve between these two trends, which is that the price of compute has to increase. As I mentioned, this is already starting to happen. And the effect is even stronger when you look at the tranche of compute that the frontier labs actually need to accumulate, because they can't just go out and buy a spot instance. They need to make sure that they get enough scale to get really good efficiency and flexibility, and also that they have the kind of compute that lends itself to the security they need for their own weights and for their customers' information.
I think a relevant case study here is to look at the compute that Google and Anthropic are renting from SpaceX. Google, for example, is paying nine hundred million dollars a month for a hundred and ten thousand GPUs that are a blend of GB200s and GB300s. The price that Google is paying here is 2x the spot price per hour for those GPUs. And that spot price itself is more than forty percent higher than it would have been in February.
I want to emphasize a key conclusion here: as AI models get smarter, they will be better able to monetize the same amount of compute. If a true human-level software engineer could run on an H100 equivalent, then at today's prices for software engineers, that H100 should rent for over 250K a year. That's over 15x the current spot price for an H100. And this is not even accounting for the fact that your AI can work nights and weekends.
Of course, you might expect that if we had ten million extra software engineers suddenly appear in the economy, the marginal value of a software engineer would decrease, and thus the revenue that that H100 would be able to generate would not be 15x higher than it is right now. But I actually don't know if this is true. If we apply this argument to people instead of AIs, then this would be the classic lump of labor fallacy. For example, economists generally believe that high-skill immigration does not decrease wages in the long run because of how innovation and specialization increase the value of labor. Maybe this labor supply shock will be so big and so fast that we can't count on this general heuristic anymore. But if you believe what standard economics says, then the marginal value of labor, and thus the marginal value of compute, should stay astonishingly high.
So let's think about what changes in such a world. One of the things that would happen is that as the top labs get better and better at monetizing compute, and the cost of compute increases, it becomes harder for anybody else to compete against them, because they have to bid for this resource against somebody who is basically able to make better use of it.
Another thing that will happen — and I think this is actually the most interesting implication of this whole thought exercise — is that if you can train the best, most efficient model, then you'll be able to charge much higher margins than you can today. This is the Alchian-Allen effect in economics, and what it's basically saying is that if it costs twenty dollars an hour to rent an H100, then it would be extremely stupid to use a weaker, less efficient model, because it's gonna burn more tokens on your expensive compute to get the exact same result. So labs will be able to charge a much larger premium if they can train a model that better economizes this scarce input. Basically, if you have a model that can get the same result by using less compute, then you've, in some sense, created more compute, and the value of compute is gonna increase.
Another thing that will happen is that a lot of current popular applications of AI will probably get priced out. The reason AI is relatively cheap right now is that AI just can't do a lot of things that top humans can do. But this, at some point, will no longer be the case. And at that point, Google or Anthropic or OpenAI will be willing to pay more for the tokens to automate AI research than you or I will be willing to pay to make more AI slop talk.
I'm a bit worried that this kind of analysis honestly pattern matches a lot onto the ways that people in the past have been wrong about scarcity. I'm thinking, for example, of the famous Simon-Ehrlich bet. Paul Ehrlich was this famous doomer about population growth, and he made this bet that a basket of commodities would increase in price rather than decrease in the decade preceding 1990. This is a very famous bet because it's supposed to illustrate how Ehrlich's Malthusian worldview was wrong, and how he did not anticipate the way in which market signals and human ingenuity can find better ways to economize scarce inputs.
I'm guessing that the analogy to this bet is probably wrong. Other analysis has shown that if that bet had been made in a different decade, Ehrlich might well have won. But more generally, I think the supply of compute is much less elastic, much less capable of absorbing large demand shocks, and much less capable of being accommodated by using different substitutes than the extraction of different metals is.
To illustrate why I think this 3x in compute capacity year over year is hard to budge or potentially even sustain: I don't see how any of the three elements that constitute that 3x can be much accelerated. 1.4x of that is coming from Moore's Law. Far from increasing it, I think it'll be a miracle if we can just keep it going for a few more years. 1.2x is coming from building new fabs. This process is ultimately gonna be bottlenecked up to 2030 and potentially even beyond by just building new ASML EUV machines. Dylan, when he was on the podcast a few months ago, talked about this in great detail. And 1.8x comes from the fact that AI is absorbing a lot of wafer allocation that was previously going to smartphones and PCs. This is probably gonna hit a wall by the end of next year, when at the leading edge N3 nodes at TSMC, AI will have gone from 60% to 86%. At some point, you have just absorbed all leading-edge wafer capacity for AI, and you can't keep increasing this number. So I don't know how we even continue to do 3x compute scaling year over year for the next few years, much less go beyond that.
At the end of the month, I go through the time-honored tradition of closing my books. I start by opening Mercury, which is my banking platform, to make sure that all my transactions are properly categorized. Auto-categorization rules handle the predictable stuff pretty well. But I'm constantly working with new contractors—tutors, researchers, and videographers—and I'm also trying new tools. Manually categorizing all of these transactions would add a couple of hours of overhead every single month.
So instead of going through them one by one, I have Command, Mercury's built-in AI, take a stab at all of them at once. Command proposes a category for each transaction and provides its rationale. I just review, fix anything that's off, and approve. Once all this work is done in Mercury, it syncs everything with QuickBooks. And Command's judgment calls are genuinely good. It does the obvious things, like looking at the vendor, but it also investigates who on my team made the purchase and looks at notes and memos to build up as much context as possible. This is just one of the ways you can use Command to automate the back end of your business. To learn more, go to mercury.com/command.
Mercury is a fintech company, not an FDIC-insured bank. Banking services are provided through Choice Financial Group and Column N.A., Members FDIC. AI-generated responses and suggested actions may vary and are not guaranteed.
Now, I wanna clarify that at some point in the future, compute will get cheap again. At some point, we'll just have robots that can convert shores of silica sand and mines of copper into new computer chips, and then the price of compute is basically the raw inputs and the tools required to do this processing. I'm just talking about this current pre-singularity regime where AI compute merely 3x's year over year, which is not enough to offset how much more valuable AI is becoming over time.
By the way, the fact that Anthropic's revenue has been 10x'ing year over year, whereas their compute has only been 3x'ing year over year, I think illustrates how strong the economies of scale are in the model business. And logically, this makes sense. When you train a model, you just have to spend this one-time cost to learn all these different skills that then get to be shared across all your users. This is very unlike human labor, where each instance has to be retrained from scratch. I wish we didn't live in a world with such strong economies of scale for intelligence, because I'm worried about power concentration, but it seems we do.
Okay, this was a narration of a blog post that I also released on my website at dwarkesh.com. Check it out for other posts or to be notified when I release a post in the future. Otherwise, I'll see you for the next full episode.
Article published