Dour warnings of an AI bubble have rocked markets in latest weeks. Not less than one large concern is misplaced, although.
Again in March, I informed you about depreciation risks for some AI firms, together with CoreWeave. In August, Jim Chanos, the man who shorted Enron, shared similar concerns.
The massive fear facilities on GPUs, the chips wanted to coach and run AI fashions. As new GPUs come out, older ones get much less beneficial, via obsolescence and put on and tear. Cloud firms should use depreciation to cut back the worth of those property over a interval that displays actuality. The sooner the depreciation, the larger the hit to earnings.
Buyers have begun to fret that GPUs solely have helpful lives of 1 or two years, whereas cloud suppliers depreciate the worth of those property over 5 or 6 years. An accounting mismatch like this might set the AI trade up for a nasty earnings hit in a couple of years.
This view has turn into virtually a consensus on Wall Avenue now. It is one of many fundamental items of proof for the argument that we’re in an enormous AI bubble. The issue is that it is flawed: Whilst Nvidia rolls out new GPU architectures each 18 months or much less, GPUs aren’t getting older out almost as quick as some traders concern.
“GPUs can profitably run for about 6 years,” Stacy Rasgon, a number one chip analyst at Bernstein, wrote in a analysis report on Monday. “The depreciation accounting of most main hyperscalers is affordable.”
Wholesome margins
The price of working a GPU in an AI knowledge heart is “very low” in comparison with market costs for renting GPUs through the cloud. That makes the “contribution margins” of operating previous GPUs for longer fairly excessive, Rasgon and his fellow analyst at Bernstein famous. (Contribution margins measure income left over after variable prices. It is a frequent manner product profitability is assessed and enterprise choices are made).
“Even with significant enhancements in value/efficiency with every GPU technology, distributors could make snug margins on 5-year-old A100s, in flip implying a 5-6 12 months depreciation lifespan is affordable,” the analysts added, referring to Nvidia’s A100 chips, which got here out in 2020.
Seven to eight years
To search out out why these GPUs are so beneficial for thus lengthy, it pays to talk with the individuals who truly run these elements at scale inside AI datacenters.
Matt Rowe, senior director of strategic enterprise improvement at AI cloud supplier Lambda, mentioned just lately that the efficient lifespan of GPUs can stretch to seven or eight years.
Whereas most corporations nonetheless use a six-year depreciation schedule for accounting functions, guarantee extensions and redeployment methods are extending their helpful life, he informed Bernstein.
Guarantee contracts are sometimes ignored by observers worrying about depreciation, Rowe defined. These warranties usually final 5 years, so if GPUs fail, they’re changed with new ones, extending the lifetime of the general GPU fleet.
He additionally famous that Amazon Net Providers supplied very early generations of GPUs, corresponding to K80s, P100s, and V100s. These all lasted effectively past six years.
Nvidia’s H100 GPUs, which debuted in 2022, are nonetheless operating effectively inside Lambda knowledge facilities. Utilization is above 85% and Lambda hasn’t lower its on-demand public cloud pricing for this GPU in additional than 12 months, Rowe famous.
“All of us suppose seven to eight years is feasible,” Rowe mentioned.
Crusoe’s expertise
I chatted this week with Erwan Menard, SVP of product administration at Crusoe, which is growing the large Stargate knowledge heart complicated in Texas. Earlier than becoming a member of Crusoe, Menard helped construct Google’s Vertex AI cloud service, so he is an actual hands-on skilled.
Menard described a lifecycle the place GPUs migrate from cutting-edge AI mannequin coaching jobs to much less demanding inference workloads.
When creating a brand new state-of-the-art mannequin, you want the newest and best GPU from Nvidia.
Then, it’s a must to run these high fashions, a course of referred to as inference. That requires highly effective GPUs, however not the newest ones.
Past that, there are literally thousands of completely different, beneficial AI workloads that may run effectively on older GPUs, in line with Menard. Which means there are various GPUs which are a number of years previous in Crusoe’s fleet and are nonetheless actively used and worthwhile.
“As a result of there’s a big range of fashions to resolve many various issues, there’s lots of room to make use of GPUs for a very long time, simply transitioning them from one sort of job to the subsequent,” Menard informed me. “It is truly a extensively accepted view within the trade.”
Free versus paid
AI cloud firms think about person expectations and finances to assist them determine which GPUs to make use of. For instance, Menard described an instance of an AI service that has a free tier and a paid model.
“You could determine that for the freemium model you are going to use an AI mannequin that may be inferenced on older, cheaper {hardware} with decrease efficiency,” he mentioned.
That is doubtless ok to create an preliminary expertise for customers. Then, some clients may migrate to the paid model. At that time, you faucet right into a extra highly effective AI mannequin that requires newer GPUs to ship a superior person expertise.
“We see lots of these alternatives,” Menard mentioned. “Not every thing is a nail requiring one single mega-model operating on the newest and best GPU.”
Open-source + older GPUs
Some AI companies are much less compute-intensive and could be run on open-source fashions, corresponding to Alibaba’s Qwen, DeepSeek, or Meta’s Llama choices. One instance is speech-to-text companies (such because the transcription service I used to transcribe my interview with Menard).
Older or less-capable fashions could be run on older GPUs, whereas nonetheless offering beneficial intelligence for AI companies that clients can pay for. (Enterprise Insider pays for these transcriptions, as an illustration).
As extra startups embrace cheaper open-source fashions, older GPUs may truly be used much more. “An open mannequin could also be completely nice and provides a extra cost-competitive construction,” Menard mentioned.
Older GPUs are cheaper
Older GPUs use extra vitality to provide the identical quantity of intelligence, so one other investor concern is that newer GPUs will all the time be most well-liked—aggravating this depreciation drawback.
That is truly not true both, in line with Menard. Older GPUs are cheaper to purchase, so the truth that they eat extra vitality does not change the truth that older GPUs are sometimes cheaper to run, when all prices are taken under consideration.
“The driving force for a given GPU goes to be price, at the start,” he defined. “So we go to the older ones as a result of they’re cheaper.”
What’s an L40?
So, I requested Menard for an instance of an previous GPU that Crusoe makes use of. He described new modular knowledge facilities Crusoe has constructed which are powered by recycled EV batteries from the startup Redwood Supplies.
“I can put L40s from Nvidia in these knowledge facilities,” Menard mentioned. “As a result of the entire deployment is energy-first in its design, I am going to have the ability to make an affect.”
I hadn’t heard of L40s and needed to ask him what they had been.
“That is an previous GPU,” he mentioned, laughing.
Join BI’s Tech Memo e-newsletter here. Attain out to me through electronic mail at abarr@businessinsider.com.
