Andrawes

When your X feed is all compute and commodities, it's time to put that Series 3 to use.

My latest Power Bytes, Future(s) of Compute is my take on whether compute futures can do what commodity markets were designed to do. 🧵

This is how people are quietly making money from the AI boom.

"Your ability to find data center capacity in the United States is an interesting one. As a matter of fact, there's a class of, you know, think about them as Miami-style real estate brokers who just happen to know a bunch of data center providers with a megawatt here, five megawatts there for a small deployment.

These guys are taking a commission on introducing you to their buddy that runs a data center that has some space in him.

They're making a crazy amount of money right now, basically on connecting someone who's looking to host some hardware to a data center in some random location."

How do we know compute is a commodity ?
We track >500 GPU compute providers. Hyper-fragmentation is a hallmark sign of a commodity offering.
@ComputeDesk is making compute tradable.

NEW POD

We Cover
– Compute still trades in group chats
– Why finding GPUs takes so long
– Paying 2x to skip the queue
– Why compute futures > perps
– What’s really bottlenecking AI?

@BrettHarrison @andrawesbahou @JasonYanowitz

TIMESTAMPS:
00:00 Intro
00:59 Why Does Compute Need Markets?
08:53 Compute Still Trades In Group Chats
13:38 Why Finding GPUs Takes So Long
19:21 How Much Compute Should You Buy?
26:58 Why Are Compute Prices Rising?
33:27 Can Architect Take On CME?
38:13 How Do You Hedge Compute?
43:08 The Risks Behind GPU Lending
52:08 What Really Constrains The AI Market?
01:02:08 What Is Everyone Missing About Compute?

This bad boy is gonna fit so many sandboxes in it

Beautiful piece of technology

On the Empire Pod set today talking compute markets with @JasonYanowitz and Compute Desk's @andrawesbahou!

The economics of a Neolab.

A neolab is loosely defined as a startup of AI researchers who raises a lot of money pre-production to be able to finance GPU compute to take on a large AI problem.

To buy 1000 GB300s or ~14 NVL72 racks will set you back $125-150M for 3yrs with 15-30% upfront. That’s about ~2-2.5MW.

Thats about enough to do 10^25 flops a quarter and get to a GPT-4 level model which is 1-2 OOMs off frontier for pretraining.

If you post-train on a great open source model, you have a better chance of getting to frontier. The risks are a) you need to spend millions on RL environments too and b) being lapped by another model release while being tied to a base model.

For this to payback, you need to give your customers a better and ideally cheaper inference service than a base model and serve them for long enough to recoup your large investment. Even at 50% margin on inference, to recoup $10M in training means serving ~10T tokens (!) if you price like Fable / Astra given a standard cache read / input / output split ($2/M blended). And you have to justify being better than a release like Opus 5.5 which is even cheaper. Often, you end up charging your customers a huge premium in terms of platform fees and compute fees on top of pure inference.

Meanwhile, every hour you’re not utilizing your GPUs you are burning money so you typically resell this compute back to a broker or run inference for open models / resell spot instances. At below a ~60% utilization on spot, you will still lose money.

Add to that insane cost of talent.

So what can you do with the compute?
- Not play the model game at all.
- Play an entirely different model game (Jev, World Labs) that if big labs played, would either a) cannibalize their business or b) be incrementally not significant revenue c) would cause too much distraction from the main main thing
- Acquire a proprietary data set (Peridodic Labs) in enough volume in a domain of usefulness to eclipse frontier quality. Often happens in robotics, biology, chemistry.

If you do overcome the challenge of building a model that is useful and well priced beyond big labs models, given the huge price of compute, you still need to play in an area where the revenue / compute ratio is signficant and market demand is large enough to payback your compute spend.

It is a difficult game.

Our latest research paper explores the growing connection between AI and digital assets and explains why broad AI adoption may drive new demand, utility and applications across the digital asset economy. blackrock.com/us/individual/…

“We track over 1000 different oil prices” @neilbradford . Who still thinks oil is a homogeneous commodity?

Thanks @Google for having @ComputeDesk × @NodalExchange. Hyperscalers can soon trade and hedge compute, stay tuned.

Everyone is talking about turning compute into a commodity. I think we’re skipping the more important question: what exactly are we financializing? x.com/i/article/2097…

1/ A common way to build a GPU forward curve: take the 6-month reserved rate and the 12-month rate, subtract the six months already priced, call the remainder the price for months 7 to 12. That subtraction doesn't work.

I'm a skeptic of GPU indices and compute futures. We haven't seen digital infra trade like a commodity yet, and I'd hate to see these go the way of bandwidth futures in the 90s.

The indices are meaningfully off of actual clearing prices because hyperscaler and lab purchases are highly confidential deal terms.

Fungibility is great in theory not in practice. Any hedge fund / scale user knows the value of a cluster has too many dimensions to reach fungibility from cluster to cluster: specific fiber paths, shell, geo, BOM, the 10 different operators in the supply chain, and a thousand+ more variables.

If it works, we'll surely use it but until then we're building something else

who's willing to take compute financing risk?

> commodities/energy traders
> hedge funds
> family offices

trad private credit funds r way more risk averse

compute has volatile residual values, merchant exposure, technological obsolescence, and weak historical recovery data - which are exactly the risks most credit funds are built to avoid

capital comes from investors used to pricing volatility, not lenders trying to eliminate it 🧵

all paths lead to becoming a neocloud

neolab, some AI product company, RL service, etc. we saw this again this week with Gimlet !!

"neocloud" means lots of things now:
- rent out their own gpus: @CoreWeave, @nebiusai , @LambdaAPI
- started closer to power and data centers: @CrusoeAI , @IREN_Ltd , @nscale
- gather compute from different providers: @runpod , @vast_ai, @akashnet
- do research/RL and also sell compute: @PrimeIntellect , @togethercompute
- built their own chips and now sell inference: @GroqLLC, @cerebras
- run inference across different chips: @gimletlabs

why though? because AI companies need more and more compute. renting it gets expensive so long-term contracts make sense. then they get their own gpus but they need an infra team or figure out financing it to keep it busy. of course they'd start selling their compute or managed workloads 🤷‍♀️

Compute Desk is partnering with @NodalExchange to launch compute futures settled against our compute indexes.

With the compute world’s nexus to electricity and power, Nodal is an exceptional venue for AI infrastructure companies, hyperscalers and financial institutions looking to hedge or gain exposure to compute, power or the spread between them.

Reach out to @ComputeDesk to trade or hedge compute.

H/T @ForgashEmily Bloomberg: bloomberg.com/news/articles/…