1
00:00:01,761 --> 00:00:03,981
All right, I think we can get started here.

2
00:00:04,701 --> 00:00:05,601
Hey, everyone.

3
00:00:05,841 --> 00:00:10,601
My name is Affaan, and I'm a lead for compute trading at Jump Trading, which is a proprietary

4
00:00:10,601 --> 00:00:13,761
trading firm, alongside my co-lead over here, Alejandro.

5
00:00:15,121 --> 00:00:20,221
So today we're going to be looking over some of the market structure as it applies to compute

6
00:00:20,221 --> 00:00:22,261
trading, what compute trading even is.

7
00:00:23,781 --> 00:00:25,721
what we can see it becoming.

8
00:00:26,881 --> 00:00:28,801
And I think we'll start with some macro,

9
00:00:29,201 --> 00:00:30,241
look at microstructure,

10
00:00:30,441 --> 00:00:32,181
and then talk a little bit about

11
00:00:32,181 --> 00:00:33,681
where we see things in policy.

12
00:00:41,901 --> 00:00:44,181
So the title of the talk is

13
00:00:44,181 --> 00:00:45,241
The Dark Pools of Compute,

14
00:00:45,341 --> 00:00:46,861
but some of you guys are in finance,

15
00:00:46,901 --> 00:00:49,261
so you might be confused as to why I'm using this terminology.

16
00:00:49,801 --> 00:00:51,461
What do I mean by dark pool here?

17
00:00:51,761 --> 00:00:52,981
Essentially what I'm referring to

18
00:00:52,981 --> 00:00:59,341
is an unlit market. Right now, most compute trades in blocks, physical compute here. And

19
00:00:59,341 --> 00:01:03,181
these are private quotes, we have a lot of brokers, and these are long bilateral contracts.

20
00:01:03,781 --> 00:01:08,321
Now, at the end of the day, these are really commercial agreements. So we're using the

21
00:01:08,321 --> 00:01:12,541
term trading here to kind of like look at how it's transacted between parties. These

22
00:01:12,541 --> 00:01:18,801
commercial agreements tend to be some mixture of an MSA, an SLA, and an order form, cluster

23
00:01:18,801 --> 00:01:25,141
specific with hardware specific language in each order.

24
00:01:26,481 --> 00:01:30,741
But when we're looking at pricing and looking at some of these other metrics, a lot of the

25
00:01:30,741 --> 00:01:35,321
time most of the talk is around the SKU, so what the chip itself is, the baseboard.

26
00:01:35,501 --> 00:01:38,701
So H100, H200, V300, etc.

27
00:01:39,881 --> 00:01:45,281
And so a lot of the negotiation here happens kind of outside the public view.

28
00:01:45,281 --> 00:01:55,341
There's no exchange for physically traded compute that's widely adopted and used for everyone goes there to take a look and trade compute.

29
00:01:55,581 --> 00:01:57,721
These are done in chats.

30
00:01:58,101 --> 00:01:59,581
They're out of the public view.

31
00:01:59,881 --> 00:02:01,361
There's public marketplaces.

32
00:02:01,581 --> 00:02:02,361
There's price references.

33
00:02:02,501 --> 00:02:04,261
There's a lot of on-demand

34
00:02:04,581 --> 00:02:11,061
places. However, for these actual tenured, so on a given term for a fixed amount of capacity

35
00:02:11,061 --> 00:02:17,981
with a certain total contract value, these trade over Slack messages, X messages, Telegram group

36
00:02:17,981 --> 00:02:22,741
chats, so a bunch of OTC kind of channels that are all over in these kind of, I want to call them

37
00:02:22,741 --> 00:02:28,361
like Neo chat. And some of you might be familiar with commodities and how a lot of, you know,

38
00:02:28,361 --> 00:02:31,501
There's chatting happening on Ice Chat, Bloomberg Chat, etc.

39
00:02:32,141 --> 00:02:36,201
This is kind of where the markets here lie.

40
00:02:37,221 --> 00:02:40,881
And so I want to look a little bit here on the macro market structure

41
00:02:40,881 --> 00:02:42,841
and a little bit on the players here as well.

42
00:02:43,161 --> 00:02:47,161
And so this was a recent kind of A16Z.

43
00:02:47,341 --> 00:02:50,001
Some of you guys might have seen this, so I'm putting this out there

44
00:02:50,001 --> 00:02:53,981
because you can go on A16Z's actual X and take a look at this graph yourself.

45
00:02:54,281 --> 00:02:56,861
What we're looking at is that the cost of a given capability

46
00:02:56,861 --> 00:02:58,361
has fallen sharply.

47
00:02:59,321 --> 00:03:02,361
The bill, of course, still depends on the model usage, capacity,

48
00:03:02,581 --> 00:03:04,741
and other factors, such as speed,

49
00:03:04,921 --> 00:03:07,821
is a big thing right now with the new ChatGPT models.

50
00:03:09,601 --> 00:03:11,401
Cheaper tokens unlock more usage,

51
00:03:11,421 --> 00:03:13,601
and a team can do more work for the same budget.

52
00:03:13,801 --> 00:03:16,201
So what we're seeing is that a lot of teams

53
00:03:16,201 --> 00:03:18,001
are simply just using way more tokens.

54
00:03:18,201 --> 00:03:20,681
So we have this latent demand that hasn't really been hit yet,

55
00:03:20,701 --> 00:03:23,581
and people are just going to use more with the same budget

56
00:03:23,581 --> 00:03:26,441
that they were using before when models cost more.

57
00:03:26,861 --> 00:03:28,441
and they're just spending more tokens.

58
00:03:29,181 --> 00:03:31,941
So applications that were too expensive become possible.

59
00:03:32,501 --> 00:03:34,681
However, as we're kind of talking about compute here,

60
00:03:34,961 --> 00:03:37,781
the capacity underneath still has to be built and paid for,

61
00:03:38,341 --> 00:03:40,561
even as the cost of a particular task comes down.

62
00:03:41,001 --> 00:03:44,081
And so the important distinction is between the price of the intelligence,

63
00:03:44,261 --> 00:03:45,361
the number of tasks people want to run,

64
00:03:45,421 --> 00:03:47,041
and the amount of hardware needed to serve them.

65
00:03:50,821 --> 00:03:53,861
And so this is where we get to kind of the Javon's paradox

66
00:03:53,861 --> 00:03:56,601
and a little bit of some of the stuff that we've been looking at lately.

67
00:03:56,861 --> 00:03:59,821
where we had a very interesting thing where, you know,

68
00:04:00,021 --> 00:04:03,541
intuitively you'd assume that compute and inference

69
00:04:03,541 --> 00:04:07,081
kind of like rises together, correlates very, you know,

70
00:04:07,421 --> 00:04:08,501
very closely, as you can see.

71
00:04:08,821 --> 00:04:11,021
You can see there's a little bit of a decoupling that happened,

72
00:04:11,561 --> 00:04:13,221
and token prices declined further,

73
00:04:13,341 --> 00:04:15,101
but compute prices remain robust.

74
00:04:15,261 --> 00:04:17,121
And so, like, what can be an explanation for this now?

75
00:04:17,401 --> 00:04:20,101
There's a lot of, like, potential explanations.

76
00:04:20,681 --> 00:04:22,921
One here is the whole Javon's paradox,

77
00:04:23,101 --> 00:04:25,541
is that even though token prices and inference

78
00:04:25,541 --> 00:04:27,301
is getting extremely efficient to serve

79
00:04:27,301 --> 00:04:30,261
on these new baseboards and GPUs,

80
00:04:30,361 --> 00:04:31,821
people are just consuming a lot more.

81
00:04:32,241 --> 00:04:34,701
And so the demand for compute rises

82
00:04:34,701 --> 00:04:36,921
and people just consume a lot more tokens,

83
00:04:36,941 --> 00:04:41,681
even if the cost per million for input and for caching

84
00:04:41,681 --> 00:04:43,921
and for output becomes a lot more efficient.

85
00:04:44,941 --> 00:04:46,601
And so making the tasks cheaper,

86
00:04:47,021 --> 00:04:47,961
encouraging the additional use

87
00:04:47,961 --> 00:04:49,261
to outweigh resource saving for tasks.

88
00:04:49,421 --> 00:04:52,021
For inference clouds, that's going to be more applications,

89
00:04:52,661 --> 00:05:03,081
serving more tokens, bigger market share. And for AppLayer individuals, that means you guys can serve to a lot more customers, anyone who's utilizing the token infrastructure.

90
00:05:03,081 --> 00:05:03,261
structure.

91
00:05:05,641 --> 00:05:12,161
And so what really goes into the macro structure, now like this is very bare bones here, but

92
00:05:12,161 --> 00:05:16,921
you've got your chips, which we've been talking about, you have power, you have buildings,

93
00:05:17,241 --> 00:05:21,941
grid connections, and then of course you need a lot of capital to get these build outs going

94
00:05:21,941 --> 00:05:23,701
and that's kind of been the whole talk here.

95
00:05:24,281 --> 00:05:29,261
And so chips, power, buildings, money, they all have to come together before you can even

96
00:05:29,261 --> 00:05:32,441
get to an hour of actual usable GPU per hour compute.

97
00:05:33,021 --> 00:05:37,401
And so the IEA April update projects that the global data center electricity is going

98
00:05:37,401 --> 00:05:42,701
to rise from 485 terawatt hours in 25 to 950 in 2030.

99
00:05:43,041 --> 00:05:48,101
And so a lot of the talk right now is around build outs about getting the power grid set

100
00:05:48,101 --> 00:05:48,241
up.

101
00:05:48,521 --> 00:05:50,181
Will we have enough power grid here?

102
00:05:50,381 --> 00:05:52,661
Is there other energy sources that we can tap into?

103
00:05:53,641 --> 00:05:57,061
Where are we able to get the land to actually be able to do this?

104
00:05:57,061 --> 00:06:03,821
And so, co-location sites and data center operators and people who are doing the financing, the build-outs, or the lenders themselves, are all kind of looking into this.

105
00:06:03,901 --> 00:06:12,901
And that type of everything that's upstream, essentially, of the compute is going to then flow down into the compute costs and also usage.

106
00:06:13,821 --> 00:06:17,581
And so, what does this, like, actual macro map look like underneath a quote?

107
00:06:17,981 --> 00:06:20,901
Chip and system vendors supply the GPU service and networking.

108
00:06:21,121 --> 00:06:26,201
Sites and operators supply the power space assembly and management.

109
00:06:26,201 --> 00:06:28,321
right here, GPUs, memory service, networking.

110
00:06:29,201 --> 00:06:30,401
Capacity providers, these are

111
00:06:30,401 --> 00:06:32,581
hyperscalers, neoclouds, and any

112
00:06:32,581 --> 00:06:34,441
other operator that's operating similar

113
00:06:34,441 --> 00:06:36,381
to how a neocloud would operate, reselling

114
00:06:36,381 --> 00:06:38,421
bare metal capacity, and then the users.

115
00:06:38,641 --> 00:06:40,201
So who's the actual end user? The startups,

116
00:06:40,661 --> 00:06:41,561
the inference clouds,

117
00:06:42,841 --> 00:06:44,361
labs, neolabs, and also anyone

118
00:06:44,361 --> 00:06:46,121
on the app layer, and enterprises

119
00:06:46,121 --> 00:06:48,301
for anyone doing open-weight

120
00:06:48,301 --> 00:06:48,781
stuff as well.

121
00:06:50,441 --> 00:06:51,801
And these boxes,

122
00:06:52,301 --> 00:06:54,041
I've kind of boxed them out here, but

123
00:06:55,081 --> 00:06:56,041
there's people

124
00:06:56,041 --> 00:06:58,061
People can be all of the above.

125
00:06:58,521 --> 00:07:02,181
A chip and system vendor, someone could do a full end-to-end build out and also be the

126
00:07:02,181 --> 00:07:03,561
end user of everything.

127
00:07:04,141 --> 00:07:08,661
And so there's a little bit of variation that occurs here.

128
00:07:15,261 --> 00:07:20,481
And so as I was kind of mentioning about the inference clouds before, they might rent a

129
00:07:20,481 --> 00:07:24,161
cluster and they'll serve tokens off of it and they'll use that revenue to pay off the

130
00:07:26,041 --> 00:07:29,481
And so that might be a use case for the actual inference cloud themselves.

131
00:07:29,901 --> 00:07:32,721
However, for hyperscalers, frontier labs, and larger NeoClouds,

132
00:07:32,761 --> 00:07:33,921
they can make substantial commitments

133
00:07:33,921 --> 00:07:36,141
because they have the ability to get into the large equity,

134
00:07:36,341 --> 00:07:37,841
they have the ability to take on more debt,

135
00:07:39,281 --> 00:07:43,501
and they have the ability to do long horizon kind of deals.

136
00:07:43,741 --> 00:07:47,461
So 10-year deal, open AI, contract it out, et cetera.

137
00:07:47,821 --> 00:07:48,381
So of that nature.

138
00:07:48,481 --> 00:07:51,261
However, anyone in the middle market isn't actually able to do this.

139
00:07:51,601 --> 00:07:53,821
Who's underwriting a Series B startup

140
00:07:53,821 --> 00:07:57,661
with enough capacity to get in a $200 million plus contract

141
00:07:58,081 --> 00:07:59,321
over three years for compute.

142
00:08:00,041 --> 00:08:02,401
How many equity rounds will they have to raise, right?

143
00:08:02,741 --> 00:08:04,521
Will they have to use their entire equity round

144
00:08:04,521 --> 00:08:06,121
to fund just the actual cluster

145
00:08:06,121 --> 00:08:07,941
and then they don't have any remaining capital

146
00:08:07,941 --> 00:08:10,281
left to use for actual costs?

147
00:08:12,121 --> 00:08:14,701
And so we're looking into a little bit of the financing chain here

148
00:08:14,701 --> 00:08:15,581
and how that plays in.

149
00:08:15,801 --> 00:08:17,481
So lenders are financing the build-out

150
00:08:18,101 --> 00:08:20,921
against contracted revenue even in many cases for inference clouds.

151
00:08:21,321 --> 00:08:22,961
And so they want to finance the build-out.

152
00:08:22,961 --> 00:08:24,941
They want confidence that their loan will get repaid.

153
00:08:25,421 --> 00:08:27,201
I mean, like, how are you going to underwrite a startup,

154
00:08:27,361 --> 00:08:28,781
like a private company that doesn't have, like,

155
00:08:29,121 --> 00:08:30,541
much else going for them right now?

156
00:08:30,621 --> 00:08:32,001
Like, what are the, you know, they're not publicly traded.

157
00:08:32,081 --> 00:08:34,181
There's no that much cash flows, balance sheet, income statement.

158
00:08:34,241 --> 00:08:36,501
They might even be pre-profit, pre-revenue, et cetera.

159
00:08:37,261 --> 00:08:38,801
And so what they're looking at is, like,

160
00:08:38,801 --> 00:08:42,321
can the person get contracted revenue over a certain scale?

161
00:08:42,621 --> 00:08:44,921
And so Core Reef's public filings give large-scale example

162
00:08:44,921 --> 00:08:46,921
of infrastructure financing supported by

163
00:08:46,921 --> 00:08:48,621
take-or-pay customer contracts.

164
00:08:48,861 --> 00:08:50,681
And so this diagram kind of shows the smaller,

165
00:08:52,221 --> 00:08:53,521
illustrated example here

166
00:08:53,521 --> 00:08:55,581
in the mismatch is that

167
00:08:55,581 --> 00:08:57,721
hardware financing, they want very long terms,

168
00:08:57,881 --> 00:08:59,541
but customers are only able to afford

169
00:08:59,541 --> 00:09:01,361
or don't really want to get into a long-term

170
00:09:01,361 --> 00:09:02,781
financing over 36 months.

171
00:09:03,081 --> 00:09:05,661
They have to put up on after 15% to 25%

172
00:09:05,661 --> 00:09:07,581
is what CoreWeave has quoted

173
00:09:07,581 --> 00:09:09,141
that they've seen their median

174
00:09:09,661 --> 00:09:11,521
prepayments to be, 15% to 25%

175
00:09:11,521 --> 00:09:12,181
up front

176
00:09:13,141 --> 00:09:15,321
on a 100 million TCV cluster.

177
00:09:15,481 --> 00:09:17,121
That's $15 to $25 million

178
00:09:17,121 --> 00:09:19,421
that a startup has to put up front.

179
00:09:19,421 --> 00:09:20,541
That might be their entire round.

180
00:09:23,001 --> 00:09:24,841
And so this is kind of a look at what I'm

181
00:09:25,601 --> 00:09:29,081
referencing here. It's like you're raising $40 million, you need a $100 million contract

182
00:09:29,081 --> 00:09:32,861
over three years, you put down 30% prepayments, this is an illustrative example,

183
00:09:33,141 --> 00:09:36,901
you've got $10 million left to run the business, and so you might have this

184
00:09:36,901 --> 00:09:40,821
kind of step function as you can see here, so that's your contract at capacity

185
00:09:41,221 --> 00:09:44,661
and what you can actually, and the demand is below that.

186
00:09:44,941 --> 00:09:48,721
And so now every time that you have to get into a new cluster, it's like tied to your

187
00:09:48,721 --> 00:09:52,481
equity round or whatever else it may be, but you might have to cap out and turn away customers.

188
00:09:52,681 --> 00:09:57,041
So what are your options? You either turn away customers until you can get into your next

189
00:09:57,961 --> 00:10:00,761
equity round so you can put down another 30% prepayment so you can get

190
00:10:00,761 --> 00:10:04,981
the next little bit of capacity going, or do you use on-demand to close the area

191
00:10:04,981 --> 00:10:09,221
under the gap and bridge the shortfall? That's like an

192
00:10:09,221 --> 00:10:12,581
option now. Now you have to look into the economics. What does that look like? What's the difference between

193
00:10:13,541 --> 00:10:16,921
maybe instead what they do is they go to a lender to finance the prepayment component,

194
00:10:16,921 --> 00:10:28,221
But if the yield on the prepayment component that they're getting financed is less than them just buying it out on demand at that time, if they're able to get into that amount of capacity, then does the economics make sense there?

195
00:10:33,821 --> 00:10:36,081
And so here's some of the operating models that we've seen.

196
00:10:36,441 --> 00:10:39,081
It's like you can build and operate the full stack.

197
00:10:39,961 --> 00:10:44,021
Hardware, site, your team is going to arrange power permit, the assembly, the management.

198
00:10:44,861 --> 00:10:49,041
You can also own servers in a co-location facility with more or less assembly and operation

199
00:10:49,041 --> 00:10:49,841
handled by a partner.

200
00:10:50,541 --> 00:10:55,241
On the other hand, an off-take contract buys defined capacity and the provider is going

201
00:10:55,241 --> 00:10:59,221
to own and operate the asset and they might not give you complete root access to the bare

202
00:10:59,221 --> 00:11:03,341
metal or there might be levels to how much access you get depending on if it's tenant,

203
00:11:03,541 --> 00:11:07,701
multi-tenant, et cetera type of configurations that can also affect the pricing.

204
00:11:09,301 --> 00:11:12,061
And so some businesses are going to value the physical access.

205
00:11:12,061 --> 00:11:16,181
Some businesses are going to value confidential compute.

206
00:11:17,041 --> 00:11:19,101
They might value certain topology changes

207
00:11:19,101 --> 00:11:20,621
or the ability to reconfigure the system.

208
00:11:20,741 --> 00:11:22,601
And so if that is something for your business,

209
00:11:22,701 --> 00:11:25,401
then they might actually want to have more and more control

210
00:11:25,401 --> 00:11:26,421
and build and operate the stack.

211
00:11:26,501 --> 00:11:28,101
And the economics for them might make sense

212
00:11:28,101 --> 00:11:30,821
given the opportunity costs they would have to give up

213
00:11:30,821 --> 00:11:33,361
if they were to just take rental off-take contracts.

214
00:11:33,381 --> 00:11:34,921
So they might be willing to pay more there.

215
00:11:42,061 --> 00:11:46,581
And so here's how, like, ownership and rental, this is a very, like, crude graph, how ownership

216
00:11:46,581 --> 00:11:48,701
and rental can compare as utilization arises.

217
00:11:49,621 --> 00:11:55,721
And so what we're looking at is that usually, you know, an off-take contract or these off-take

218
00:11:55,721 --> 00:11:59,661
contracts are getting to the cost of actually almost building out and financing your own

219
00:11:59,661 --> 00:12:00,081
cluster.

220
00:12:00,261 --> 00:12:03,401
And it's not, because, I mean, if you're spending $100 million over a three-year period,

221
00:12:03,881 --> 00:12:07,801
then maybe if the economics makes sense, it would have made more sense to simply buy out

222
00:12:07,801 --> 00:12:11,721
the compute operator yourself outside of the time cost and then obviously the operational

223
00:12:11,721 --> 00:12:17,861
costs that might exist. So there's an equilibrium that exists that has to be followed where

224
00:12:19,001 --> 00:12:27,741
the ownership costs outweigh the rental costs, and the rental costs should sit below the

225
00:12:27,741 --> 00:12:32,821
ownership costs. But if you're utilizing at such a rate and in such variable kind of increments,

226
00:12:32,881 --> 00:12:34,801
it might be best to just own the stack outright.

227
00:12:39,861 --> 00:12:41,881
And so we're seeing this new dynamic

228
00:12:41,881 --> 00:12:43,981
that the compute procurement

229
00:12:43,981 --> 00:12:46,821
affects how much runway a growing company actually has.

230
00:12:47,761 --> 00:12:49,701
And so this also gives investors

231
00:12:49,701 --> 00:12:50,841
an interest in compute procurement.

232
00:12:51,061 --> 00:12:52,081
So we're kind of seeing this.

233
00:12:52,481 --> 00:12:54,301
Access and payment terms affect how much runway

234
00:12:54,301 --> 00:12:56,201
their port codes are going to have

235
00:12:56,201 --> 00:12:57,081
from a VC perspective.

236
00:12:57,401 --> 00:13:00,741
So helping a company buy a suitable capacity

237
00:13:00,741 --> 00:13:02,041
can improve the underlying investment.

238
00:13:02,101 --> 00:13:03,421
And they're very connected, right?

239
00:13:03,481 --> 00:13:04,421
So they're able to procure

240
00:13:04,421 --> 00:13:07,321
and they're able to actually find compute capacity

241
00:13:07,321 --> 00:13:09,461
for their port codes and fund that.

242
00:13:09,781 --> 00:13:12,161
And so that also brings another interesting aspect

243
00:13:12,161 --> 00:13:13,001
to the macro structure.

244
00:13:13,321 --> 00:13:14,761
Into the micro structure is that

245
00:13:14,761 --> 00:13:16,341
who helps place the capacity

246
00:13:16,341 --> 00:13:18,521
and how does this offer get compared

247
00:13:18,521 --> 00:13:20,881
and what risks these contracts can transfer.

248
00:13:24,201 --> 00:13:26,041
So we have a lot of different players

249
00:13:26,041 --> 00:13:27,321
around this GPU hour

250
00:13:27,321 --> 00:13:29,541
that are helping kind of shape this market here.

251
00:13:29,721 --> 00:13:31,621
You've got your price discovery individuals.

252
00:13:31,621 --> 00:13:34,241
So people with indexes, data,

253
00:13:34,241 --> 00:13:38,361
benchmarks that we're all looking at right now and referencing which would be

254
00:13:38,361 --> 00:13:43,341
used to you know in the future launch the futures based on the benchmark it's

255
00:13:43,341 --> 00:13:50,261
you know talked about in the associated contracts per spec per pending approval

256
00:13:50,261 --> 00:13:54,741
and then there's the physical matching venues auctions aggregators so anyone

257
00:13:54,741 --> 00:13:59,581
who can match capacity we're looking at essentially decentralized marketplaces

258
00:13:59,581 --> 00:14:05,501
for on-demand compute. You're looking at liquid pools of compute that offer liquid pool services

259
00:14:05,501 --> 00:14:09,401
and part it out themselves and have their own standardized contract. You're looking at bare

260
00:14:09,401 --> 00:14:14,481
metal resellers. You're looking at neoclouds themselves doing larger capacity sales. And

261
00:14:14,481 --> 00:14:17,461
then you're looking at individual brokers and individual operators here as well on the bare

262
00:14:17,461 --> 00:14:22,841
metal reselling. You have destined brokers. They're looking at the bilateral quotes and execution.

263
00:14:23,001 --> 00:14:27,101
They might know a lot of people. They might be able to move fast. They might be able to see quotes

264
00:14:27,101 --> 00:14:28,881
have very similar specs in one place

265
00:14:28,881 --> 00:14:30,121
and know that there's a buyer

266
00:14:30,121 --> 00:14:33,161
that is extremely inelastic to price right now,

267
00:14:33,401 --> 00:14:34,421
needs to compute immediately,

268
00:14:34,481 --> 00:14:35,841
and they can get the RFS in time

269
00:14:35,841 --> 00:14:37,741
so they can charge a decent spread on it.

270
00:14:38,401 --> 00:14:40,461
And then what they do is they're going to have the source

271
00:14:40,461 --> 00:14:41,861
negotiating, and then they're going to be carrying

272
00:14:41,861 --> 00:14:42,601
and placing the risk.

273
00:14:43,041 --> 00:14:44,781
They're going to have to operate that cluster for the person,

274
00:14:44,881 --> 00:14:46,461
but it has to be worth that spread.

275
00:14:47,081 --> 00:14:49,301
And you have looks in financial markets,

276
00:14:49,561 --> 00:14:52,021
the exchanges that are coming that are based on the GPU

277
00:14:52,081 --> 00:14:53,661
hour, the futures and derivatives.

278
00:14:54,041 --> 00:14:59,381
People are going to be using it to take a look at how to maybe hedge out and look into more financial structures.

279
00:14:59,621 --> 00:15:04,901
So hedge out risk, so hedge out balance sheet risk, maybe get into more interesting agreements.

280
00:15:05,121 --> 00:15:10,401
We're seeing kind of agreements such as prepayment, payment facilities are coming online.

281
00:15:10,401 --> 00:15:12,401
we're seeing people utilize futures

282
00:15:12,401 --> 00:15:14,121
and are interested in utilizing futures

283
00:15:14,121 --> 00:15:14,881
and other kind of

284
00:15:17,041 --> 00:15:18,441
options to be able to

285
00:15:19,121 --> 00:15:20,141
there's demand for this

286
00:15:20,141 --> 00:15:22,161
to be able to hedge out the risk that they currently have

287
00:15:22,161 --> 00:15:24,141
on the books. We have buy side

288
00:15:24,141 --> 00:15:26,421
individuals looking for flex contracts

289
00:15:26,421 --> 00:15:27,501
flexible contracts like hey

290
00:15:27,501 --> 00:15:30,141
is there ability that if I lock myself into a 3-5 year

291
00:15:30,141 --> 00:15:31,781
contract that I'll be able to

292
00:15:31,781 --> 00:15:33,741
at least upgrade one of the new chips that comes out

293
00:15:33,741 --> 00:15:36,281
or maybe I'll be able to roll it over or maybe let me sell it back to you

294
00:15:37,461 --> 00:15:38,081
and then there's

295
00:15:38,081 --> 00:15:40,221
credit and financing so equipment lenders, leasers

296
00:15:40,801 --> 00:15:41,801
specialist credit providers.

297
00:15:43,461 --> 00:15:44,981
And so the way we're seeing it right now

298
00:15:44,981 --> 00:15:47,081
is that every deal is bespoke.

299
00:15:47,381 --> 00:15:49,201
There's different spec sheets, there's formats.

300
00:15:49,521 --> 00:15:51,081
The bill of materials may not match

301
00:15:51,761 --> 00:15:53,161
the actual delivery,

302
00:15:53,261 --> 00:15:55,161
or there might be a subject to change

303
00:15:55,161 --> 00:15:57,181
if a part is missing or a part is out of stock,

304
00:15:57,321 --> 00:15:59,421
and there's very long lead times

305
00:15:59,881 --> 00:16:01,521
that some of these OEM and ODM

306
00:16:01,521 --> 00:16:03,381
kind of providers have.

307
00:16:04,001 --> 00:16:05,161
And the same term can mean

308
00:16:05,161 --> 00:16:06,841
different things when you're negotiating very quickly.

309
00:16:06,981 --> 00:16:08,261
If someone's talking about memory,

310
00:16:08,681 --> 00:16:09,821
are we talking about, you know,

311
00:16:10,781 --> 00:16:12,821
VRAM, are we talking about the CPU memory,

312
00:16:12,901 --> 00:16:14,201
are we talking about additional storage here?

313
00:16:14,341 --> 00:16:15,681
Like, what did you exactly mean there?

314
00:16:15,941 --> 00:16:18,741
It affects quite a bit what type of loads can be run

315
00:16:18,741 --> 00:16:21,421
and, you know, the actual end users' needs,

316
00:16:21,501 --> 00:16:24,241
and also with pricing and delivery dates.

317
00:16:25,101 --> 00:16:26,361
So there's a lot of back and forth

318
00:16:26,361 --> 00:16:27,601
before anyone can really compare,

319
00:16:27,641 --> 00:16:28,901
and this is where the concept of, like,

320
00:16:28,981 --> 00:16:30,801
non-fungibility kind of comes into place

321
00:16:30,801 --> 00:16:32,201
and how to partition that out,

322
00:16:32,301 --> 00:16:33,421
what parts can be fungible,

323
00:16:33,521 --> 00:16:35,381
what parts hold the most of the weight,

324
00:16:35,441 --> 00:16:36,561
such as the skew, et cetera,

325
00:16:36,761 --> 00:16:38,261
and what parts are still, like,

326
00:16:38,341 --> 00:16:39,561
traded as if it was a product.

327
00:16:39,561 --> 00:16:44,081
e-commerce, et cetera, where it's only for that particular cluster that this price applies.

328
00:16:46,141 --> 00:16:48,441
And so prices can update daily or faster.

329
00:16:48,641 --> 00:16:49,801
Quotes are still a good starting point.

330
00:16:49,901 --> 00:16:50,601
Bids can get shopped.

331
00:16:50,681 --> 00:16:52,481
Gradwarding has changes of what's on offer.

332
00:16:52,661 --> 00:16:55,001
There's all these kind of latent factors, hidden factors as well,

333
00:16:55,161 --> 00:16:56,461
why we're calling it a dark market.

334
00:16:57,361 --> 00:17:01,481
These are some public H100 references to NAP-chotted today.

335
00:17:01,681 --> 00:17:02,321
I updated them.

336
00:17:02,581 --> 00:17:03,841
So take a look at what the public...

337
00:17:03,841 --> 00:17:06,941
So those are the references for if you wanted to go on their websites to see.

338
00:17:07,381 --> 00:17:08,841
This is just an example here.

339
00:17:08,841 --> 00:17:14,801
of what a public on-demand or spot pricing looks like for an H100 today.

340
00:17:15,401 --> 00:17:18,101
You can see various prices here.

341
00:17:18,561 --> 00:17:21,661
People might use this or they might use their own internal,

342
00:17:21,881 --> 00:17:26,661
what they've seen, to actually assume or use it to charge

343
00:17:26,661 --> 00:17:29,301
what they want to charge on their actual cluster specification,

344
00:17:29,481 --> 00:17:32,341
taking into account every other kind of factor that goes into it.

345
00:17:34,321 --> 00:17:37,501
Why are these new structures needed and what are these microstructures?

346
00:17:37,681 --> 00:17:38,141
Interesting.

347
00:17:38,841 --> 00:17:40,641
It's mainly to solve for these mismatches, right?

348
00:17:40,881 --> 00:17:42,721
Bridging capacity, more flexible commitments,

349
00:17:42,921 --> 00:17:45,121
resale rights, and protection against price moves.

350
00:17:46,581 --> 00:17:48,301
You know, you're selling a fixed-price service

351
00:17:48,301 --> 00:17:49,861
from an inference cloud point of view.

352
00:17:49,961 --> 00:17:51,121
You'd be selling a fixed-price service

353
00:17:51,121 --> 00:17:52,801
while the GPU bill floats,

354
00:17:52,861 --> 00:17:54,341
and that creates price exposure.

355
00:17:54,581 --> 00:17:57,001
And so locking capacity for years can solve a part of that,

356
00:17:57,181 --> 00:18:00,581
but it'll create idle hour exposure elsewhere as well,

357
00:18:00,761 --> 00:18:01,841
unless you're always using it.

358
00:18:02,161 --> 00:18:04,141
Reselling spare capacity only helps

359
00:18:04,141 --> 00:18:05,601
if someone actually can buy the hours.

360
00:18:05,881 --> 00:18:08,141
A financial hedge can address a reference price,

361
00:18:08,141 --> 00:18:10,621
but can't deliver the cluster right now in terms of cash settlement

362
00:18:10,621 --> 00:18:12,521
or guarantee a utilization.

363
00:18:12,901 --> 00:18:15,381
So the useful structure has to start with the particular risk

364
00:18:15,381 --> 00:18:17,561
that the business has and needs to manage.

365
00:18:21,641 --> 00:18:24,641
And I'll go pretty fast on this policy section

366
00:18:24,641 --> 00:18:26,281
because we're running out of time.

367
00:18:27,021 --> 00:18:30,741
Everything that Affaan has said sits inside of a structure

368
00:18:30,741 --> 00:18:36,761
that is regulated and is affected by policy.

369
00:18:38,141 --> 00:18:52,361
In various ways. One way is export controls. One of the examples, in January, the U.S. Commerce Department moved qualifying H200 class license reviews for China and Macau to case-by-case review with conditions.

370
00:18:53,741 --> 00:18:59,821
furthermore open weight models connect policy back to the u.s economics we started with the

371
00:18:59,821 --> 00:19:05,021
u.s ai action plan explicitly encourages open source and open weight ai and recommends improving

372
00:19:05,021 --> 00:19:11,281
the financial market for compute so that startups and academics can access large-scale computing

373
00:19:11,281 --> 00:19:18,901
power so i'm going to tie this out right here but where's this opportunity like we have this

374
00:19:18,901 --> 00:19:23,201
growing middle market. We need balance sheets that can bridge buyer and supplier needs. There needs

375
00:19:23,201 --> 00:19:27,861
to be some level of standardization, a standard capacity contract, common specs, common terms,

376
00:19:28,001 --> 00:19:32,761
and definitions. And so growing companies whose needs and commitment do not fit the contracts,

377
00:19:32,901 --> 00:19:37,301
the hyperscalers, and Frontier Labs consign. Those companies have real workloads. We want them to

378
00:19:37,301 --> 00:19:41,841
grow. They still face this financing problem, a timing problem, a specification problem. A supplier

379
00:19:41,841 --> 00:19:44,781
might want the long-term commitment for the build-out while the buyer wants to expand in

380
00:19:44,781 --> 00:19:49,141
smaller blocks as demand becomes clearer, as it's so volatile, front of curve. The buyer needs a way

381
00:19:49,141 --> 00:19:53,241
to avoid paying for capacity it can't use yet, and so participants with strong balance sheets

382
00:19:53,241 --> 00:19:57,201
that are uncorrelated with the market and the ability to assess those risks with a strong

383
00:19:57,201 --> 00:20:02,001
credit profile can help bridge the gap between what buyers need and what suppliers offers.

384
00:20:02,201 --> 00:20:06,061
That requires more than quoting the actual GPU hour. It means understanding the delivery,

385
00:20:06,221 --> 00:20:11,221
workload, all of that, and so there's very few kind of players in the market, and a very well-suited

386
00:20:11,221 --> 00:20:15,121
player, can be a firm that's uncorrelated with market, has a strong balance sheet,

387
00:20:15,221 --> 00:20:21,121
strong credit rating, and incredible amounts of hardware talent that sits inside for high-performance

388
00:20:21,121 --> 00:20:21,581
computing.

389
00:20:21,821 --> 00:20:24,621
And so, just making a CTA here, let's build this together.

390
00:20:24,861 --> 00:20:27,701
Thank you guys so much for attending our talk.