[
  {
    "startMs": 12820,
    "text": "Um, hey everyone. My name is Affaan, and, um, I'm a lead for compute trading at Jump Trading, which is a proprietary trading firm, alongside my co-lead over here, Alejandro. So today we're gonna be looking over some of the market structure as it applies to compute trading, what compute trading even is, um, what we can see it becoming. And I think we'll start with some macro, look at microstructure, and then talk a little bit about where we see things in policy."
  },
  {
    "startMs": 44600,
    "text": "So the title of the talk is The Dark Pools of Compute, but some of you guys are in finance, so you might be confused as to why I'm using this terminology. Uh, what do I mean by dark pool here? Essentially, what I'm referring to is an unlit market. Um, right now most compute trades in blocks, physical compute here, and these are private quotes. We have a lot of brokers, and these are long bilateral contracts. Now, at the end of the d-day, these are really commercial agreements, so we're using the term trading here to kind of like look at how it's transacted between"
  },
  {
    "startMs": 74460,
    "text": "parties. These commercial agreements tend to be some mixture of an MSA, an SLA, and an order form, cluster-specific, uh, with hardware-specific, uh, language in each kind of, um, in each order. And but when we're looking at pricing and looking at some of these other kind of, uh, metrics, a lot of the time most of the talk is around the SKU. So what the chip itself is, the baseboard. So H100, H200, B300, uh, et cetera. And so, um,"
  },
  {
    "startMs": 104460,
    "text": "a lot of the negotiation here happens kind of outside the public view. There's no, you know, like exchange for physically traded compute that's widely, you know, adopted and, you know, used for everyone goes there to like take a look and trade, uh, compute. These are done in chats, right? They're out of the public view. Um, there's public marketplaces, there's price references, there's a lot of, um, you know, on-demand marketplaces. However, for these actual tenored, so, you know, on a given term for a fixed amount of capacity, uh, with a"
  },
  {
    "startMs": 134420,
    "text": "certain ter-- total contract value, these trade over Slack messages, X messages, Telegram group chats. There's a bunch of OTC kind of channels that are all over in these kind of, I wanna call them like Neo-Neochat. Um, and some of you might be familiar with commodities and how a lot of, you know, there's chatting happening on ICE Chat, Bloomberg chat, et cetera. This is kind of where, where the, where the kind of ma-markets here lie. And so I want to look a little bit here on the macro market structure and a little bit on the"
  },
  {
    "startMs": 164320,
    "text": "players here as well. And so this was a recent kind of, uh, a16z. Some of you guys might have seen this, so I'm, I'm putting this out there because, you know, you can go on a16z's, uh, actual X and take a look at this graph yourself. What we're looking at is that the cost of a given capability has fallen sharply. Um, the bill of course still depends on the model usage, capacity, and other factors such as speed is a big thing right now with the new co-- uh, ChatGPT models. Um, cheaper tokens unlock more usage and a"
  },
  {
    "startMs": 194340,
    "text": "team can do more work for the same budget. So what we're seeing is that a lot of teams are simply just using way more tokens. So we have this latent demand that hasn't really been hit yet, and people are just going to use more with the same budget that they were using before when models cost more, um, and they're just spending more tokens. Um, so applications that were too expensive become possible. Um, however, as we're kind of talking about compute here, the capacity underneath still has to be built and paid for, um, even as the cost of a particular task comes down. And so the"
  },
  {
    "startMs": 224380,
    "text": "important distinction is between the price of the intelligence, the number of tasks people want to run, and the amount of hardware needed to serve them."
  },
  {
    "startMs": 233600,
    "text": "And so this is where we get to kind of the Jevons paradox and a little-- some of the stuff that we've been looking at lately, where we had a very interesting thing where, you know, intuitively you'd assume, um, that compute and inference kind of like rises together, correlates very, you know, very closely as you can see. You can see there's a little bit of a decoupling that happened and token prices declined further, but compute prices re-remain robust. And so like what, what can be an explanation for this? Now, there's a lot of like potential explanations. One"
  },
  {
    "startMs": 263560,
    "text": "here is the whole Jevons paradox is that even though token prices and inference is getting extremely efficient to serve on these new, uh, baseboards and, and, and GPUs, people are just consuming a lot more. And so the demand for compute rises and people just consume a lot more tokens even if the cost per, uh, cost per million, uh, uh, for input and for caching and, you know, for output becomes a lot more efficient. And so making the tasks cheaper, encouraging the additional use to outweigh resource saving per task. Uh, for inference clouds, that's going"
  },
  {
    "startMs": 293560,
    "text": "to mean more applications, serving more tokens, uh, bigger market share. Um, and for app layer individuals, that means you guys can serve to a lot more kinda customers. So anyone who's building utilizing the token, uh, infrastructure."
  },
  {
    "startMs": 308360,
    "text": "And so what really goes into, um, you know, the macro structure. Now like this is very, you know, bare bones here, but you've got your chips, which we've been talking about. Uh, you have power, you have buildings, grid connections, and then of course you need a lot of capital to get these build-outs going, and then that's kind of been the whole talk here. And so, um, chips, powers, buildings, money, they all have to come together before, you know, you can even get to an hour of like actual usable GPU-per-hour compute. And so the IEA April update projects that the glo-global data"
  },
  {
    "startMs": 338320,
    "text": "center electricity is gonna rise from four hundred and eighty-five terawatt-hours in '25, um, to nine hundred and fifty in 2030. And so a lot of the talk right now is around, um, build-outs, about getting the power grid set up. Will we have enough power grid here? Is there other energy sources that we can tap into? Um, where are we, you know, able to get the land to actually be able to do this? And so colocation sites and data center operators and people who are doing the financing, the build-outs or the lenders themselves are all kind of looking into this and that type of, uh, everything"
  },
  {
    "startMs": 368040,
    "text": "that's upstream essentially of the compute is going to then flow down into the compute, uh, costs and also usage. And so what does this like actual macro map look like underneath it? Quote, \"Chip and system vendors supply the GPU servers and networking. Sites and operators supply the power, um, space assembly and management.\" Right here, GPUs, memory servers, and networking. Um, capacity providers, these are hyperscalers and neoclouds and any other operator that's like operating similar to how a neocloud would"
  },
  {
    "startMs": 397852,
    "text": "operate, reselling bare metal capacity. And then the users. So who's the actual end user? The startups, the inference clouds, uh, labs, neo labs, and also anyone on the app layer, uh, and enterprises for anyone doing, um, open-weight stuff as well. And, um, these boxes, like I've kind of boxed them out here, but, you know, there's... People can be all of the above. A chip and system vendor, like someone could do a full end-to-end build-out and also be the end user of everything. And so there's a"
  },
  {
    "startMs": 427812,
    "text": "little bit of, um, variation that, that, that occurs here."
  },
  {
    "startMs": 438032,
    "text": "And so as I was kind of mentioning about the inference clouds before, they might rent a cluster, and they'll serve tokens off of it, and they'll use that revenue to pay off the cluster. Um, and so that might be a use case for the actual inference cloud themselves. However, for hyperscalers, frontier labs, and larger neoclouds, they can make substantial commitments because they have the ability to get into the large equity, they have the ability to take on more debt, and, uh, they have the ability to do long horizon kind of, um, deals. So ten-year deal, OpenAI"
  },
  {
    "startMs": 467952,
    "text": "contracted out, um, et cetera. So of that nature. However, anyone in the middle market isn't actually able to do this. Who's underwriting a Series B startup with enough ca-capacity to get into two hundred million plus contract over three years for compute? How many equity rounds will they have to raise, right? Will they have to use their entire equity round to fund just the actual cluster, and then they don't have any remaining capital left to use for actual costs?"
  },
  {
    "startMs": 494832,
    "text": "And so we're looking into a little bit of the financing chain here and how that plays in. So lenders are financing the build-out, um, against contracted revenue even in many cases for inference clouds. Um, and so they want to finance the build-out, and they want confidence that their loan will get repaid. Wha-- I mean, like, how are you gonna underwrite a startup, like a private company that doesn't have, like, much else going for them right now? Like, what are the... You know, they're not publicly traded. There's no that much cash flows, balance sheet income statement. They might even be pre-re-- pre-profit, pre-revenue, et cetera. Um, and so what they're looking at is like, can the person get contracted revenue over a certain scale?"
  },
  {
    "startMs": 525132,
    "text": "And so CoreWeave's public filings give large-scale example of infrastructure financing supported by take-or-pay customer contracts. And so this diagram kind of shows the smaller, um, illustrative example here. Um, and the mismatch is that hardware financing, they want very long terms, but customers are only able to afford or don't really want to, like, get into a long-term financing over thirty-six months. They have to put up on up to fifteen to twenty-five percent is what CoreWeave, um, has quoted that they've seen their median, um, prepayment to be fifteen to twenty-five percent upfront,"
  },
  {
    "startMs": 555212,
    "text": "um, on a hundred million PCV cluster. That's fifteen to twenty-five million dollars that a, um, startup has to put upfront. That might be their entire round."
  },
  {
    "startMs": 565732,
    "text": "And so this is kind of a look at what I'm, you know, referencing here. It's like you're raising forty million. You need a hundred million contract over three years. You put down thirty percent prepayment as an illustrative example. You've got ten per-- ten million left to run the business. Um, and so you might have this kind of step function as you can see here. So that's your contracted capacity and what you can actually... And the demand is like, you know, uh, below that. And so now every time that you have to get into a new cluster, it's like tied to your equity round or whatever else it may be, but you might have to cap out and turn away customers. So what are your options?"
  },
  {
    "startMs": 596032,
    "text": "Do you either turn away customers until you can get into your next, um, you know, equity round so you can put down another thirty percent prepayment so you can get next little bit of capacity going? Or do you use on-demand to close the area under the gap and bridge the shortfall? That's like an, that's like an option now. Now you have to look into the economics. What does that look like? What's the difference between, you know, maybe instead what they do is they go to a lender to finance the prepayment component. But if the yield on the prepayment component that they're getting financed is less than, um, them just buying it out on"
  },
  {
    "startMs": 625932,
    "text": "demand at that time, if they're able to get into that amount of capacity, then does the economics make sense there?"
  },
  {
    "startMs": 636572,
    "text": "And so here are some of the operating models that we've seen. It's like you can build and operate the full stack. Hardware, site, your team is gonna arrange power permit, the assembly, the management. You can also own servers in a colocation facility with more or less the assembly and operation handled by a partner. Um, on the other end, an offtake contract buys defined capacity, and the provider is going to own and operate the asset, and they might not give you complete root access to the bare metal, or there might be kind of like levels to how much access you get depending on if it's tenant, multi-tenant,"
  },
  {
    "startMs": 666852,
    "text": "and et cetera type of like configurations that can also affect then the pricing. Um, and so some businesses are gonna value the physical access. Some bus-businesses are gonna value, um, you know, confidential compute. Uh, you know, they might value certain topology changes or the ability to reconfigure the system. And so if that is something for a business, then they might actually want to have more and more control and build and operate the stack, and the economics for them might make sense given the opportunity cost they would have to give up, um, if they were to just take rental offtake contracts. So they might be willing to"
  },
  {
    "startMs": 696872,
    "text": "pay more there."
  },
  {
    "startMs": 704572,
    "text": "And so here's how, like, ownership and rental... This is a very, like, crude graph. Um, how ownership and rental can compare as utilization r-rises. Um, and so what we're looking at is that usually, you know, an offtake contract or the-- these offtake contracts are getting to the cost of actually almost building out and financing your own cluster. And it's not... 'Cause I mean, if you're spending a hundred million over a three-year period, then maybe if the economics make sense, it would've made more sense to simply buy out the compute operator itself outside of the time cost and then obviously the operational cost that"
  },
  {
    "startMs": 734572,
    "text": "might exist. So there's a, there's a equilibrium that exists, um, that, that has to be followed where You know, the ownership costs outweigh, um, the, um, you know, rental costs, and the rental costs should sit below the ownership costs. But if you're utilizing it at such a rate and, uh, you know, in, in such variable kind of increments, it might be best to just own the stack outright."
  },
  {
    "startMs": 762668,
    "text": "And so we're seeing this new dynamic that compute, uh, the compute procurement affects how much runway a growing company actually has. And so this also gives, uh, investors an interest in compute procurement. So we're kind of seeing this. Um, access and payment terms affect how much runway their portcos are going to have from a VC perspective. Uh, so helping a company buy a suitable capacity can improve the underlying investment. And they're very connected, right? So they're able to procure, and they're able to actually find, uh, compute capacity for their portcos and, and fund that. And so"
  },
  {
    "startMs": 792588,
    "text": "that also brings another interesting, uh, aspect to the macro structure, um, into the microstructure is that who helps place the capacity and, uh, how does this offer get compared and what risks these contracts can, can transfer."
  },
  {
    "startMs": 807028,
    "text": "So we have a lot of different players around this GPU hour that are helping kind of shape this, this market here. You've got your price discovery individuals, so people with, uh, indexes, data, the benchmarks that we're all looking at right now and referencing, um, which would be used to, you know, in the future, uh, um, launch the futures based on the benchmarks, uh, you know, uh, talked about, um, in, in the associated contracts, um, per spec, per pending approval. Um, and then there's the physical matching venues, auctions, aggregators. So"
  },
  {
    "startMs": 836968,
    "text": "anyone who can match capacity. We're looking at, um, essentially, uh, decentralized marketplaces for on-demand compute. You're looking at, um, liquid pools of compute that offer like liquid pool services and part it out themselves and have their own standardized contract. You're looking at bare metal resellers. You're looking at neoclouds, um, themselves doing larger capacity, uh, sales. And then you're looking at individual brokers and individual operators here as well on the bare metal reselling. Um, you have desks and brokers. They're l-looking at the bilateral quotes and execution. They might know a lot of people."
  },
  {
    "startMs": 866988,
    "text": "Um, they might be able to move fast. They might be able to see quotes at very similar specs in one place and know that there's a buyer that is, uh, extremely, uh, inelastic to price right now, needs the compute immediately, and they can get the RFS in time, so they can charge a decent spread on it. And then what they do is they're gonna have the source negotiating, and then they're gonna be carrying and placing the risk. Um, they're gonna have to operate that cluster for the person and-- but it has to be worth that spread. Um, and you know, you have looks in financial markets, uh, the exchanges that are coming, uh, that are based on the GPU hour, the futures and derivatives. Um, people are"
  },
  {
    "startMs": 897008,
    "text": "gonna be using it to take a look at how to maybe hedge out and look into more, you know, financial structures. So hedge out risk. So hedge out, you know, my balance sheet risk, maybe get into more interesting, um, agreements. We're looking-- we're seeing kind of agreements such as, uh, prepayment payment facilities are coming online. We're seeing people utilize futures and-- or are interested in utilizing futures and other kind of, um, um, uh, options to be able to, um... There's demand for this to be able to hedge out the risk that they currently have on the books. We have buy-side"
  },
  {
    "startMs": 926668,
    "text": "individuals, uh, looking for flex contracts, flexible contracts like, \"Hey, is there ability that if I lock myself into a three to five year contract that I would be able to, you know, at least upgrade when a new chipset comes out? Or maybe I'll be able to roll it over. Maybe let me sell it back to you.\" And then there's like credit and financing, so equipment lenders, leasers, um, specialist credit providers. And so the way we're seeing it right now is that every deal is bespoke. There's different spec sheets. There's formats. The bill of materials may not match, um, the actual like delivery, or there"
  },
  {
    "startMs": 956608,
    "text": "might be subject to change if a part is missing or a part is out of stock. And, you know, there's like very long lead times, um, that some of these OEM and ODM kind of, uh, providers have. And the same term can mean different things when you're negotiating very quickly. If someone's talking about memory, um, are we talking about, you know, uh, VRAM? Are we talking about the CPU memory? Are we talking about additional storage here? Like, what did you exactly mean there? It affects quite a bit what type of loads can be run and, you know, the actual end user's, um, needs, and also with pricing and delivery"
  },
  {
    "startMs": 986428,
    "text": "dates. Um, so there's a lot of back and forth before anyone can really compare, and this is where the concept of like non-fungibility kind of comes into place and how to partition that out, what parts can be fungible, what parts hold the most of the weight, such as the SKU, et cetera, and what parts are still like traded as if it was a product, e-commerce, et cetera, where, you know, it's only for that particular cluster that this price applies."
  },
  {
    "startMs": 1008848,
    "text": "And so prices can update daily or faster. Quotes still a good starting point. Bids can get shopped. Credit wording as changes what's on offer. There's all these kind of latent factors, hidden factors as well, why we're calling it a dark market. Um, these are some public H100 references snapshotted, um, today. I updated them. So take a look at the-- what the public-- So those are the references for if you wanted to go on their websites to see. Um, this is just an example here, um, of like what a public on-demand or spot pricing looks like for an H100, um, today. You can see kind"
  },
  {
    "startMs": 1038808,
    "text": "of various, various prices here. And so people might use this, or m-they might use their own kind of internal, what they've seen, um, to actually like assume or, or use it to charge what they want to charge on their actual cluster specification, take into account every other kind of factor that goes into it."
  },
  {
    "startMs": 1056968,
    "text": "And so why are these new structures kind of needed, and what are these microstructures, um, interesting? Um, it's mainly to solve for these mismatches, right? Bridging capacity, more flexible commitments, resale rights, and protection against price moves. Um, you know, you're selling a fixed price service, uh, from an inference cloud point of view. You'd be selling a fixed price service while the GPU bill floats, and that creates price exposure. And so locking capacity for years can solve a part of that, but it'll create idle hour, uh, idle hour exposure elsewhere as well, unless you're always using it. Reselling spare capacity only helps if someone"
  },
  {
    "startMs": 1086908,
    "text": "actually can buy the hours. Uh, a financial hedge can address a reference price but can't deliver the cluster right now in terms of cash settlement, um, or guarantee a utilization. Um, so the useful structure has to start with the particular risk that the business has, um, and needs to manage."
  },
  {
    "startMs": 1104484,
    "text": "And I'll, I'll go pretty fast on this policy section because we're running out of time. Um, everything that Affaan has said sits inside of a structure that is regulated and is affected by policy in various ways. One way is export controls. One of the examples, in January, the U.S. Commerce Department moved qualifying H200 class license reviews for China and Macau to case-by-case review with"
  },
  {
    "startMs": 1134284,
    "text": "conditions. Furthermore, open-weight models connect policy back to the U.S. economics we started with. The U.S. AI Action Plan explicitly encourages open source and open-weight AI and recommends improving the financial market for compute so that startups and academics can access large-scale computing power."
  },
  {
    "startMs": 1157464,
    "text": "So I'm going to tie this out right here, but where does this opportunity lie? We have this growing middle market. We need balance sheets that can bridge buyer and supplier needs. There needs to be some level of standardization, a standard capacity contract, common specs, common terms, um, and definitions. And so growing companies whose needs and commitment do not fit the contracts that hyperscalers and frontier labs can sign, those companies have real workloads. We want them to grow. They still face this financing problem, a timing problem, a specification problem. A supplier might want the long-term commitment for the build-out while the buyer wants to expand in smaller"
  },
  {
    "startMs": 1187424,
    "text": "blocks as demand becomes clearer, as it's so volatile, front of curve. Uh, the buyer needs a way to avoi-avoid paying for capacity it can't use yet. And so participants with strong balance sheets that are uncorrelated with the market and the ability to assess those risks with a strong credit profile can help bridge the gap between what, um, buyers need and what suppliers offers. That requires more than quoting the actual GPU hour. It means understanding the delivery, workload, all of that. And so there's very few kind of players in the market and, you know, a very well-suited player, um, can be a firm that's uncorrelated with market, has a strong balance"
  },
  {
    "startMs": 1217384,
    "text": "sheet, strong credit rating, and incredible amounts of hardware talent that sits inside for high-performance computing. And so just making a CTA here, let's build this together. Thank you guys so much for attending our talk."
  }
]