[
  {
    "startMs": 12720,
    "text": "So this is a talk about compute markets, um, but it's kind of more of a talk about why to make a compute market. Um, if you don't know me, I'm Evan. Um, we run a company called San Francisco Compute. Um, we make a physically settled order-- or a physically settled forwards market. Um, so if you were here for the last talk, the last talk was basically why all the operators of GPU clusters, um, don't really work very well. Um, when we say physically settled, we mean we're the operator."
  },
  {
    "startMs": 42760,
    "text": "We're both the operator of the hardware in the same way that a cloud is the operator of the hardware, and then we're also the market at the same time. It's kind of like a grid for compute. So the general gist of this is that in order to run and operate a supercomputer, um, we think problems start right at the design of the supercomputer, um, right in the design of the bill of materials. Um, it's control of the backplane fabric as well. It's also control of, um, bare-metal orchestration. It's also services on"
  },
  {
    "startMs": 72560,
    "text": "top, VMs, and so on. Um, there's a lot of different parts of the stack that you have to do effectively. Otherwise, um, the folks who have a data center, um, and would like to make more money on that data center by becoming the GPU cloud, um, end up failing for the customer. So that's what we do. We offer what's called supercomputing as a service. We go to folks who want to become a cloud. That's like folks who want to become what NVIDIA would call an NCP, um, or"
  },
  {
    "startMs": 102440,
    "text": "venture firms that want to have clusters for their portfolio, um, and want to be able to reallocate that capacity when a company like Instinct is taking off, um, or AI companies who would like the residual value on the compute because they've realized that's really important to their financials, um, but don't want to operate a supercomputer. So that's physical settlement. And then on top of that, we run a live order book for capacity that you can buy reserved either right now or in the future. Um, this, um, together is not at all"
  },
  {
    "startMs": 132420,
    "text": "what we thought we were going to be doing. Originally, w-we were this company called Junelarc, um, that was trying to make kind of like Suno or Udio, but prior to Suno or Udio. Um, at the time, um, what you needed to do to realistically start a lab was you needed to raise a big deposit, a huge one, w-which was ten million dollars at the time. Um, we thought that was a crazy amount of money, um, and so we thought, \"No, that's stupid. We're gonna raise five hundred K, and obviously what we're gonna do is we're gonna rent from somebody else.\" Um,"
  },
  {
    "startMs": 162400,
    "text": "obviously today, um, it's not ten million dollars, it's a billion dollars. And, um, nonetheless, um, we had raised our five hundred K. We buy a big cluster, and then we're underwater on our cluster every month. Um, and if we didn't sublease our cluster, we would have gone bankrupt. Um, so if you've heard of GPU brokers, we were the first GPU broker because we were forced into being a GPU broker. So the first year of the company was us just trying to sell our cluster, otherwise we would go bankrupt. Um, so I would wander around with these, like, actual, um, pieces of"
  },
  {
    "startMs": 192420,
    "text": "paper at NeurIPS, which both say H100 and A100 at the same time because that's how I made, uh, I made the, uh, poster so quickly. Um, and so this talk is not really about, um, necessarily like how a compute market works. It's how a compute market, um, it gets to exist. Uh, we like to think about it like there's this pamphlet, and on the pamphlet on the front side, it's like, \"Man, starting a neo-lab sucks. There's all these different, like, financial and economic problems.\" On the back of the pa--"
  },
  {
    "startMs": 222340,
    "text": "back of the pamphlet, it says, \"Compute markets.\" It's, it's not super clear how one gets to the other until you open the pamphlet and then like, oh my, holy shit, there's like so much stuff in this pamphlet, and it keeps going, and there's so many folds. It's just a very difficult and challenging problem because you need to solve two, like, miracle problems to get working. You have to both make a market, which is, uh, a uniquely difficult problem because it has a cold-start problem. You also have to build a cloud, and then you have to not become a cloud, um, which is a whole separate challenge. But these are the problems that"
  },
  {
    "startMs": 252300,
    "text": "the labs are facing, and it's about to be the problems that everyone is about to facing. So if you work at a startup, these are your problems now. Um, right now everybody is sold out of compute. Um, this is, um, probably not going to change for a while. Um, so that means that you're gonna have to buy a new cluster, a net new one, and there's a lead time. You have no way to pay for that cluster to come online now. You're gonna pay for it to come online later. There's nothing in the market that can realistically get that cluster online, online for you now. Um, meanwhile, you're gonna pay your entire bank"
  },
  {
    "startMs": 282080,
    "text": "balance as a deposit. Um, the reason you're paying your entire bank balance as a deposit is because the lender has no idea how to under-underwrite AGI. Um, how in the world are you supposed to do that? You've got two-person startups in San Francisco who, or maybe New York, um, who say, \"I'm gonna build superintelligence.\" No lender knows how to underwrite superintelligence, um, so they have no idea what the asset is worth. Um, meanwhile, your cluster is very likely to be delayed. That's because the lender needed to underwrite against the credit risk of the"
  },
  {
    "startMs": 312020,
    "text": "customer, and so they needed to know who is the customer first before you deploy the cluster. So right when the customer signs is the riskiest point in the entire operation. Um, you wanted it to be the case that you built the cluster first and then you sell it. This also helps reduce all sorts of different types of problems, um, where clusters break. Meanwhile, most of the people who are setting up clusters are people who have the actual asset that makes the thing cheap. Like you've got the data center space already. What that doesn't get is supercomputing expertise, um, as"
  },
  {
    "startMs": 341980,
    "text": "we saw in the last talk if you've, um, watched that one. Um, meanwhile, you're gonna run into the exact same problem that Junelarc did, which is you're gonna either way overbuy or you're gonna way underbuy. Um, that's because AI doesn't have margins that look like SaaS companies, and so you don't have very much margin for error. And so the chances that you either underbuy and somebody else wins because they did it correctly, or you overbuy and then you just become Stability AI or you blow up, you, um, end up like Character AI, um, are really high. So these are all"
  },
  {
    "startMs": 371928,
    "text": "the same problem. They're all just different types of risk. They're technical risk, they're financial risk, they're credit risk. Um, they're all one big bundled risk problem, uh, which is the problem that we end up solving, and it's also the problem that we think compute markets broadly, the entire industry solves. Um, so we'll walk through each one of these and walk how that gets to a compute market because if you don't understand these, it's kind of like unclear why you would want a compute market. Um, and they almost seem like this problem that somebody thinks, \"Oh, wow, compute markets are cool, so I'm gonna go start one.\" We think they should be derived from the actual problems that people face,"
  },
  {
    "startMs": 401868,
    "text": "which is, first, everyone is sold out. Why are they sold out? Because the lender couldn't finance the cluster unless you had a long-term offtake agreement. In CPUs, uh, this wasn't the case because CPU buyers are SaaS companies. SaaS companies have high margins, and because they have high margins, um, you can build a product that rewards you for having, um, a great, um-- that rewards you with high margins, um, for your, um, your CPUs. Um,  think about it like this way. If you're a SaaS"
  },
  {
    "startMs": 431668,
    "text": "company and your underlying cloud doubles their prices, you don't go bankrupt. You hate your cloud, you're mad, but you don't go bankrupt. If you're an AI company and your underlying cloud doubles their prices, you go bankrupt unless you go to some other underlying cloud. And what that means is that you're actually price sensitive, and because you're actually price sensitive, um, and there's nothing keeping you at any particular cloud, so you can't sell on demand because your customers will immediately leave to whoever is cheaper. And so, um, that means that in order to finance GPU clusters, you have to"
  },
  {
    "startMs": 461548,
    "text": "have long-term take-or-pay contracts, hopefully with someone who has investment-grade credit. However, if you don't have liquidity on those, if you're selling reserved and your customer can't get out, then that also means that some other customer who wants the capacity right now because they're like Instinct and their product is taking off, can't pay for anything. They have to finance a new cluster, which has a lead time because the physical world just exists, and it's gonna take a while. Sometimes you even have to build the damn data center. Um, and then because you're financing a new cluster, you're gonna pay a deposit because almost"
  },
  {
    "startMs": 491308,
    "text": "nobody in AI has any reasonable credit. And because this is a brand-new technology, all these companies are new.  Meanwhile, even as you, um, start to scale up that ladder, you're still gonna run into credit issues all the way through. It's the number one bottleneck of AI at the moment. It's not power. You hit credit way before you hit AI or way before you hit, um, um,  uh, power. So, um, what that means is every inference company, um, uh, every neo-lab, they're bottlenecked by how much cash do they have in the bank account. And so they go back to the venture capitalist who"
  },
  {
    "startMs": 521208,
    "text": "says, \"Oh, I'm gonna fund at the model layer, the application layer, the inference layer, because I don't wanna fund all of that CapEx stuff.\" Um, and so the venture capitalist then funds all the CapEx stuff by giving the, um, the new neo-lab or new inference company, um, the check that they then send to the cloud, um, which is the deposit for the cluster, and that's how you underwrite the customer. So, um, the clusters then are delayed because, um, the cus--"
  },
  {
    "startMs": 551248,
    "text": "the cloud couldn't actually deploy the cluster until they had the customer in hand, and so that was the riskiest point. So therefore, um, what should have happened was the cloud had bought the cluster ahead of time and then waited until it was landed in the data center and then sold it, and that means you have way less likelihood of delays. Meanwhile, the cluster is very likely to break when you get the cluster as the customer because you don't have liquidity. And so if the failure rate is higher than what you expect and you don't have the ability"
  },
  {
    "startMs": 581148,
    "text": "to transition a node in, then you don't have anywhere you can pull nodes from, which means that if you need new nodes to make up for the failed ones, you have to RMA them, which means you have to get a new node. And then that gets us back to several steps before  where you have liqui-- uh, the lead time problem again. And so everything breaks for everybody all the time. Once you have gotten through all of that as a new AI lab, um, your problems are you, you, you know, you... Nobody was sold-- Like, everyone was sold out. Um, you then had to wait a lead time. You paid this big"
  },
  {
    "startMs": 610908,
    "text": "deposit. Um, the thing was delayed, and then it broke. And finally, you got through all of that, um, and the likelihood that you predict your demand correctly is very low. And so you're either gonna be way over-provisioned or way under-provisioned, and you have no way out. Um, there's no way to sell your capacity to somebody else, and there's no way to buy more capacity if you need it. So compute markets. Um, to solve these problems, you need to solve a bunch of different problems all at once. Um, you need to underwrite the value of the GPU cluster rather than underwriting the value or underwriting the credit risk of the customer."
  },
  {
    "startMs": 640928,
    "text": "Um, this is how it could work. Um, it's not guaranteed to work like this. In other industries, what happened is everything just consolidated. You got one big credit-worthy customer that does the whole damn industry. That's what the path of AI is currently on track to do. It's instead of going to be, um, a liquid market where anybody can participate, it's gonna look like TSMC. You're gonna get one buyer, um, or a few buyers that are the clouds. You're gonna get a few model labs that are the AI labs, and then"
  },
  {
    "startMs": 670888,
    "text": "you won't get other startups. Um, this is what we think is gonna happen, and this is why three years ago, when we were doing our AI lab, we had a chance. We could have said, \"No, we're not gonna do our new, um... W-we're not gonna, like, you know, stick it out for several years just trying to keep our company alive. Um, we're just gonna go do a new A-AI lab.\" But we saw this problem, and we said, \"Oh, shit. If we don't do anything about this, then we will just hit this three years from now.\" And here we are three years from now hitting this problem and where"
  },
  {
    "startMs": 700748,
    "text": "everyone's gonna hit this problem. So, um, as... If you watched the last talk, um, it's very hard to do-- solve this problem unless you are operating both the market and the clusters at the same time. Um, so this is Dylan Page. Um, Dylan is the co-chair of the CNCF Infrastructure Group. That's the organization that makes Kubernetes. Formerly worked at, um, Lambda doing infrastructure there. Um, and the reason-- And he joined San Francisco Compute because, um, GPU compute Isn't sufficiently standardized or reliable for the market operator to be divorced from the physical"
  },
  {
    "startMs": 730512,
    "text": "infrastructure. It doesn't work if you aren't the one who's buying the thing. Or sorry, it doesn't work if the person you're buying from isn't operating it. To give an example of why this doesn't work is I'm sure some of you are flying in from overseas or sorry, um, out of town, um, and you have a hotel room. Um, wouldn't it be great if you could, you know, sell that hotel room to somebody else? I go to you, and I wanna buy that hotel room, um, and I say, \"Can I get room service? Can I get my room cleaned? Um, I've got a problem with my room."
  },
  {
    "startMs": 760492,
    "text": "Who do I call?\" Um, if you don't operate the hotel, you can't fix any of those problems. That's why brokering clusters doesn't work. You have to operate the cluster like the hotel. You have to run the whole damn thing, otherwise you can't actually make a market. It doesn't work. You can't do physical settlement. Um, so why does nobody do physical settlement? Um, the problem is there's a very strong incentive to never do physical settlement. Um, that's because if you build out physical settlement for compute, what that literally means is you're building out the"
  },
  {
    "startMs": 790512,
    "text": "entire operating capacity of a GPU cloud. And if you are building out the entire operating capacity of a GPU cloud, you should just be a GPU cloud because it looks to be the case that you will make an ungodly amount of money. Um, you will sign billion-dollar deal after billion-dollar deal. And so fuck it. Let's not make a GPU cl-- or let's not make a compute market. Let's instead just become a GPU cloud, which is what basically everybody does because the numbers are astronomical, and it's a very reasonable thing to do. Um, the reason why we ended up"
  },
  {
    "startMs": 820472,
    "text": "building physical settlement was because we knew that if we didn't start this, um, we would then not be able to start an AI lab, which was the real damn thing that we wanted to do in the first place. We wanted to make a lab. We didn't get to make a lab unless this exists. So when we started, it was ten million, um, as the deposit that you had to pay, um, to start a new lab. That's just like how you get over the credit risk problem. Um, then the cluster sizes got bigger. The sort of minimum ante into the space got bigger. Then it was a hundred million. Now it's a billion."
  },
  {
    "startMs": 850412,
    "text": "We don't think it can continue. We don't think there is a world in which you can get a ten billion dollar round right out the bat before you raise, um-- or before you have revenue or traction or your product or anything. Um, and what that means is that if you don't solve this problem, we are currently on track to not have any startups in AI. Um, now, one of the things that you could think could happen, um, is maybe there can be new inference companies and new model labs, and the way that they solve the credit problem is that instead they just ship the risk to the application"
  },
  {
    "startMs": 880452,
    "text": "layer. They say, \"Look, I will, um, I, I need to get over my credit, um, problem, so what I'm gonna do is I'm gonna only sign customers who are investment-grade c-um, credit, you know, Fortune five hundred. Um, I'm gonna only sign customers on five-year contracts for tokens. I'm gonna only sign customers for tokens that require a big deposit up front.\" Um, that is the end of everything. Um, if you have inference layer"
  },
  {
    "startMs": 910472,
    "text": "getting bottlenecked by credit, what happens is you can't get tokens unless you have credit or you have a big deposit. And the problem of that is that tokens are used by the whole damn economy. There is no rule that says that startups have to exist. Um, if every startup is suddenly bottlenecked by credit, um, in order to access intelligence, what happens is existing firms just ship everything. They themselves get to make the product, um, because they've got credit, and then they make"
  },
  {
    "startMs": 940472,
    "text": "the revenue from that product, and then it makes their credit better. And that's a flywheel that makes consolidation happen throughout the entire economy. If you don't fix this bottleneck, there aren't any startups. This is why we started doing this and because at one point, um, uh-- or what we think is gonna happen is you're gonna end up in, like, three to five years, you're gonna look out and say, \"I wanna start a new startup.\" And to get there, you need to buy so many tokens in order to compete with the existing partner or, like, existing players."
  },
  {
    "startMs": 970892,
    "text": "So you need to buy a hundred million dollars of tokens. Um, we think this is what you are actually going to have to do in order to start a new company because other people are spending hundreds of millions of dollars of tokens. If you just draw the line out, this is what happens. So you need a big, giant deposit because you're two people who have no idea what they're doing, and so nobody trusts you. So you're gonna go and raise a hundred million dollars from your venture capitalist. Um, meanwhile, nobody wants to work with you because you're a two-person startup. Then you're gonna be asked to pay for, um, that hundred--"
  },
  {
    "startMs": 1000952,
    "text": "or you're gonna be asked to pay for a five-year contract on that hundred million, um, if you're not paying it all as a deposit, and then you just don't get new model labs. You don't get new startups. Um, you get a consolidation in the industry. So we, at one point, um, uh, were this little lab, um, and what we did is we lived in San Francisco, um, and we knew all the other people who were also just two little folks, um, who had started a startup and wanted to be able to participate in the most important technology of our time. Um,"
  },
  {
    "startMs": 1030212,
    "text": "so I want these people to exist. I want this person to exist, and I want this person to exist. I want this startup to exist and this startup to exist and this startup to exist. This one and this one and this one and this one and this one and this one and this one and this one. If you get these companies going, maybe they'll become big folks that build your cancer cures, or they build something that nobody thought of, um, or something that wasn't just sort of AI slopped out. Uh, this is kind of the problem that compute markets solve, and if you don't solve"
  },
  {
    "startMs": 1060212,
    "text": "it, uh, w-we don't think there's gonna be any startups."
  }
]