The Geopolitics of AI Infrastructure

Talk transcript

1. 0:00

   [on-hold music] So gonna talk about China, US Middle East, um, and then US infrastructure.

2. 0:21

   Um, and then a bit of a Q&A if we have time. I don't know. I, I yap a lot, so it's gonna be hard. Um, so s- Twenty minutes.

3. 0:26

   Yeah, yeah, yeah. Twenty minutes. Oh, I got twenty. I got twenty. That's plenty. Um- It's eleven now. Nine good to go. [laughs] Ti-time's ticking. Time's ticking. Should I just, should I just stall so I don't have Q&A?

4. 0:36

   Crowd's tough. Um, so, so wanted to talk about, like, Huawei's chips, right? 'Cause we never get to talk about them. Uh, their... Huawei is, is really, really cracked. Um, they absolutely destroyed everyone in 5G and telecom, um, by just engineering better.

5. 0:51

   Um, and now they've done something that's, that's super interesting. Um, so they have the Ascend 910B and C, which you may have heard of, which are chips that they make.

6. 1:01

   Um, and then they've turned it into this, like, really cool system architecture, right? Uh, you know, NVIDIA talks a lot about Blackwell NVL 72. It's one rack with seventy-two GPUs, um, connected together in NVLink.

7. 1:13

   Um, Huawei's done something similar. It's called the CloudMatrix 384. Three hundred and eighty-four of their ch-chips, right, connected together in not one rack, but, like, twelve. Um, and there's a ton of optics and power connecting them all.

8. 1:26

   What's interesting is that their architecture is actually one that NVIDIA tried to deploy and they failed at. Um, and that was called, uh, DGX H100 Ranger, right? Which was two hundred and fifty-six NVIDIA GPUs connected together in one NVLink network, uh, with optics.

9. 1:41

   Um, they, they tried, and it did not work. They could not bring it to production because it was expensive, power-hungry, and unreliable. Um, and, and that sort of was the impetus for why Blackwell, they instead stopped going, you know, hey, ten, twenty kilowatts per server to a hundred and twenty kilowatts in one server that is effectively one

10. 1:57

    whole rack because they wanted to keep it all in copper, right? Whereas DGX H100 Ranger, and also the Huawei system, uses optics to connect all the GPUs at super high bandwidth in the high bandwidth, you know, NVLink or, uh, a s- I can't remember what the, uh, uh, Ascend CloudMatrix pod is called, their, their scale-up network.

11. 2:15

    But it's quite interesting that Huawei was able to engineer something that NVIDIA effectively failed at, right? Um, now obviously this, this is, um, way more, you know, power hungry, of course.

12. 2:26

    Um, potentially unreliable. There's no data. Um, and it would be expensive except for the fact that, like, you know, China's really good at making things cheap, right? So th-that's, that's something interesting about the Huawei system.

13. 2:36

    Um, what's interesting about the geopolitics of this is that despite the fact that Huawei is a sanctioned entity, China is a sanctioned country, they're still a-- they were still able to actually access, uh, TSMC to manufacture these chips, right?

14. 2:49

    Um, the, the chip is manufactured through, uh, at TSMC through SoFGO, uh, which is a Bitcoin cryptocurrency mining company, uh, that pretended to not be attached to Huawei and just bought these chips.

15. 3:01

    Um, and then there's, you know, HBM, which is high-bandwidth memory from Samsung and Hynix in Korea. Um, and then all the equi-- all the equipment to package it together is from the US, Netherlands, and Japan.

16. 3:13

    So it's funny that sanctions are completely useless because they're still able to access this stuff. Um, and what's interesting is, you know, what, what I'm showing on the right is that they ca-- they've made, you know...

17. 3:22

    They have roughly three million chips worth. They have two point nine million chips worth, uh, from TSMC. That's allegedly stopped now, right? The US gave TSMC, like, a billion dollar fine, uh, for five hundred million dollars of revenue that they made.

18. 3:34

    Uh, which, which doesn't... s-seems like a slap on the wrist, honestly. Um, but you know, SMIC is also gonna s-- which is Ch-China's TSMC, is gonna start making it as well.

19. 3:43

    Um, the other thing I would say is, what's funny is that HBM was also banned to China entirely. But instead of just, like, "Hey, H-HBM's banned," Ch- there's, there's cool ways to circumvent this, right?

20. 3:54

    Which is, uh, Samsung sells to this company called CoAsia in Taiwan, and then CoAsia sells to this company called Faraday, which packages it into a chip. Like, there's a fake chip, basically, right?

21. 4:04

    A chip that does literally nothing, and it has HBM packaged on it. Then they ship it to China, and then they take the HBM off of that chip and then put it on the Ascend.

22. 4:12

    Um, and this actually circumvents all the rules and regulations, so this is completely legal, um, which I think is, like, very funny. Um, and so Huawei has stockpiled roughly thirteen million HBM stacks, um, you know, already.

23. 4:26

    Um, and they're continuing to receive more shipments, uh, which is fun. Uh, very, very interesting that, you know, that's kinda possible. The other thing is, um, you know, domestically, they have not been able to manufacture in the past, but now they're able to, right?

24. 4:38

    Um, China's, uh, SMIC, right, their TSMC, uh, has enough tools for fifty thousand wafers a month. Today, the only seven-nanometer chip that anyone's found from SMIC is their smartphone chip, right?

25. 4:50

    Smartphone chips are smaller, easier to make. Um, they yield better than large AI chips. Um, so when you look at, like, hey, five nanometer, the first five nanometer chip was an iPhone chip in twenty-twenty, but NVIDIA didn't release a five nanometer GPU until, uh, twenty-twenty-two, twenty-twenty-three with the H100, right?

26. 5:06

    Uh, and so likewise, right, SMIC is making seven nanometer for phones. Uh, they're going to get to the point where they can start making seven nanometer for AI chips, likely in very high volumes this year.

27. 5:16

    Um, and based on the yields, they can actually get, like, you know, millions and millions, right? So it's, it's, uh... You know, the, the thought that China will not have equivalent compute is, like, kind of wrong.

28. 5:26

    They will, they will have a lot of compute, right? Um, which is, which, which will be interesting, right? Because, uh, you know, there's already, like, big announcements from DeepSeek that they're gonna work and use, use Huawei chips, uh, to try and train their next generation models.

29. 5:38

    Um, and so that, that'll be really interesting. Um, the other thing that's interesting is, um, you know, recently NVIDIA got banned from selling their H20. This is like a cut down H100 or H200 to China.

30. 5:52

    Um, they wrote down five billion dollars worth of inventory, and on the earnings call, Colette, the CFO, said, "If we didn't have export restrictions, we would've sold fifty billion dollars worth of GPUs to China this year," which I thought was, like, a very interesting comment.

31. 6:04

    Um, but, uh, the, the ban stopped about a million GPUs, and, um, if you... Yeah, it, it, it's, it's, it's, it's quite interesting that, like, you know- That amount of compute was also blocked.

32. 6:17

    Um, I think the other sort of geopolitical thing that everyone's talking about besides China is the Middle East, right? Um, so recently there was a deal in the Middle East.

33. 6:26

    Uh, you know, Trump went to the Middle East. He didn't go to Israel. He only went to Saudi and UAE, which I thought was quite interesting. Um, you know, not to get political, but, you know, that's, that's interesting.

34. 6:35

    Um, [laughs] no, no more, no more, no more. [laughs]

35. 6:41

    Um, so the, the details of the deal are quite, uh, are cool, right? So on the right is a satellite photo of a data center complex in the UAE, uh, that G42 is making.

36. 6:49

    Um, and the deal is basically that G42 can buy five hundred thousand GPUs a year, um, and they get to keep twenty percent of them to do whatever they want.

37. 6:58

    The other eighty percent have to go to, uh, US hyperscalers and cloud companies and AI companies, right? And so, uh, G42 is building a five gigawatt data center campus.

38. 7:09

    The one photo-- pictured is the first parts of a gigawatt data center campus, right? Um, that is, that is absolutely ridiculously big, right? xAI's, uh, data center is like two hundred megawatts, right?

39. 7:21

    Um, you know, OpenAI and all these other guys are two hundred or less megawatts for the models they've released. So these are absolutely ginormous, right? Stargate, the first, uh, first six parts of it in total is one point two gigawatts, right?

40. 7:34

    So like this is a, this is a massive, um, data center that they're building. Um, G42 has a large customer and investor, Microsoft, and so Microsoft is going to be a big one.

41. 7:46

    But if everyone remembers back like in twenty twenty-three, Sam was always like, "Hey, seven trillion dollars," right? You know, as he's throwing this crazy-ass number out or everyone kept reporting about it.

42. 7:55

    Um, the interesting thing is that part of this was that, um, OpenAI wanted to build GPU clusters in the Middle East, right? Um, and so, uh, not, not like publicly stated, but it's like, it's like OpenAI is gonna have a cluster in the Middle East, right?

43. 8:11

    Um, and so that's a big part of this deal as well. Um, and, and in, in concession, right, the US gets a couple benefits, right? Uh, UAE is, uh, providing matching investments to, uh, US, right?

44. 8:24

    For any dollars they spend in UAE-- the UAE spends in UAE on AI infrastructure, they're also gonna spend in the US. Um, and so that's already started, right? G42 has sites in like Kentucky and New York, um, that they're spending in.

45. 8:38

    Nothing that's five gigawatts, but I'm sure they'll, they'll get something. Uh, we'll see if this like all follows through. Uh, the other thing is that most of the compute goes to, uh, US companies, right?

46. 8:46

    Eighty percent of it. Is this intended for inference or training? Like what's the goal of this data center? Um, I think, you know, you, you could spin it any way you want, right?

47. 8:57

    Um, but the lines are also blurring, right? Um, people today now, uh, in their inference clusters at night in the US are just running reinforcement learning with verifiable rewards, generating, you know, trajectories and then, you know, keeping the good tokens, uh, when there's low utilization, right?

48. 9:11

    So like inference clusters are now also training clusters, right? So I think, I think the- there's like the lines are blurring between what is an inference and training cluster.

49. 9:17

    But also this is a five gigawatt data center. Like that's, that's, that's big. Um, you could do training for sure. Let's, let-- Can we keep questions? Yeah, sure, sure, sure, sure.

50. 9:26

    Yeah. We can actually do questions in this round, but come up to him afterwards. Sorry. Um, no worries, no worries. Uh, the other one, the other one is, uh...

51. 9:35

    I, I, I think, I think if we want to, we can yell questions, you know. Shh. [laughs] Um, the other one is, uh, the Kingdom of Saudi Arabia. Um, Data Vault is a company there.

52. 9:45

    Uh, if you've heard of The Line, uh, in Saudi Arabia, it's an absolutely ridiculous project. It's actually really cool. They're building a city that's a straight up line, and on both sides they have like huge sets of like mirrors so they don't have a ton of sun like making the city way too hot and it's clean and

53. 10:00

    whatever, right? Like it's a really interesting thing. There's a lot of YouTube slop videos that you could watch that show how cool it is. Um, but anyways, part of that, uh, Line project is also Data Vault who's making a data center.

54. 10:12

    And this is real and this is definitely happening. They've already b- struck, uh, broke ground on like a two gigawatt data center. Um, and again, like for context, xAI's entire training infrastructure at, today is two hundred megawatts, right?

55. 10:25

    Uh, and they've spent like ten plus billion dollars on it, right? Like you gotta, you gotta rationalize these numbers. Uh, it's like ten X more, right? Um, so, so Data Vault's gonna invest twenty billion dollars in US data centers in addition to building a bunch in, um, the Middle East.

56. 10:40

    Um, and then, and then a bunch of American companies are gonna invest as well. Uh, so total investment number's like eighty billion dollars. Uh, Data Vault is the data center company for Humane, which is, uh, sort of the clou- neo cloud for the data center company, right?

57. 10:55

    Um, and so Humane, uh, they've signed deals to like get custom CPUs from Qualcomm and buy a bunch of AMD GPUs as well as NVIDIA GPUs. Uh, Humane is actually the Aramco, uh, folks, the Aramco, Aramco Digital folks, which is the vast majority of Groq's revenue as well.

58. 11:12

    So there's, uh, likely Groq stuff in there. Um, I think like ninety plus percent of Groq's revenue or, or s-- uh, they've got fun- money from them is coming, coming from, uh, from, from the s- Aramco Digital folks.

59. 11:24

    Um, likewise, um, the, the, you know, there-- And then there's also an AWS thing, right? So there's all, all these American companies are also like coming together with the Middle East.

60. 11:33

    So the question is like, you know, why the heck are we sending all these GPUs to the Middle East? Is what like the EAs would say. And then the like capitalists will be like, "Fuck yeah, GPUs and money," right?

61. 11:44

    So like it depends on what side of the fence you're on. Um, and I like to think, you know, I'm, uh, I'm on one side of the fence. I won't tell you which side. [laughs]

62. 11:52

    Uh, you could guess. You could guess. Um, so you know, there, there are real criticisms, right? Like what if these GPUs get smuggled, right? To China. Um, 'cause it's not like Saudi Arabia and, and the UAE are allies.

63. 12:03

    Um, they are, uh, they, they conveniently play both sides, right? Um, and, and always have. So there, there's always that risk. There's all these security requirements like, okay, what if-- Okay, they're not-- Maybe the GPUs don't get smuggled there.

64. 12:16

    Maybe they get rented to China, right? Um- You know, there's the argument that these GPUs would've been used in the US anyways. Um, and so, and, and, and, and we know that export restrictions and things like that, the enforcement of them sucks.

65. 12:28

    So it's not like w- the US government apparatus is gonna actually be able to effectively enforce this, right? So there's, there are real risks here. Um, and, and also, like, giving power to authoritarian count- countries and, that have, like, you know, kingdoms, right?

66. 12:42

    It's, it's kingdom of Saudi Arabia. It's princes, right? You know, it's a little bit odd. Um, so, so that's sort of the arguments against it. Um, and then, and then the supporter are obviously, right, like, "Yes, more GPUs."

67. 12:53

    Most of these GPUs go to American companies, right? Like, an easy example is OpenAI, right? Uh, they, they, they want a lot, lot more GPUs. Uh, Microsoft-OpenAI deal sort of split up partially because OpenAI wanted so many GPUs, and Microsoft was like, "No, you're crazy.

68. 13:09

    Like, that's way too much. Like, you don't have revenue." Um, and OpenAI's like, "But we can convince someone to build it for us," right? 'Cause OpenAI's really nice, right?

69. 13:17

    Like, they get to rent the GPUs. Uh, and so, like, but the thing is, their provider has to build the data center and buy all the GPUs, and then they don't get payback for two or three years, right, of renting GPUs, even though the contract is for, like, five years.

70. 13:29

    So yeah, there's all this theoretical profit, but you don't actually get the money back on the cluster until year two or three. So if Microsoft now builds a $100 billion data center for OpenAI, and then OpenAI doesn't raise $150 billion to pay for it, um, then, then, you know, Microsoft is just left holding the bag.

71. 13:47

    And who wants that big of a data center, right? So unless you believe, like, AI's demand is, like, unlimited, then, you know, you kind of... There's a big risk here.

72. 13:53

    And the nice thing is, like, you know, fools are easily parted with their money, um, and SoftBank and the Middle East are the i- most stupid investors in the world potentially. [audience laughing]

73. 14:02

    Um, and so, you know, I actually, I think they're fine. I think they're fine. I think these are good deals. But, like, you know, [laughs]

74. 14:08

    um, you know, OpenAI gets to have someone pay for the cluster, um, buy all these GPUs, build all the data centers, and then, like, with the promise they're gonna rent them, right?

75. 14:16

    And the same applies to all the other AI labs across the, uh, West, right? Um, and so the argument that, like, I would make is that, you know, OpenAI, it, because of this, will have more compute in 2027 or 2028 than they would've had without it, right?

76. 14:30

    It wouldn't have been built in the US. Also, there's big problems with America in terms of being able to build GPUs, right? Um, and that is that there is a massive deficit of power, right?

77. 14:41

    Um, so, uh, uh, there's no axis on the chart on the left for a reason, all right, for people taking a picture. Um, but on the right, you know, M- Middle East is building, you know, by 2030, like four gigawatts, right?

78. 14:51

    Uh, uh, I said they promised five, but, like, from what we see, they'll have, like, four by 2030. Um,

79. 14:57

    the US has a humongous, humongous deficit on, um, power. Um, I will skip forward a bit. Um, power, China's good at making power. US is not, right? This is an interesting one, right?

80. 15:08

    Um, so presented to this to, uh, to the Secretary Wright and Bergman in February, um, i, i- in, in DC, and it's a really interesting chart, right? It's like, so on the right is, like, sort of our data center data, right?

81. 15:21

    It's saying there is a 63 gigawatt da- uh, shortfall of power in the US, right, based on the data centers that we see under construction. And we're tracking every single one site by site, et cetera, building by building, um, with satellite photos, permits, regulatory filings, all this stuff, right?

82. 15:35

    And on the left is the US power grid, right? Um, so you, you can have a lot of, like, strong assumptions about renewables, about batteries, um, and then all of the single cycle and dual cycle gas reactors that are being installed.

83. 15:47

    And then you say, "Hey, actually, almost none of the coal's gonna turn off that is planned to turn off." And you still have this massive, mass... You have 44 gigawatts being added, but you have a hun- um, on, on the power side, um, and that's, that's assuming, you know, w- with the fluctuations and all that, right?

84. 16:02

    Uh, whereas you're adding 100 gigawatts of data center capacity. So the US simply just doesn't have enough power unless we do something. And so the argument sort of is like, well, geopolitically, the US can't build, um, enough data centers, can't build enough power, whether it's for the lack of skilled labor, whether it's for the lack of, you

85. 16:18

    know, regulatory regulators holding stuff up. Um, federal, not as much anymore, but local, state, and a- also in America, we have this beautiful thing called utility companies that are regulated monopolies that get to do whatever they want, right?

86. 16:31

    If anyone has a power bill in California, they understand that the utilities fucking suck. Um, anyways, um, there, the, there's all these issues with building power in the US, right?

87. 16:42

    And if we go back, China has no problem, right? They added an entire US grid in seven years, right? Um, and so, and we're talking about adding, you know, you see four terawatt hours on the, on the right graph.

88. 16:52

    I'm talking about adding 100 gigawatts as a problem, right? Um, because the US just doesn't know how to,

89. 16:58

    uh, or rather hasn't done it in, in, like, four decades. Um, and so that's the big sort of, uh, challenge. So for every Stargate that's out there, right, um, you know, 220 megawatts for these four building, like, things, and there's f- two of them already com- almost completed, and then there's another six going up.

90. 17:14

    Uh, that's 1.2 gigawatts total for Stargate. Um, for every one of these that's happening, there's so many projects that are failing, right? Like, I've literally pitched a coal stock to my clients in Ohio, and the stock's 3X because there's power issues.

91. 17:26

    Like, like, it's, it's crazy how, um, how power constrained America is, right? And so the geopolitics here is like, do you build in the Middle East or not? Uh, China's got p- no pro- power problems, even if their chips are less efficient, doesn't really matter, right?

92. 17:40

    Like, I think the AI race is a very geopolitical and interesting one. Um, and so I'll sort of leave it there. I know there's, there's, there's a couple more slides, but yeah.

93. 17:49

    Couple questions, two minutes. [audience applauding] Is SMIC dependent on ASML or are you working on an all- SMIC, SMIC, China's TSMC, is, uh, dependent entirely on Western tools today. Uh, but there are s- many, uh, Chinese tool companies.

94. 18:06

    What's interesting is they'll buy billions of dollars of, uh, American, Japanese, and Dutch equipment, including ASML, but then they'll put it next to their domestic tool, and then they'll run, uh, wafers through both, and then they'll just, like, learn how to improve it.

95. 18:18

    They'll also tear them down and reverse engineer them. Um, yeah, so it's cool. [upbeat music]