[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12680,
    "text": ">> Welcome everybody."
  },
  {
    "startMs": 14240,
    "text": "Sorry for everybody who was already here"
  },
  {
    "startMs": 16240,
    "text": "and missed the Coinbase guy. I have no"
  },
  {
    "startMs": 17720,
    "text": "idea where he went or why he didn't"
  },
  {
    "startMs": 19800,
    "text": "come. I was actually pretty excited to"
  },
  {
    "startMs": 21400,
    "text": "hear about his his comments. But today"
  },
  {
    "startMs": 24320,
    "text": "I'm going to talk about which AI"
  },
  {
    "startMs": 25520,
    "text": "startups actually win enterprise"
  },
  {
    "startMs": 27000,
    "text": "contracts."
  },
  {
    "startMs": 28160,
    "text": "So to begin"
  },
  {
    "startMs": 30840,
    "text": "I thought this was going to be a"
  },
  {
    "startMs": 31720,
    "text": "different audience. I didn't realize"
  },
  {
    "startMs": 32920,
    "text": "this was going to be mostly people on"
  },
  {
    "startMs": 34560,
    "text": "the leadership track. I thought I was"
  },
  {
    "startMs": 36080,
    "text": "going to be speaking to more AI"
  },
  {
    "startMs": 37600,
    "text": "engineers. So maybe just by show of"
  },
  {
    "startMs": 39520,
    "text": "hands, how many of you represent like"
  },
  {
    "startMs": 41480,
    "text": "the engineering or startup or like"
  },
  {
    "startMs": 43080,
    "text": "seller side?"
  },
  {
    "startMs": 45000,
    "text": "And then how many of you represent maybe"
  },
  {
    "startMs": 46360,
    "text": "like the buyer side? Like you're in the"
  },
  {
    "startMs": 48000,
    "text": "enterprise, you're trying to get these"
  },
  {
    "startMs": 49360,
    "text": "tools in. Okay, so we got a good mix."
  },
  {
    "startMs": 51840,
    "text": "I'm going to try to balance that out"
  },
  {
    "startMs": 53200,
    "text": "today. I work on product stuff at"
  },
  {
    "startMs": 55400,
    "text": "Millennium, which is a hedge fund. We"
  },
  {
    "startMs": 57600,
    "text": "build a lot of stuff. I can't talk about"
  },
  {
    "startMs": 59520,
    "text": "any of it. So I'm going to talk about"
  },
  {
    "startMs": 61000,
    "text": "stuff that"
  },
  {
    "startMs": 62280,
    "text": "we we look at and evaluate. It's going"
  },
  {
    "startMs": 64440,
    "text": "to be pretty generic. I tried to make it"
  },
  {
    "startMs": 66360,
    "text": "as interesting as possible while still"
  },
  {
    "startMs": 67960,
    "text": "getting my compliance department to be"
  },
  {
    "startMs": 69560,
    "text": "okay with me doing this."
  },
  {
    "startMs": 71280,
    "text": "But also need to say legally that I'm"
  },
  {
    "startMs": 73800,
    "text": "speaking as an individual. I am not"
  },
  {
    "startMs": 75440,
    "text": "representing my company and all opinions"
  },
  {
    "startMs": 77480,
    "text": "are my own."
  },
  {
    "startMs": 78720,
    "text": "So with that we can dive in."
  },
  {
    "startMs": 81880,
    "text": "So I ran through this with my parents"
  },
  {
    "startMs": 84280,
    "text": "last week. I don't I grew up not that"
  },
  {
    "startMs": 86240,
    "text": "far from here. And my mom basically"
  },
  {
    "startMs": 88200,
    "text": "said, \"Why are you spending your time"
  },
  {
    "startMs": 89800,
    "text": "teaching vendors how to sell to you?"
  },
  {
    "startMs": 91480,
    "text": "Aren't you busy enough already?\""
  },
  {
    "startMs": 93720,
    "text": "And the real answer to that question is"
  },
  {
    "startMs": 96160,
    "text": "I really like how stuff works. I like"
  },
  {
    "startMs": 98240,
    "text": "seeing stuff come together. My"
  },
  {
    "startMs": 100360,
    "text": "bachelor's degree was in economics. And"
  },
  {
    "startMs": 102640,
    "text": "I really love seeing things just work"
  },
  {
    "startMs": 104200,
    "text": "well. So AI's been really interesting"
  },
  {
    "startMs": 106920,
    "text": "because it's kind of a whole new"
  },
  {
    "startMs": 108880,
    "text": "paradigm of how businesses are doing"
  },
  {
    "startMs": 110560,
    "text": "work. That's the whole point of this"
  },
  {
    "startMs": 112120,
    "text": "track, this AI native enterprise track."
  },
  {
    "startMs": 114480,
    "text": "And so even though I don't need more"
  },
  {
    "startMs": 117120,
    "text": "people DMing me on LinkedIn, um I'm"
  },
  {
    "startMs": 119920,
    "text": "actually really excited to talk about"
  },
  {
    "startMs": 121320,
    "text": "this."
  },
  {
    "startMs": 122560,
    "text": "So, my hypothesis in short is basically"
  },
  {
    "startMs": 125280,
    "text": "at current model intelligence, most of"
  },
  {
    "startMs": 127280,
    "text": "the value available is already being"
  },
  {
    "startMs": 129119,
    "text": "left on the table. Um this is not a hot"
  },
  {
    "startMs": 131280,
    "text": "take for most people, I think who work"
  },
  {
    "startMs": 133360,
    "text": "in enterprise. You've probably seen this"
  },
  {
    "startMs": 135240,
    "text": "problem. This little stat at the bottom"
  },
  {
    "startMs": 137560,
    "text": "is uh pretty heavily uh repeated for a"
  },
  {
    "startMs": 140560,
    "text": "lot of people who work inside of"
  },
  {
    "startMs": 141680,
    "text": "business circles, and they all kind of"
  },
  {
    "startMs": 142840,
    "text": "like laugh, and they're like, \"Yeah,"
  },
  {
    "startMs": 144320,
    "text": "yeah, you know, all these AI tools, how"
  },
  {
    "startMs": 146320,
    "text": "much are they actually doing?\" Um and I"
  },
  {
    "startMs": 148320,
    "text": "want to talk about why. So, there's kind"
  },
  {
    "startMs": 151040,
    "text": "of two sides to this becoming gen AI"
  },
  {
    "startMs": 153160,
    "text": "native. Um you have models and products,"
  },
  {
    "startMs": 155440,
    "text": "which are one side, that's the seller"
  },
  {
    "startMs": 156880,
    "text": "side, and then you also have systems and"
  },
  {
    "startMs": 159000,
    "text": "all of what's inside of the enterprise,"
  },
  {
    "startMs": 160400,
    "text": "that's the buyer side. So, that's the"
  },
  {
    "startMs": 161920,
    "text": "side that I deal with a lot."
  },
  {
    "startMs": 164560,
    "text": "Um so, we're going to talk first about"
  },
  {
    "startMs": 165720,
    "text": "the seller side, and then we're going to"
  },
  {
    "startMs": 166760,
    "text": "talk about the buyer side. So, per pain"
  },
  {
    "startMs": 168760,
    "text": "point, uh a lot of my job is kind of go"
  },
  {
    "startMs": 170600,
    "text": "around the company and figure out like"
  },
  {
    "startMs": 172320,
    "text": "what are the pain points? Uh what are we"
  },
  {
    "startMs": 173800,
    "text": "trying to solve for? Uh can we buy it?"
  },
  {
    "startMs": 175920,
    "text": "Can we build it?"
  },
  {
    "startMs": 177640,
    "text": "So, let's say for a given pain point,"
  },
  {
    "startMs": 180120,
    "text": "maybe I identify 10 to 15 startups that"
  },
  {
    "startMs": 182480,
    "text": "look really interesting. Like, \"Huh,"
  },
  {
    "startMs": 184120,
    "text": "maybe these guys can solve our problem"
  },
  {
    "startMs": 185320,
    "text": "for us, we don't have to build it.\""
  },
  {
    "startMs": 187239,
    "text": "Um of those, after doing a little bit of"
  },
  {
    "startMs": 189760,
    "text": "due diligence on my own, I might"
  },
  {
    "startMs": 191640,
    "text": "schedule two to three demo calls."
  },
  {
    "startMs": 194519,
    "text": "Of those, we probably will land zero or"
  },
  {
    "startMs": 197519,
    "text": "one pilots."
  },
  {
    "startMs": 199200,
    "text": "And of those,"
  },
  {
    "startMs": 201200,
    "text": "probably one in four of those longer"
  },
  {
    "startMs": 203040,
    "text": "term will actually end up with a"
  },
  {
    "startMs": 204320,
    "text": "contract."
  },
  {
    "startMs": 205640,
    "text": "So, what does this mean? This means"
  },
  {
    "startMs": 206600,
    "text": "about 5% of all of our demo calls"
  },
  {
    "startMs": 209560,
    "text": "actually end up in a signed contract. Um"
  },
  {
    "startMs": 211680,
    "text": "and this tracks with the industry. I had"
  },
  {
    "startMs": 213200,
    "text": "no idea that this was actually a"
  },
  {
    "startMs": 214760,
    "text": "benchmark, um but it turns out that"
  },
  {
    "startMs": 217400,
    "text": "there's quite a bit out there that"
  },
  {
    "startMs": 218519,
    "text": "indicates that this is really similar"
  },
  {
    "startMs": 220080,
    "text": "across the board."
  },
  {
    "startMs": 223239,
    "text": "So, I want to talk about what enterprise"
  },
  {
    "startMs": 224920,
    "text": "ready actually means from the inside, uh"
  },
  {
    "startMs": 227080,
    "text": "because we have a lot of startups that"
  },
  {
    "startMs": 228080,
    "text": "tell me what enterprise ready means, and"
  },
  {
    "startMs": 230000,
    "text": "And we go through all of our"
  },
  {
    "startMs": 231120,
    "text": "requirements, and then we have a very"
  },
  {
    "startMs": 232840,
    "text": "different idea of what enterprise ready"
  },
  {
    "startMs": 234480,
    "text": "actually means. Um so we're going to"
  },
  {
    "startMs": 236360,
    "text": "talk about what breaks down and why."
  },
  {
    "startMs": 239640,
    "text": "So 40% of this is efficacy, so just"
  },
  {
    "startMs": 242320,
    "text": "value, uh commercial issues."
  },
  {
    "startMs": 244800,
    "text": "Then there's a lot that dies in"
  },
  {
    "startMs": 246200,
    "text": "security. There's other things that die"
  },
  {
    "startMs": 248000,
    "text": "in reliability, and then there's some"
  },
  {
    "startMs": 249600,
    "text": "stuff that dies in legal."
  },
  {
    "startMs": 252200,
    "text": "So we're going to start with the"
  },
  {
    "startMs": 253280,
    "text": "requirements that we put forward and"
  },
  {
    "startMs": 254760,
    "text": "then some of the things that we've seen"
  },
  {
    "startMs": 255920,
    "text": "go wrong across various AI companies"
  },
  {
    "startMs": 258000,
    "text": "that we work with."
  },
  {
    "startMs": 259480,
    "text": "So our requirements for efficacy, maybe"
  },
  {
    "startMs": 261720,
    "text": "unsurprisingly, the product actually"
  },
  {
    "startMs": 263160,
    "text": "needs to solve the problem."
  },
  {
    "startMs": 264919,
    "text": "Um that seems pretty clear, but that's"
  },
  {
    "startMs": 267000,
    "text": "not always super clear."
  },
  {
    "startMs": 269080,
    "text": "Uh the next one is pricing models that"
  },
  {
    "startMs": 271200,
    "text": "need to reflect real value."
  },
  {
    "startMs": 273919,
    "text": "Clear demonstration of integrations on"
  },
  {
    "startMs": 275720,
    "text": "day one, not a hypothetical."
  },
  {
    "startMs": 278480,
    "text": "And we define the success criteria, not"
  },
  {
    "startMs": 281000,
    "text": "the vendor."
  },
  {
    "startMs": 282760,
    "text": "So things we've seen go wrong, uh"
  },
  {
    "startMs": 284200,
    "text": "vaporware in short. Um we've had a lot"
  },
  {
    "startMs": 286560,
    "text": "of startups who come in, they pitch us"
  },
  {
    "startMs": 288240,
    "text": "an idea, uh and it's something that our"
  },
  {
    "startMs": 290280,
    "text": "platform team can rebuild in about 6"
  },
  {
    "startMs": 291919,
    "text": "weeks. So this is not a knock, uh this"
  },
  {
    "startMs": 294520,
    "text": "is actually just what's going on in the"
  },
  {
    "startMs": 296480,
    "text": "industry everywhere, um on all sides of"
  },
  {
    "startMs": 298880,
    "text": "the equation. Um sometimes it's actually"
  },
  {
    "startMs": 301040,
    "text": "better for us to build, and sometimes it"
  },
  {
    "startMs": 302400,
    "text": "is still better for us to buy even if we"
  },
  {
    "startMs": 303840,
    "text": "could rebuild."
  },
  {
    "startMs": 306320,
    "text": "Upside down pricing, so this one's"
  },
  {
    "startMs": 307640,
    "text": "crazy. Um"
  },
  {
    "startMs": 309000,
    "text": "We had a startup just recently tell us,"
  },
  {
    "startMs": 311400,
    "text": "\"Hey, um we know that all of the LLM"
  },
  {
    "startMs": 313600,
    "text": "traffic that we're using for our wrapper"
  },
  {
    "startMs": 316440,
    "text": "is passing through your LLM gateway, but"
  },
  {
    "startMs": 319080,
    "text": "we want you to report your gateway"
  },
  {
    "startMs": 321000,
    "text": "telemetry to us so that we can then"
  },
  {
    "startMs": 323840,
    "text": "price a huge margin on top of that, even"
  },
  {
    "startMs": 326480,
    "text": "though none of it's running through our"
  },
  {
    "startMs": 327919,
    "text": "infrastructure.\""
  },
  {
    "startMs": 329120,
    "text": "Um"
  },
  {
    "startMs": 330080,
    "text": "that did not work."
  },
  {
    "startMs": 331640,
    "text": "Another one is promises in demo calls,"
  },
  {
    "startMs": 333680,
    "text": "but no ETAs after 2 months. Um this is"
  },
  {
    "startMs": 335960,
    "text": "pretty common. Um"
  },
  {
    "startMs": 337800,
    "text": "not a lot to say here."
  },
  {
    "startMs": 340000,
    "text": "Um and then repitching features we've"
  },
  {
    "startMs": 341600,
    "text": "already declined. So"
  },
  {
    "startMs": 343640,
    "text": "if you're a salesperson, um"
  },
  {
    "startMs": 345840,
    "text": "my best advice to you is listen to your"
  },
  {
    "startMs": 347120,
    "text": "customers. It's not novel, but uh it"
  },
  {
    "startMs": 349280,
    "text": "still seems to be a struggle for some."
  },
  {
    "startMs": 351480,
    "text": "Uh it's really just better to address"
  },
  {
    "startMs": 352960,
    "text": "the things that we've asked for. So, the"
  },
  {
    "startMs": 354720,
    "text": "other thing I want to point out at the"
  },
  {
    "startMs": 355640,
    "text": "very bottom of this slide is the pilot"
  },
  {
    "startMs": 357320,
    "text": "window collapsing. So,"
  },
  {
    "startMs": 359640,
    "text": "uh I've been at Millennium for a little"
  },
  {
    "startMs": 360840,
    "text": "over 2 years, and when I started, a lot"
  },
  {
    "startMs": 363240,
    "text": "of these pilot timelines that people"
  },
  {
    "startMs": 364480,
    "text": "were used to were like, \"Oh, maybe we'll"
  },
  {
    "startMs": 365800,
    "text": "run a pilot for 6 months.\" And then, not"
  },
  {
    "startMs": 368200,
    "text": "that long after that, it was like, \"Oh,"
  },
  {
    "startMs": 370000,
    "text": "maybe we only need it for 3 months.\" And"
  },
  {
    "startMs": 372680,
    "text": "anymore, it's like, \"Maybe we can do"
  },
  {
    "startMs": 374320,
    "text": "this pilot for 2 weeks.\" Uh because it's"
  },
  {
    "startMs": 376480,
    "text": "just accelerated so rapidly."
  },
  {
    "startMs": 380000,
    "text": "Um so, then moving on to security. Uh"
  },
  {
    "startMs": 381840,
    "text": "this is a huge one. I'm not a security"
  },
  {
    "startMs": 383360,
    "text": "expert, but I do run kind of frontline"
  },
  {
    "startMs": 385240,
    "text": "defense on talking to a lot of startups"
  },
  {
    "startMs": 387120,
    "text": "about security. And so, these are a lot"
  },
  {
    "startMs": 388800,
    "text": "of the things that that come up over and"
  },
  {
    "startMs": 390360,
    "text": "over."
  },
  {
    "startMs": 391400,
    "text": "Uh ZDR. So, this is a really hot topic."
  },
  {
    "startMs": 393919,
    "text": "Obviously, a lot going on with Fable, uh"
  },
  {
    "startMs": 396120,
    "text": "mandatory data retention requirements,"
  },
  {
    "startMs": 398360,
    "text": "uh and then a whole other battleground"
  },
  {
    "startMs": 400080,
    "text": "around customer-managed encryption keys."
  },
  {
    "startMs": 402640,
    "text": "So, ZDR is always best, of course. If"
  },
  {
    "startMs": 405880,
    "text": "that's not possible, customer-managed"
  },
  {
    "startMs": 407680,
    "text": "encryption keys"
  },
  {
    "startMs": 409120,
    "text": "and, with a big parentheses, that don't"
  },
  {
    "startMs": 410960,
    "text": "break the product. Um there are a lot of"
  },
  {
    "startMs": 412960,
    "text": "things that people are like, \"Oh, yeah,"
  },
  {
    "startMs": 414200,
    "text": "it's fine. It'll work with"
  },
  {
    "startMs": 415040,
    "text": "customer-managed encryption keys.\""
  },
  {
    "startMs": 416880,
    "text": "And then, it breaks the product. Uh so,"
  },
  {
    "startMs": 418720,
    "text": "that's a big product uh issue that we"
  },
  {
    "startMs": 420880,
    "text": "have to work through with people."
  },
  {
    "startMs": 423080,
    "text": "Other requirements, bring your own"
  },
  {
    "startMs": 424440,
    "text": "gateway. We prefer to route all of our"
  },
  {
    "startMs": 426000,
    "text": "own traffic through our own gateway and"
  },
  {
    "startMs": 428000,
    "text": "BYO infrastructure. Uh we would prefer"
  },
  {
    "startMs": 430160,
    "text": "to host it in our own cloud"
  },
  {
    "startMs": 431240,
    "text": "infrastructure and have something that's"
  },
  {
    "startMs": 432680,
    "text": "deployable in our systems."
  },
  {
    "startMs": 435720,
    "text": "This is another really big one. Um"
  },
  {
    "startMs": 437360,
    "text": "SCIM-tied RBAC. So, for all of you who"
  },
  {
    "startMs": 440720,
    "text": "who get that jargon, um"
  },
  {
    "startMs": 442480,
    "text": "it's really important that we can tie"
  },
  {
    "startMs": 444120,
    "text": "our AD groups or other permission and"
  },
  {
    "startMs": 446480,
    "text": "entitlement groups to role-based access"
  },
  {
    "startMs": 448840,
    "text": "control. We want to make sure that we"
  },
  {
    "startMs": 450480,
    "text": "don't just turn on features for"
  },
  {
    "startMs": 451800,
    "text": "everybody across the board. A lot of"
  },
  {
    "startMs": 453560,
    "text": "people don't think about this when"
  },
  {
    "startMs": 454760,
    "text": "they're designing their systems. They're"
  },
  {
    "startMs": 455800,
    "text": "like, \"Oh, this is a great feature. We"
  },
  {
    "startMs": 457440,
    "text": "should just turn it on for everybody.\""
  },
  {
    "startMs": 459160,
    "text": "Um when you work at a a"
  },
  {
    "startMs": 460720,
    "text": "enterprise, that's not something that"
  },
  {
    "startMs": 462440,
    "text": "people want to do. Um there are usually"
  },
  {
    "startMs": 464600,
    "text": "different groups who should have"
  },
  {
    "startMs": 465480,
    "text": "different access at different times, and"
  },
  {
    "startMs": 467360,
    "text": "most of all we want it to be"
  },
  {
    "startMs": 468280,
    "text": "configurable via API."
  },
  {
    "startMs": 471440,
    "text": "Um"
  },
  {
    "startMs": 472400,
    "text": "for smaller companies, we want to see at"
  },
  {
    "startMs": 474120,
    "text": "least one real security hire. So, this"
  },
  {
    "startMs": 477160,
    "text": "is something that's really important. We"
  },
  {
    "startMs": 478280,
    "text": "know that security is not the first"
  },
  {
    "startMs": 480240,
    "text": "thing that people hire for. Um but in"
  },
  {
    "startMs": 482160,
    "text": "the age of AI, this is a very real"
  },
  {
    "startMs": 483960,
    "text": "problem, and we need to make sure that"
  },
  {
    "startMs": 485520,
    "text": "the startups we're working with actually"
  },
  {
    "startMs": 486800,
    "text": "have somebody who can understand what's"
  },
  {
    "startMs": 488720,
    "text": "going on from the security standpoint."
  },
  {
    "startMs": 491040,
    "text": "Uh so, some of the things we've seen go"
  },
  {
    "startMs": 492200,
    "text": "wrong,"
  },
  {
    "startMs": 493280,
    "text": "um"
  },
  {
    "startMs": 494400,
    "text": "outright people just sending data to"
  },
  {
    "startMs": 496160,
    "text": "their vendors, uh cloud servers, and not"
  },
  {
    "startMs": 499440,
    "text": "following"
  },
  {
    "startMs": 500560,
    "text": "any of what we've asked for. Um this has"
  },
  {
    "startMs": 502919,
    "text": "been a problem in pilots. Uh thankfully,"
  },
  {
    "startMs": 504880,
    "text": "all of our pilots run non-production"
  },
  {
    "startMs": 506400,
    "text": "data."
  },
  {
    "startMs": 508520,
    "text": "Another one, like we kind of talked"
  },
  {
    "startMs": 509680,
    "text": "about, um read write all default scopes."
  },
  {
    "startMs": 512360,
    "text": "So, there's a lot of really cool tools"
  },
  {
    "startMs": 514320,
    "text": "out there, integrations, features."
  },
  {
    "startMs": 516760,
    "text": "They're really flashy. You can click a"
  },
  {
    "startMs": 518800,
    "text": "button, and it'll integrate with"
  },
  {
    "startMs": 520120,
    "text": "everything. And then you get a little"
  },
  {
    "startMs": 521960,
    "text": "bit deeper and find out the only way"
  },
  {
    "startMs": 523400,
    "text": "that it'll work is if you literally give"
  },
  {
    "startMs": 525200,
    "text": "it read write all to everything, uh"
  },
  {
    "startMs": 527320,
    "text": "which is a huge problem."
  },
  {
    "startMs": 529640,
    "text": "Another one, uh kind of along the same"
  },
  {
    "startMs": 531480,
    "text": "lines, all or new beta features on by"
  },
  {
    "startMs": 533680,
    "text": "default with each release. So, if you're"
  },
  {
    "startMs": 535680,
    "text": "an enterprise, you don't want everything"
  },
  {
    "startMs": 537400,
    "text": "just turned on with each release."
  },
  {
    "startMs": 540200,
    "text": "Um so, being able to control that,"
  },
  {
    "startMs": 542320,
    "text": "and then"
  },
  {
    "startMs": 543960,
    "text": "the line that we hear a lot, which is"
  },
  {
    "startMs": 545640,
    "text": "we'll get you the security architecture"
  },
  {
    "startMs": 547160,
    "text": "diagram next week."
  },
  {
    "startMs": 548680,
    "text": "Uh we do weekly check-in calls during a"
  },
  {
    "startMs": 550360,
    "text": "pilot, and then we hear this over and"
  },
  {
    "startMs": 551720,
    "text": "over. Uh it's not usually a great sign."
  },
  {
    "startMs": 557280,
    "text": "Uh the question that we often have our"
  },
  {
    "startMs": 559240,
    "text": "CISO end up asking, which is um"
  },
  {
    "startMs": 562200,
    "text": "what are you going to do if there's a"
  },
  {
    "startMs": 563120,
    "text": "breach?"
  },
  {
    "startMs": 564280,
    "text": "And we get this response, well, we"
  },
  {
    "startMs": 565640,
    "text": "haven't had a breach yet."
  },
  {
    "startMs": 567520,
    "text": "Uh"
  },
  {
    "startMs": 568120,
    "text": "with the subtext of we don't know what"
  },
  {
    "startMs": 569480,
    "text": "we would do if we did."
  },
  {
    "startMs": 571640,
    "text": "Um okay, reliability, this is another"
  },
  {
    "startMs": 573280,
    "text": "one. So, a control plane that actually"
  },
  {
    "startMs": 575120,
    "text": "works."
  },
  {
    "startMs": 576240,
    "text": "We want to see every admin setting"
  },
  {
    "startMs": 577720,
    "text": "available via API."
  },
  {
    "startMs": 579560,
    "text": "We want to see audit logs on config"
  },
  {
    "startMs": 581360,
    "text": "changes. So, if there are five different"
  },
  {
    "startMs": 583240,
    "text": "people who are given admin access and"
  },
  {
    "startMs": 585000,
    "text": "somebody accidentally changes something"
  },
  {
    "startMs": 587160,
    "text": "or does it because uh maybe it was"
  },
  {
    "startMs": 589960,
    "text": "really late at night and maybe they had"
  },
  {
    "startMs": 591040,
    "text": "too many drinks. Uh we actually want to"
  },
  {
    "startMs": 592440,
    "text": "see what happened. Uh we want to be able"
  },
  {
    "startMs": 594760,
    "text": "to control the rollout on these changes."
  },
  {
    "startMs": 597640,
    "text": "Uh we want to see real SLAs and a"
  },
  {
    "startMs": 599520,
    "text": "reachable support engineer. That goes a"
  },
  {
    "startMs": 600960,
    "text": "very, very long way."
  },
  {
    "startMs": 602760,
    "text": "So, uh things we've seen go wrong,"
  },
  {
    "startMs": 605280,
    "text": "a lot of apps that are rapidly"
  },
  {
    "startMs": 606720,
    "text": "prototyping, they're shipping so quickly"
  },
  {
    "startMs": 608600,
    "text": "that they are maybe shipping updates"
  },
  {
    "startMs": 610520,
    "text": "multiple times a day and there's a"
  },
  {
    "startMs": 611960,
    "text": "really attractive little button that"
  },
  {
    "startMs": 613120,
    "text": "says relaunch to update and it happens"
  },
  {
    "startMs": 615240,
    "text": "across 3,000 people. We have no way of"
  },
  {
    "startMs": 617160,
    "text": "tracking what's going wrong. Maybe then"
  },
  {
    "startMs": 619280,
    "text": "like SSL certificates break in one of"
  },
  {
    "startMs": 621480,
    "text": "the new releases and then we have no way"
  },
  {
    "startMs": 622840,
    "text": "of tracking because everybody's on a"
  },
  {
    "startMs": 624240,
    "text": "different version"
  },
  {
    "startMs": 625560,
    "text": "um and we have no way of being able to"
  },
  {
    "startMs": 626960,
    "text": "deploy at scale. Um that's really"
  },
  {
    "startMs": 628960,
    "text": "challenging."
  },
  {
    "startMs": 630720,
    "text": "No documentation versioning."
  },
  {
    "startMs": 632600,
    "text": "So, support articles with new terms or"
  },
  {
    "startMs": 634440,
    "text": "risks that are not actually in the legal"
  },
  {
    "startMs": 636280,
    "text": "contract but show up in the website"
  },
  {
    "startMs": 638040,
    "text": "somewhere in a random support page and"
  },
  {
    "startMs": 639680,
    "text": "then we have no way of tracking what"
  },
  {
    "startMs": 640760,
    "text": "they were before versus after and it"
  },
  {
    "startMs": 642880,
    "text": "just says updated yesterday."
  },
  {
    "startMs": 644880,
    "text": "All of these are real examples, by the"
  },
  {
    "startMs": 646080,
    "text": "way. I am not naming and shaming. Um I'm"
  },
  {
    "startMs": 648120,
    "text": "just shaming. So,"
  },
  {
    "startMs": 650560,
    "text": "uh maybe if any of you are familiar, you"
  },
  {
    "startMs": 652480,
    "text": "can put it together. Um core API's down"
  },
  {
    "startMs": 655320,
    "text": "for multiple hours during a busy trading"
  },
  {
    "startMs": 657200,
    "text": "day. Uh that is a really big problem for"
  },
  {
    "startMs": 659640,
    "text": "us because we run production systems. We"
  },
  {
    "startMs": 662280,
    "text": "are trading billions of dollars."
  },
  {
    "startMs": 664440,
    "text": "Um this is a really big issue for us."
  },
  {
    "startMs": 667040,
    "text": "And then lastly, no SLA roadmap or"
  },
  {
    "startMs": 669320,
    "text": "status page. Um the status page is a big"
  },
  {
    "startMs": 671480,
    "text": "one."
  },
  {
    "startMs": 672880,
    "text": "Okay, last, legal issues. So, we don't"
  },
  {
    "startMs": 675760,
    "text": "want anybody training on our data"
  },
  {
    "startMs": 677160,
    "text": "regardless of what type of feature or"
  },
  {
    "startMs": 678839,
    "text": "product it is."
  },
  {
    "startMs": 680440,
    "text": "Uh we also want to see a lot of"
  },
  {
    "startMs": 681720,
    "text": "transparency in the sub processors."
  },
  {
    "startMs": 684400,
    "text": "Um any fourth-party risk becomes our"
  },
  {
    "startMs": 686320,
    "text": "risk."
  },
  {
    "startMs": 688080,
    "text": "We want to see IP indemnification with"
  },
  {
    "startMs": 690240,
    "text": "reasonable liability caps. Uh we do not"
  },
  {
    "startMs": 692240,
    "text": "control the models, so if there's output"
  },
  {
    "startMs": 694200,
    "text": "that is IP infringing, we don't want to"
  },
  {
    "startMs": 695840,
    "text": "be held liable for it."
  },
  {
    "startMs": 697920,
    "text": "So, we have seen in pilots that people"
  },
  {
    "startMs": 699880,
    "text": "claim they have ZDR, they have it"
  },
  {
    "startMs": 701560,
    "text": "legally, but then they find out or we"
  },
  {
    "startMs": 703840,
    "text": "find out later that they actually retain"
  },
  {
    "startMs": 705400,
    "text": "some of our data because they say, \"Hey,"
  },
  {
    "startMs": 707040,
    "text": "we were looking at something and we"
  },
  {
    "startMs": 708160,
    "text": "noticed this thing.\" And we're like,"
  },
  {
    "startMs": 709440,
    "text": "\"How did you notice that? You weren't"
  },
  {
    "startMs": 710800,
    "text": "supposed to have this data.\" And they're"
  },
  {
    "startMs": 712520,
    "text": "like, \"Oh, yeah, you're right.\""
  },
  {
    "startMs": 714440,
    "text": "Um so, that's not great. If you say ZDR,"
  },
  {
    "startMs": 717400,
    "text": "do ZDR. Um"
  },
  {
    "startMs": 719640,
    "text": "next, every feature that is conveniently"
  },
  {
    "startMs": 721800,
    "text": "beta with permissive data retention"
  },
  {
    "startMs": 723560,
    "text": "clauses. So, we've seen some vendors who"
  },
  {
    "startMs": 726560,
    "text": "they will stop releasing new features in"
  },
  {
    "startMs": 729640,
    "text": "general availability. They will only"
  },
  {
    "startMs": 731360,
    "text": "make them beta, and then the beta comes"
  },
  {
    "startMs": 733760,
    "text": "with a secret little clause that says"
  },
  {
    "startMs": 735520,
    "text": "that they're allowed to retain our data,"
  },
  {
    "startMs": 736880,
    "text": "which is a very sneaky way of trying to"
  },
  {
    "startMs": 738440,
    "text": "get our data. We don't like that. Um not"
  },
  {
    "startMs": 740760,
    "text": "great."
  },
  {
    "startMs": 742320,
    "text": "Another one kind of similar is"
  },
  {
    "startMs": 743880,
    "text": "fourth-party risk that's tucked away on"
  },
  {
    "startMs": 745880,
    "text": "a random website page that's not listed"
  },
  {
    "startMs": 748080,
    "text": "in the contract. Uh this is a really big"
  },
  {
    "startMs": 750040,
    "text": "problem for us managing risk."
  },
  {
    "startMs": 754120,
    "text": "So, um"
  },
  {
    "startMs": 755400,
    "text": "it was the best of times, it was the"
  },
  {
    "startMs": 756480,
    "text": "worst of times. As a recap, the best"
  },
  {
    "startMs": 758839,
    "text": "startups have security architecture that"
  },
  {
    "startMs": 760600,
    "text": "actually works, support engineers who"
  },
  {
    "startMs": 762560,
    "text": "respond, an admin API from the"
  },
  {
    "startMs": 764800,
    "text": "beginning, a 90-day plan that deploys"
  },
  {
    "startMs": 767080,
    "text": "into our infrastructure and cloud, and"
  },
  {
    "startMs": 769360,
    "text": "success criteria that we write."
  },
  {
    "startMs": 771520,
    "text": "The worst AI startups don't have any"
  },
  {
    "startMs": 772839,
    "text": "security architecture diagrams, no path"
  },
  {
    "startMs": 774839,
    "text": "to a support engineer, no deployment"
  },
  {
    "startMs": 776720,
    "text": "control or audit logs, no ETAs,"
  },
  {
    "startMs": 779480,
    "text": "and salesmanship over solid product"
  },
  {
    "startMs": 781200,
    "text": "building."
  },
  {
    "startMs": 782160,
    "text": "Um this is really just kind of a recap"
  },
  {
    "startMs": 784880,
    "text": "of like what I have been through over"
  },
  {
    "startMs": 787000,
    "text": "the last 2 years. Um I actually don't"
  },
  {
    "startMs": 788880,
    "text": "think that any of this is novel,"
  },
  {
    "startMs": 790760,
    "text": "um but it is codifying a lot of what I"
  },
  {
    "startMs": 792839,
    "text": "feel like is good and best practice."
  },
  {
    "startMs": 795560,
    "text": "Um okay, so a new frontier model comes"
  },
  {
    "startMs": 797880,
    "text": "out on average every 11 days, but your"
  },
  {
    "startMs": 799839,
    "text": "architecture might be a decade or more"
  },
  {
    "startMs": 801480,
    "text": "old. So, you've got a bunch of cool new"
  },
  {
    "startMs": 803000,
    "text": "models, there's some amazing"
  },
  {
    "startMs": 804400,
    "text": "capabilities is there, and then you have"
  },
  {
    "startMs": 806560,
    "text": "profitability on the other side of it."
  },
  {
    "startMs": 808040,
    "text": "And what's in the middle? Maybe it's"
  },
  {
    "startMs": 809960,
    "text": "your legacy architecture, probably a lot"
  },
  {
    "startMs": 811840,
    "text": "of security and privacy issues, and a"
  },
  {
    "startMs": 813760,
    "text": "lot of change management. Um ChatGPT has"
  },
  {
    "startMs": 816520,
    "text": "only been out for 43 months. There are a"
  },
  {
    "startMs": 818080,
    "text": "lot of companies who are still doing an"
  },
  {
    "startMs": 819400,
    "text": "ERP migration that might have been from"
  },
  {
    "startMs": 821240,
    "text": "5 years ago. Um so, the timelines are"
  },
  {
    "startMs": 823960,
    "text": "very asymmetric. Uh and I think that"
  },
  {
    "startMs": 826160,
    "text": "sometimes we forget about that."
  },
  {
    "startMs": 829240,
    "text": "Um okay. So, my thesis again, half or"
  },
  {
    "startMs": 832120,
    "text": "more of getting to AI native is unsexy"
  },
  {
    "startMs": 834560,
    "text": "and has absolutely nothing to do with"
  },
  {
    "startMs": 836480,
    "text": "AI."
  },
  {
    "startMs": 837480,
    "text": "Um AI models and products today can't"
  },
  {
    "startMs": 839480,
    "text": "fix your legacy architecture. Although,"
  },
  {
    "startMs": 841280,
    "text": "if any of you are startup people, that's"
  },
  {
    "startMs": 842880,
    "text": "a great one to go for."
  },
  {
    "startMs": 844560,
    "text": "Um and it also can't run your change"
  },
  {
    "startMs": 846000,
    "text": "management. These are unscientific"
  },
  {
    "startMs": 848080,
    "text": "numbers that I'm putting up here, but I"
  },
  {
    "startMs": 850320,
    "text": "hypothesize that 40% of getting to AI"
  },
  {
    "startMs": 852920,
    "text": "native is AI models and products. The"
  },
  {
    "startMs": 855200,
    "text": "other 60% is all the other stuff that no"
  },
  {
    "startMs": 856920,
    "text": "one really likes talking about anymore,"
  },
  {
    "startMs": 858840,
    "text": "uh which is like data hygiene, clean"
  },
  {
    "startMs": 860360,
    "text": "architecture, having good integration,"
  },
  {
    "startMs": 862400,
    "text": "strong enablement, and change"
  },
  {
    "startMs": 863800,
    "text": "management."
  },
  {
    "startMs": 866440,
    "text": "Um I really look at AI as a flashlight,"
  },
  {
    "startMs": 868760,
    "text": "not a band-aid. Um I really think that"
  },
  {
    "startMs": 870960,
    "text": "AI shines a light on a lot of what's"
  },
  {
    "startMs": 873080,
    "text": "already working or not working. It can"
  },
  {
    "startMs": 874960,
    "text": "accelerate what's working really well,"
  },
  {
    "startMs": 876600,
    "text": "and it breaks down very quickly when"
  },
  {
    "startMs": 878360,
    "text": "things don't work well."
  },
  {
    "startMs": 879839,
    "text": "Um I don't think that it's a band-aid,"
  },
  {
    "startMs": 881440,
    "text": "and I think that for everybody who's in"
  },
  {
    "startMs": 882600,
    "text": "tech leadership, it's really important"
  },
  {
    "startMs": 883960,
    "text": "to remember that if you have issues in"
  },
  {
    "startMs": 886560,
    "text": "your technology estate, those need to be"
  },
  {
    "startMs": 888640,
    "text": "addressed before trying to plug in AI"
  },
  {
    "startMs": 891440,
    "text": "and just having everything rip. Um it's"
  },
  {
    "startMs": 894000,
    "text": "it's not going to work."
  },
  {
    "startMs": 895640,
    "text": "Um so, hehehe again, maybe an unpopular"
  },
  {
    "startMs": 898520,
    "text": "message, but I really believe that we"
  },
  {
    "startMs": 900520,
    "text": "all need to start with the boring 60%. I"
  },
  {
    "startMs": 903240,
    "text": "think that's where we all need to start"
  },
  {
    "startMs": 904640,
    "text": "to get to the other side of the road."
  },
  {
    "startMs": 907839,
    "text": "So,"
  },
  {
    "startMs": 908839,
    "text": "what did we learn as we shine the"
  },
  {
    "startMs": 910320,
    "text": "flashlight internally?"
  },
  {
    "startMs": 912000,
    "text": "Again, not revealing anything super"
  },
  {
    "startMs": 913520,
    "text": "proprietary, but I do think these are"
  },
  {
    "startMs": 915120,
    "text": "big picture lessons."
  },
  {
    "startMs": 916960,
    "text": "Number one, entitlements. Entitlements"
  },
  {
    "startMs": 918920,
    "text": "need a new paradigm."
  },
  {
    "startMs": 920560,
    "text": "Uh there are a lot of people in a lot of"
  },
  {
    "startMs": 922560,
    "text": "large enterprises who are over entitled,"
  },
  {
    "startMs": 924320,
    "text": "under entitled. The entitlements model"
  },
  {
    "startMs": 926480,
    "text": "and how it works and how it's managed,"
  },
  {
    "startMs": 928600,
    "text": "all of that breaks down when you think"
  },
  {
    "startMs": 930040,
    "text": "about agents and how quickly you want"
  },
  {
    "startMs": 931600,
    "text": "agents to work and what you want them to"
  },
  {
    "startMs": 933000,
    "text": "work on and their ability to exercise"
  },
  {
    "startMs": 934840,
    "text": "judgment. Um the entire paradigm just"
  },
  {
    "startMs": 937280,
    "text": "shifts."
  },
  {
    "startMs": 938840,
    "text": "Another one is cross-platform"
  },
  {
    "startMs": 940120,
    "text": "integration moved up the stack. So, AI"
  },
  {
    "startMs": 942440,
    "text": "is only as good as what it reaches and"
  },
  {
    "startMs": 944600,
    "text": "we want it everywhere. Uh so, having"
  },
  {
    "startMs": 946880,
    "text": "things that can integrate across"
  },
  {
    "startMs": 948640,
    "text": "platforms is really important uh even"
  },
  {
    "startMs": 950800,
    "text": "more than it already was."
  },
  {
    "startMs": 953000,
    "text": "Another one is centralized knowledge."
  },
  {
    "startMs": 954440,
    "text": "So, this is something that um Emil"
  },
  {
    "startMs": 957160,
    "text": "brought up this morning in his keynote,"
  },
  {
    "startMs": 959400,
    "text": "which is that basically we need thinner"
  },
  {
    "startMs": 961000,
    "text": "agents and a smarter substrate. Um"
  },
  {
    "startMs": 963320,
    "text": "centralized knowledge is really key to"
  },
  {
    "startMs": 964720,
    "text": "that. So, all of your documentation, all"
  },
  {
    "startMs": 966280,
    "text": "your support articles, everything that's"
  },
  {
    "startMs": 967880,
    "text": "going on inside of your company that's"
  },
  {
    "startMs": 969160,
    "text": "making it work, um all of that needs to"
  },
  {
    "startMs": 971160,
    "text": "be centralized and easily consumable."
  },
  {
    "startMs": 973680,
    "text": "Even better if AI can help write that in"
  },
  {
    "startMs": 976200,
    "text": "real time in a feedback loop. Uh that's"
  },
  {
    "startMs": 978040,
    "text": "something we've been talking about with"
  },
  {
    "startMs": 979000,
    "text": "some of our vendors."
  },
  {
    "startMs": 981200,
    "text": "Another one is a separate ecosystem for"
  },
  {
    "startMs": 982840,
    "text": "experimentation. Um some companies may"
  },
  {
    "startMs": 985280,
    "text": "need to get here."
  },
  {
    "startMs": 987000,
    "text": "That gap between your legacy"
  },
  {
    "startMs": 988440,
    "text": "architecture and where you want to go"
  },
  {
    "startMs": 989720,
    "text": "might be so vast that you actually just"
  },
  {
    "startMs": 991839,
    "text": "decide, \"Hey, maybe we need a separate"
  },
  {
    "startMs": 993839,
    "text": "ecosystem to do a lot of this work,"
  },
  {
    "startMs": 995760,
    "text": "figure out what does work and what"
  },
  {
    "startMs": 997240,
    "text": "doesn't and then kind of go from there.\""
  },
  {
    "startMs": 999520,
    "text": "Um and that's something that we thought"
  },
  {
    "startMs": 1000640,
    "text": "about as well."
  },
  {
    "startMs": 1002240,
    "text": "So, to just put a finer point on the"
  },
  {
    "startMs": 1004720,
    "text": "agents and the entitlement thing,"
  },
  {
    "startMs": 1006480,
    "text": "uh agents inherit your foundations. So,"
  },
  {
    "startMs": 1008960,
    "text": "I strongly recommend that everybody fix"
  },
  {
    "startMs": 1011280,
    "text": "their entitlements if they are not"
  },
  {
    "startMs": 1012480,
    "text": "working really well now"
  },
  {
    "startMs": 1014120,
    "text": "um because this is something that if you"
  },
  {
    "startMs": 1016280,
    "text": "think about the problems that you run"
  },
  {
    "startMs": 1017440,
    "text": "into when things go rogue, processes go"
  },
  {
    "startMs": 1019560,
    "text": "rogue, people go rogue, agents are going"
  },
  {
    "startMs": 1021600,
    "text": "to like 100X that problem. Um so, this"
  },
  {
    "startMs": 1024199,
    "text": "is really something that's worth"
  },
  {
    "startMs": 1025839,
    "text": "figuring out now."
  },
  {
    "startMs": 1028760,
    "text": "So, to kind of recap uh as I wrap up"
  },
  {
    "startMs": 1031640,
    "text": "here, the recipe, if you are one of the"
  },
  {
    "startMs": 1033640,
    "text": "people in the first half who are raising"
  },
  {
    "startMs": 1035520,
    "text": "your hand on like, \"What do I need to do"
  },
  {
    "startMs": 1037400,
    "text": "if I'm a startup and I want to work with"
  },
  {
    "startMs": 1038760,
    "text": "a really difficult large customer?"
  },
  {
    "startMs": 1040880,
    "text": "Millennium's got like 8,000 people. We"
  },
  {
    "startMs": 1043120,
    "text": "have very, very tight security,"
  },
  {
    "startMs": 1044800,
    "text": "compliance, regulatory requirements."
  },
  {
    "startMs": 1047439,
    "text": "Um this is the stuff that we care about."
  },
  {
    "startMs": 1050200,
    "text": "And we want to see more startups doing"
  },
  {
    "startMs": 1053520,
    "text": "work that allows us to work with them."
  },
  {
    "startMs": 1055920,
    "text": "Um I really view this as like one of the"
  },
  {
    "startMs": 1058240,
    "text": "highest bars. We're probably not the"
  },
  {
    "startMs": 1059920,
    "text": "highest, um although we're probably"
  },
  {
    "startMs": 1061680,
    "text": "pretty close."
  },
  {
    "startMs": 1063120,
    "text": "Um and I think if you can architect your"
  },
  {
    "startMs": 1064840,
    "text": "startup to work with companies like this"
  },
  {
    "startMs": 1066600,
    "text": "with this kind of architecture, um"
  },
  {
    "startMs": 1068400,
    "text": "you're probably going to be able to"
  },
  {
    "startMs": 1069240,
    "text": "satisfy basically everybody else."
  },
  {
    "startMs": 1072440,
    "text": "Um on the other side"
  },
  {
    "startMs": 1074320,
    "text": "for anybody who's buying, uh these are"
  },
  {
    "startMs": 1076240,
    "text": "the things that I think again that kind"
  },
  {
    "startMs": 1077640,
    "text": "of that boring 60% that really deserves"
  },
  {
    "startMs": 1080440,
    "text": "a lot of work. Um off entitlements,"
  },
  {
    "startMs": 1083040,
    "text": "governance, audit logging, etc. Um these"
  },
  {
    "startMs": 1085640,
    "text": "are the things that I think we need to"
  },
  {
    "startMs": 1086840,
    "text": "have in terms of systems to get it"
  },
  {
    "startMs": 1089000,
    "text": "working on the other side of the"
  },
  {
    "startMs": 1090360,
    "text": "equation."
  },
  {
    "startMs": 1092200,
    "text": "So,"
  },
  {
    "startMs": 1093200,
    "text": "that's it. Um"
  },
  {
    "startMs": 1095040,
    "text": "my only motivation here is to getting"
  },
  {
    "startMs": 1097000,
    "text": "stuff working better and having better"
  },
  {
    "startMs": 1099480,
    "text": "enterprise grade AI."
  },
  {
    "startMs": 1101480,
    "text": "Uh that's a QR code to my LinkedIn and I"
  },
  {
    "startMs": 1103800,
    "text": "appreciate all of your time. Thank you."
  },
  {
    "startMs": 1106962,
    "text": ">> [applause]"
  },
  {
    "startMs": 1120717,
    "text": "[music]"
  }
]