[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12520,
    "text": ">> Test."
  },
  {
    "startMs": 13360,
    "text": "Okay, amazing."
  },
  {
    "startMs": 15080,
    "text": "It's great to meet everyone. I'm Taís."
  },
  {
    "startMs": 16920,
    "text": "I'm the founder of Taste Labs. Uh for"
  },
  {
    "startMs": 19360,
    "text": "those of you who don't know us, we came"
  },
  {
    "startMs": 20640,
    "text": "out of stealth a few weeks ago and our"
  },
  {
    "startMs": 22760,
    "text": "whole mission is basically how do we end"
  },
  {
    "startMs": 24720,
    "text": "AI slop? I that's my personal enemy. Um"
  },
  {
    "startMs": 27960,
    "text": "and so we really believe that to solve"
  },
  {
    "startMs": 30720,
    "text": "this problem of slop, we have to like"
  },
  {
    "startMs": 32360,
    "text": "decode subjective domains. Uh there's"
  },
  {
    "startMs": 34720,
    "text": "been so much effort being put into"
  },
  {
    "startMs": 36080,
    "text": "getting models and agents amazing at"
  },
  {
    "startMs": 38200,
    "text": "things like coding and math. Uh and it's"
  },
  {
    "startMs": 40360,
    "text": "time that we put all that same effort"
  },
  {
    "startMs": 41960,
    "text": "into making them great at things like"
  },
  {
    "startMs": 43120,
    "text": "design uh and writing. And so design is"
  },
  {
    "startMs": 45280,
    "text": "this first pillar that we're starting"
  },
  {
    "startMs": 46400,
    "text": "with and it's been it's been incredibly"
  },
  {
    "startMs": 48400,
    "text": "exciting. Um"
  },
  {
    "startMs": 50560,
    "text": "We work primarily in two ways. So we"
  },
  {
    "startMs": 52560,
    "text": "work a lot with the frontier labs on how"
  },
  {
    "startMs": 54520,
    "text": "do we evaluate their models, understand"
  },
  {
    "startMs": 56720,
    "text": "where they're breaking, understand what"
  },
  {
    "startMs": 58360,
    "text": "could be better about them, and then"
  },
  {
    "startMs": 59760,
    "text": "construct the right either post-training"
  },
  {
    "startMs": 61440,
    "text": "data or RL environments to basically fix"
  },
  {
    "startMs": 63880,
    "text": "that problem. And part of this is like"
  },
  {
    "startMs": 65519,
    "text": "how do you take something as fuzzy and"
  },
  {
    "startMs": 66960,
    "text": "large as design and break it down to a"
  },
  {
    "startMs": 69200,
    "text": "level that you can identify what is best"
  },
  {
    "startMs": 71720,
    "text": "solved through each method. What are"
  },
  {
    "startMs": 73440,
    "text": "elements of design that are almost like"
  },
  {
    "startMs": 75080,
    "text": "once you kind of boil down the problem"
  },
  {
    "startMs": 76680,
    "text": "becomes so specific that they almost"
  },
  {
    "startMs": 78280,
    "text": "become deterministic. So for example, uh"
  },
  {
    "startMs": 80680,
    "text": "if you're trying to train a model to be"
  },
  {
    "startMs": 81960,
    "text": "good at selecting color palettes or have"
  },
  {
    "startMs": 83960,
    "text": "contrast or alignment, those are things"
  },
  {
    "startMs": 86160,
    "text": "that if you define the problem and the"
  },
  {
    "startMs": 87480,
    "text": "context in a specific enough way, uh you"
  },
  {
    "startMs": 89960,
    "text": "you can get to an answer that's like"
  },
  {
    "startMs": 91120,
    "text": "pretty objective or that at least most"
  },
  {
    "startMs": 92480,
    "text": "experts would agree to. But maybe other"
  },
  {
    "startMs": 94400,
    "text": "things like uh aesthetics, you naturally"
  },
  {
    "startMs": 97080,
    "text": "will see this expert disagreement. And"
  },
  {
    "startMs": 98680,
    "text": "so then you want to lean on to things"
  },
  {
    "startMs": 100000,
    "text": "that are closer to to data. So anyway,"
  },
  {
    "startMs": 102280,
    "text": "we spend a lot of time thinking about"
  },
  {
    "startMs": 103320,
    "text": "all those problems. Uh but on the other"
  },
  {
    "startMs": 104840,
    "text": "side is also"
  },
  {
    "startMs": 106520,
    "text": "without even touching the model layer,"
  },
  {
    "startMs": 107680,
    "text": "right? How do we actually help agents"
  },
  {
    "startMs": 109720,
    "text": "and app layer companies produce better"
  },
  {
    "startMs": 111280,
    "text": "things? And there's a lot that goes into"
  },
  {
    "startMs": 113320,
    "text": "that, right? You have these different"
  },
  {
    "startMs": 114440,
    "text": "sets of problems at the application"
  },
  {
    "startMs": 115600,
    "text": "layer because you're using an"
  },
  {
    "startMs": 116840,
    "text": "off-the-shelf model that tends to"
  },
  {
    "startMs": 118600,
    "text": "collapse in terms uh uh of style tends"
  },
  {
    "startMs": 120640,
    "text": "to collapse to the mean. So, how do we"
  },
  {
    "startMs": 122200,
    "text": "force that creativity back to the"
  },
  {
    "startMs": 123600,
    "text": "system? How do we avoid these patterns"
  },
  {
    "startMs": 125640,
    "text": "of slop, which we'll talk about a lot"
  },
  {
    "startMs": 127160,
    "text": "today? Uh how do you understand like"
  },
  {
    "startMs": 129160,
    "text": "user preferences or brand preferences"
  },
  {
    "startMs": 131600,
    "text": "preference so that you can uh maintain"
  },
  {
    "startMs": 133680,
    "text": "endurance to that style? Uh so, there's"
  },
  {
    "startMs": 135720,
    "text": "lots of things that are actually need to"
  },
  {
    "startMs": 137800,
    "text": "be solved as context or judgment or"
  },
  {
    "startMs": 139959,
    "text": "verification at the app layer, which is"
  },
  {
    "startMs": 142000,
    "text": "why we kind of work across both."
  },
  {
    "startMs": 145720,
    "text": "Maybe I'll start with more of a a"
  },
  {
    "startMs": 147240,
    "text": "philosophical question of like how how"
  },
  {
    "startMs": 148800,
    "text": "do you define something that is great?"
  },
  {
    "startMs": 150240,
    "text": "Like how do you define greatness? And"
  },
  {
    "startMs": 152800,
    "text": "for something like math, it's easier,"
  },
  {
    "startMs": 154760,
    "text": "right? Because there's kind of one"
  },
  {
    "startMs": 156080,
    "text": "objective answer, and uh great is the"
  },
  {
    "startMs": 158720,
    "text": "same as correct. But then for something"
  },
  {
    "startMs": 160440,
    "text": "like writing or design,"
  },
  {
    "startMs": 163000,
    "text": "it's much harder, right? Like how do you"
  },
  {
    "startMs": 164040,
    "text": "define what's like a great tweet or"
  },
  {
    "startMs": 165760,
    "text": "what's a great art piece or what's a"
  },
  {
    "startMs": 167480,
    "text": "great website? Um I don't know what's"
  },
  {
    "startMs": 170280,
    "text": "the last time that you interacted with a"
  },
  {
    "startMs": 172400,
    "text": "poem or walked into a coffee shop and"
  },
  {
    "startMs": 173920,
    "text": "for some reason it kind of like hit"
  },
  {
    "startMs": 175160,
    "text": "different and it felt"
  },
  {
    "startMs": 176959,
    "text": "very special. Uh but probably it's a"
  },
  {
    "startMs": 178840,
    "text": "combination of things that it it felt"
  },
  {
    "startMs": 180720,
    "text": "very unique. It felt almost a little"
  },
  {
    "startMs": 182080,
    "text": "different. It kind of called your"
  },
  {
    "startMs": 183120,
    "text": "attention. Uh it felt like there it was"
  },
  {
    "startMs": 184840,
    "text": "made with a lot of care and attention to"
  },
  {
    "startMs": 187080,
    "text": "to detail and craft, and it almost had"
  },
  {
    "startMs": 188760,
    "text": "the sense of of like authenticity. Um"
  },
  {
    "startMs": 191320,
    "text": "and I think that's a lot of what AI is"
  },
  {
    "startMs": 192600,
    "text": "missing today. It's like how do we take"
  },
  {
    "startMs": 194920,
    "text": "uh things that are not necessarily"
  },
  {
    "startMs": 196280,
    "text": "average, right? How do we produce things"
  },
  {
    "startMs": 197720,
    "text": "that are purposely like out of"
  },
  {
    "startMs": 198880,
    "text": "distribution? Um and slop is kind of the"
  },
  {
    "startMs": 201320,
    "text": "opposite of that, right? I think it is"
  },
  {
    "startMs": 203040,
    "text": "hard to define what is great sometimes,"
  },
  {
    "startMs": 204560,
    "text": "but I think it's pretty pretty easy to"
  },
  {
    "startMs": 206160,
    "text": "define what is slop in the sense that"
  },
  {
    "startMs": 207600,
    "text": "most people would agree. I think the"
  },
  {
    "startMs": 209160,
    "text": "sense of like repetition of kind of"
  },
  {
    "startMs": 211160,
    "text": "soullessness is something that all of us"
  },
  {
    "startMs": 212840,
    "text": "feel right now when using AI, and I"
  },
  {
    "startMs": 214720,
    "text": "think it's quite magical, by the way,"
  },
  {
    "startMs": 215959,
    "text": "that AI has gotten to a point that any"
  },
  {
    "startMs": 218160,
    "text": "human on the planet that is not even a"
  },
  {
    "startMs": 219680,
    "text": "designer, that is not an engineer, can"
  },
  {
    "startMs": 221680,
    "text": "click a button and suddenly make an"
  },
  {
    "startMs": 223480,
    "text": "entire PowerPoint or make a website or"
  },
  {
    "startMs": 225200,
    "text": "make a web app. That's pretty cool. But"
  },
  {
    "startMs": 227640,
    "text": "it comes with consequences, right? Uh it"
  },
  {
    "startMs": 229519,
    "text": "comes with consequences of suddenly now"
  },
  {
    "startMs": 231800,
    "text": "the cost of generation is basically"
  },
  {
    "startMs": 233600,
    "text": "going to zero. Uh and But the average"
  },
  {
    "startMs": 235960,
    "text": "person hasn't necessarily honed their"
  },
  {
    "startMs": 237720,
    "text": "taste. Like I does think about the"
  },
  {
    "startMs": 239560,
    "text": "amount of effort and work that a"
  },
  {
    "startMs": 241000,
    "text": "designer puts in throughout their life"
  },
  {
    "startMs": 243200,
    "text": "to like build up their taste, right?"
  },
  {
    "startMs": 244720,
    "text": "Like there's all this process of like"
  },
  {
    "startMs": 246760,
    "text": "getting exposed to many things and"
  },
  {
    "startMs": 248760,
    "text": "learning to like spot patterns and"
  },
  {
    "startMs": 250360,
    "text": "learning to develop a point of view and"
  },
  {
    "startMs": 251920,
    "text": "like kind of do things in a in a"
  },
  {
    "startMs": 253480,
    "text": "courageous way that maybe are a little"
  },
  {
    "startMs": 254880,
    "text": "bit against the norm. Learning what not"
  },
  {
    "startMs": 256600,
    "text": "to do and how to like have restraint and"
  },
  {
    "startMs": 258600,
    "text": "that's very hard. Like the average"
  },
  {
    "startMs": 260480,
    "text": "person doesn't necessarily have the the"
  },
  {
    "startMs": 262880,
    "text": "time nor the skills to go and develop"
  },
  {
    "startMs": 264520,
    "text": "taste in everything, let's say in"
  },
  {
    "startMs": 265880,
    "text": "design. And so"
  },
  {
    "startMs": 267320,
    "text": "um"
  },
  {
    "startMs": 268080,
    "text": "I think it would be a bad case scenario"
  },
  {
    "startMs": 269400,
    "text": "for us to just like be like, \"Okay, the"
  },
  {
    "startMs": 270840,
    "text": "way to fix slop is for everyone to have"
  },
  {
    "startMs": 272400,
    "text": "taste.\" cuz I don't think that's"
  },
  {
    "startMs": 273640,
    "text": "necessarily realistic. Um I think how do"
  },
  {
    "startMs": 275919,
    "text": "we how can we understand this better so"
  },
  {
    "startMs": 277680,
    "text": "that we can make even for the average"
  },
  {
    "startMs": 279080,
    "text": "person the ability to create something"
  },
  {
    "startMs": 281120,
    "text": "great and to understand maybe their own"
  },
  {
    "startMs": 282720,
    "text": "taste um"
  },
  {
    "startMs": 284280,
    "text": "easy more more more easy. So that's"
  },
  {
    "startMs": 287080,
    "text": "that's a lot of what we're we're"
  },
  {
    "startMs": 288120,
    "text": "focusing on. Um"
  },
  {
    "startMs": 289680,
    "text": "So yeah, I mean this phenomenon of slop,"
  },
  {
    "startMs": 291440,
    "text": "by the way, is not new. Uh if you were"
  },
  {
    "startMs": 293480,
    "text": "in the internet uh as social media"
  },
  {
    "startMs": 295400,
    "text": "emerged, you probably saw a lot of slop"
  },
  {
    "startMs": 297240,
    "text": "before that. But I do think that AI has"
  },
  {
    "startMs": 299400,
    "text": "been this kind of like accelerating"
  },
  {
    "startMs": 300720,
    "text": "force, right? Of like being able to"
  },
  {
    "startMs": 301880,
    "text": "create things very easily uh with a"
  },
  {
    "startMs": 303760,
    "text": "click of a button and that like"
  },
  {
    "startMs": 304720,
    "text": "thoughtlessness around it. And there's"
  },
  {
    "startMs": 306280,
    "text": "kind of these three characteristics that"
  },
  {
    "startMs": 307720,
    "text": "I I would say repeat in slop. Uh so A,"
  },
  {
    "startMs": 310840,
    "text": "repetition. So you start seeing the same"
  },
  {
    "startMs": 313120,
    "text": "thing many many many times. Um the"
  },
  {
    "startMs": 315520,
    "text": "second is lack of fit, which I actually"
  },
  {
    "startMs": 316880,
    "text": "think is is very related. So"
  },
  {
    "startMs": 319360,
    "text": "fit is kind of this ability for"
  },
  {
    "startMs": 320840,
    "text": "something to feel correct for a specific"
  },
  {
    "startMs": 322520,
    "text": "context, right? For a specific moment in"
  },
  {
    "startMs": 324040,
    "text": "time, for a specific person. Uh but"
  },
  {
    "startMs": 326200,
    "text": "suddenly if you have repetition and"
  },
  {
    "startMs": 327680,
    "text": "let's say one person asks for a website"
  },
  {
    "startMs": 330360,
    "text": "for their pet shop and the other one"
  },
  {
    "startMs": 331960,
    "text": "asks for a website for their finance"
  },
  {
    "startMs": 333560,
    "text": "firm and somehow those designs converge"
  },
  {
    "startMs": 336000,
    "text": "and look the same."
  },
  {
    "startMs": 337840,
    "text": "That's quite odd, right? Like if that"
  },
  {
    "startMs": 339240,
    "text": "was in if you were actually crafting"
  },
  {
    "startMs": 340800,
    "text": "that with care, that wouldn't you"
  },
  {
    "startMs": 342160,
    "text": "wouldn't converge necessarily on those"
  },
  {
    "startMs": 343880,
    "text": "things. And so this lack of fit and lack"
  },
  {
    "startMs": 346040,
    "text": "of understanding of context is actually"
  },
  {
    "startMs": 347720,
    "text": "huge problem that like leads to slop. Um"
  },
  {
    "startMs": 350040,
    "text": "and the third is maybe low intent, which"
  },
  {
    "startMs": 351360,
    "text": "is probably a mix of Yeah, you're going"
  },
  {
    "startMs": 353160,
    "text": "to have a bunch of people prompting"
  },
  {
    "startMs": 354160,
    "text": "really quickly and maybe just wanting to"
  },
  {
    "startMs": 355760,
    "text": "one shot something. But I think there's"
  },
  {
    "startMs": 357280,
    "text": "actually this like intent interpretation"
  },
  {
    "startMs": 359120,
    "text": "piece that's missing in the systems that"
  },
  {
    "startMs": 360560,
    "text": "we're building. Like how can you help"
  },
  {
    "startMs": 362280,
    "text": "your user, right? Like how can you help"
  },
  {
    "startMs": 363600,
    "text": "them better understand the intent that"
  },
  {
    "startMs": 364920,
    "text": "they have um so that you can add more"
  },
  {
    "startMs": 367600,
    "text": "color and add more context on onto what"
  },
  {
    "startMs": 369280,
    "text": "you're trying to create."
  },
  {
    "startMs": 372160,
    "text": "Okay. And I I'm a big believer by the"
  },
  {
    "startMs": 374080,
    "text": "way that you in order to fix something"
  },
  {
    "startMs": 376560,
    "text": "first have to measure it and you first"
  },
  {
    "startMs": 378040,
    "text": "have to understand it. I think that's"
  },
  {
    "startMs": 380280,
    "text": "exactly why we're so focused on like how"
  },
  {
    "startMs": 381960,
    "text": "do we"
  },
  {
    "startMs": 382960,
    "text": "uh"
  },
  {
    "startMs": 383840,
    "text": "turn these domains into something a bit"
  },
  {
    "startMs": 385280,
    "text": "more verifiable so that we can attach a"
  },
  {
    "startMs": 387200,
    "text": "measure to it. So uh"
  },
  {
    "startMs": 389120,
    "text": "you'll you'll go on a little bit of a"
  },
  {
    "startMs": 390160,
    "text": "research journey with me here now, but"
  },
  {
    "startMs": 391840,
    "text": "we basically wanted to figure out can we"
  },
  {
    "startMs": 393400,
    "text": "measure slop? Like can we actually"
  },
  {
    "startMs": 395120,
    "text": "measure this quantitatively and spot"
  },
  {
    "startMs": 396840,
    "text": "this and what does that like look like?"
  },
  {
    "startMs": 401160,
    "text": "So we analyzed over 2 million websites"
  },
  {
    "startMs": 403800,
    "text": "from the past like 10 years kind of like"
  },
  {
    "startMs": 405280,
    "text": "way back machine style to try to"
  },
  {
    "startMs": 406600,
    "text": "understand all the trends across like"
  },
  {
    "startMs": 408240,
    "text": "design, how is the internet changing, uh"
  },
  {
    "startMs": 410919,
    "text": "how are"
  },
  {
    "startMs": 411880,
    "text": "how is like design changing over time?"
  },
  {
    "startMs": 413960,
    "text": "And two things were interesting. And we"
  },
  {
    "startMs": 415720,
    "text": "also met by the way then kind of"
  },
  {
    "startMs": 417440,
    "text": "synthetically generated a set of uh"
  },
  {
    "startMs": 420200,
    "text": "design websites so we could kind of like"
  },
  {
    "startMs": 421800,
    "text": "compare like how does human-made sites"
  },
  {
    "startMs": 424960,
    "text": "compare to AI-generated ones? And there"
  },
  {
    "startMs": 427440,
    "text": "were a few things that were interesting."
  },
  {
    "startMs": 428440,
    "text": "So one was that you already kind of saw"
  },
  {
    "startMs": 431240,
    "text": "a a bit of like a collapse"
  },
  {
    "startMs": 433800,
    "text": "uh on the internet before even AI. So"
  },
  {
    "startMs": 435960,
    "text": "you saw kind of the internet becoming"
  },
  {
    "startMs": 437440,
    "text": "more homogeneous, using more similar"
  },
  {
    "startMs": 439240,
    "text": "color palettes, using more similar"
  },
  {
    "startMs": 440720,
    "text": "layouts, uh which is probably a function"
  },
  {
    "startMs": 442880,
    "text": "of more uh"
  },
  {
    "startMs": 444320,
    "text": "I would say this like kind of trends"
  },
  {
    "startMs": 445720,
    "text": "spreading more and more and more"
  },
  {
    "startMs": 446680,
    "text": "quickly, let's say. Uh but with AI I"
  },
  {
    "startMs": 448520,
    "text": "think you saw this repetition happening"
  },
  {
    "startMs": 450080,
    "text": "a lot more and being almost more like um"
  },
  {
    "startMs": 452520,
    "text": "identified kind of regardless of"
  },
  {
    "startMs": 453680,
    "text": "context. So even in completely different"
  },
  {
    "startMs": 455080,
    "text": "buckets you saw patterns that were very"
  },
  {
    "startMs": 456880,
    "text": "similar."
  },
  {
    "startMs": 457919,
    "text": "So we we built this I I call this"
  },
  {
    "startMs": 459800,
    "text": "probes, but basically we uh we did two"
  },
  {
    "startMs": 462080,
    "text": "things. So we did this like pattern"
  },
  {
    "startMs": 463240,
    "text": "mining on all this data to understand"
  },
  {
    "startMs": 465040,
    "text": "like what are features that we can"
  },
  {
    "startMs": 466360,
    "text": "extract from all these sites. What are"
  },
  {
    "startMs": 467800,
    "text": "all these characteristics that we can"
  },
  {
    "startMs": 469200,
    "text": "make more objective, right? Colors,"
  },
  {
    "startMs": 471240,
    "text": "typography, layout, audience. Like, how"
  },
  {
    "startMs": 473480,
    "text": "can we like distill this down into"
  },
  {
    "startMs": 474880,
    "text": "things that become almost like uh"
  },
  {
    "startMs": 476320,
    "text": "structured? And then how do we uh train"
  },
  {
    "startMs": 478880,
    "text": "up these like probes? So, think of these"
  },
  {
    "startMs": 480400,
    "text": "as like baby classifiers. Like, how do"
  },
  {
    "startMs": 482360,
    "text": "we uh train the ability to spot this one"
  },
  {
    "startMs": 484480,
    "text": "characteristic?"
  },
  {
    "startMs": 485880,
    "text": "And for all these slop sites, we"
  },
  {
    "startMs": 487760,
    "text": "identify we started identifying like"
  },
  {
    "startMs": 489600,
    "text": "what are the probes that basically mean"
  },
  {
    "startMs": 491360,
    "text": "the site is very likely to be AI slop."
  },
  {
    "startMs": 494800,
    "text": "Um and especially when you start"
  },
  {
    "startMs": 496640,
    "text": "combining them use and you see the"
  },
  {
    "startMs": 497840,
    "text": "frequency of multiple of these happening"
  },
  {
    "startMs": 499360,
    "text": "at once, it became very likely that you"
  },
  {
    "startMs": 501320,
    "text": "could actually like measure"
  },
  {
    "startMs": 503240,
    "text": "uh and predict slop. And we saw a a"
  },
  {
    "startMs": 505960,
    "text": "super high basically ability to do that"
  },
  {
    "startMs": 507760,
    "text": "prediction, which was really cool to"
  },
  {
    "startMs": 509200,
    "text": "see. This performed better, by the way,"
  },
  {
    "startMs": 510640,
    "text": "than like most LLM as a judge methods of"
  },
  {
    "startMs": 512760,
    "text": "like asking an LLM to like judge if that"
  },
  {
    "startMs": 515360,
    "text": "uh is like great human quality versus"
  },
  {
    "startMs": 516960,
    "text": "like AI-generated slop. Uh so, that was"
  },
  {
    "startMs": 519039,
    "text": "pretty cool to see. And I think kind of"
  },
  {
    "startMs": 520159,
    "text": "shows this pattern that we see in AI"
  },
  {
    "startMs": 523520,
    "text": "really being uh an actually quantitative"
  },
  {
    "startMs": 526240,
    "text": "thing that we can see in slop, uh which"
  },
  {
    "startMs": 528000,
    "text": "I find really cool. But, obviously, we"
  },
  {
    "startMs": 529800,
    "text": "don't want to stop there, right? We"
  },
  {
    "startMs": 530839,
    "text": "don't want to just measure slop. We want"
  },
  {
    "startMs": 532560,
    "text": "to also solve it. And so,"
  },
  {
    "startMs": 534240,
    "text": "um there's a few I I think I mentioned"
  },
  {
    "startMs": 536080,
    "text": "this before, but like the as the cost of"
  },
  {
    "startMs": 538480,
    "text": "production basically goes to zero, I"
  },
  {
    "startMs": 540120,
    "text": "think the thing that becomes"
  },
  {
    "startMs": 542040,
    "text": "expensive and matters more than ever is"
  },
  {
    "startMs": 544839,
    "text": "judgment. Um I don't even want to use"
  },
  {
    "startMs": 546720,
    "text": "the word taste here."
  },
  {
    "startMs": 547839,
    "text": "Is judgment. I think it's this ability"
  },
  {
    "startMs": 549240,
    "text": "to discern what's right. It's this"
  },
  {
    "startMs": 550960,
    "text": "ability to break down a problem so that"
  },
  {
    "startMs": 552520,
    "text": "you can actually understand it and"
  },
  {
    "startMs": 553800,
    "text": "create solutions for it. And so, yes,"
  },
  {
    "startMs": 556200,
    "text": "there's the side of judgment that is"
  },
  {
    "startMs": 557320,
    "text": "human judgment that I actually think is"
  },
  {
    "startMs": 558560,
    "text": "more valuable than ever. But, there's"
  },
  {
    "startMs": 560040,
    "text": "also the side of like how do we build"
  },
  {
    "startMs": 561360,
    "text": "the right tools and systems to like fix"
  },
  {
    "startMs": 563360,
    "text": "pieces of this problem, right?"
  },
  {
    "startMs": 567480,
    "text": "So, yeah, how do we how do we fight"
  },
  {
    "startMs": 568680,
    "text": "slop, my my enemy?"
  },
  {
    "startMs": 570560,
    "text": "Um"
  },
  {
    "startMs": 571880,
    "text": "And, by the way, I think there's there's"
  },
  {
    "startMs": 573560,
    "text": "a lot of conversation going around how"
  },
  {
    "startMs": 575880,
    "text": "do you fight slop at the model layer?"
  },
  {
    "startMs": 577520,
    "text": "Like, how do we make models better? How"
  },
  {
    "startMs": 579240,
    "text": "do we make models have a higher bar?"
  },
  {
    "startMs": 580720,
    "text": "Which don't get me wrong, it has to be"
  },
  {
    "startMs": 582240,
    "text": "solved and we're working very hard to"
  },
  {
    "startMs": 583520,
    "text": "solve that, too. But I actually think"
  },
  {
    "startMs": 585360,
    "text": "this problem of inference time is"
  },
  {
    "startMs": 586800,
    "text": "equally, if not even more important."
  },
  {
    "startMs": 589200,
    "text": "Because that's actually when you"
  },
  {
    "startMs": 590840,
    "text": "interact with the end user. And this"
  },
  {
    "startMs": 592960,
    "text": "kind of back and forth of how do you"
  },
  {
    "startMs": 594360,
    "text": "understand this context and intent"
  },
  {
    "startMs": 595640,
    "text": "happens at the moment of inference time."
  },
  {
    "startMs": 597120,
    "text": "So, I don't think that we can ignore and"
  },
  {
    "startMs": 598720,
    "text": "just make models better and not solve"
  },
  {
    "startMs": 600240,
    "text": "this, otherwise slop will keep existing."
  },
  {
    "startMs": 602480,
    "text": "Um so, maybe breaking down a few of"
  },
  {
    "startMs": 604480,
    "text": "those pieces and kind of"
  },
  {
    "startMs": 606240,
    "text": "um"
  },
  {
    "startMs": 607000,
    "text": "a few of the ways that we've thought"
  },
  {
    "startMs": 608000,
    "text": "about solving this or a few solutions"
  },
  {
    "startMs": 609240,
    "text": "that we built to solve this. But I"
  },
  {
    "startMs": 610200,
    "text": "think, for example, for something like"
  },
  {
    "startMs": 611720,
    "text": "repetition, one of the things that we're"
  },
  {
    "startMs": 613480,
    "text": "working on is I I've nicknamed it, I"
  },
  {
    "startMs": 615080,
    "text": "don't know if that's going to be the"
  },
  {
    "startMs": 615880,
    "text": "official name, but like the creativity"
  },
  {
    "startMs": 617160,
    "text": "API. How can we create a system that"
  },
  {
    "startMs": 619320,
    "text": "almost becomes an inspiration machine"
  },
  {
    "startMs": 621280,
    "text": "for your agent? So that it can produce"
  },
  {
    "startMs": 623040,
    "text": "something that's actually out of"
  },
  {
    "startMs": 624000,
    "text": "distribution instead of something that"
  },
  {
    "startMs": 625640,
    "text": "is"
  },
  {
    "startMs": 626520,
    "text": "in that same average and kind of mean"
  },
  {
    "startMs": 628200,
    "text": "that we're seeing happen with like the"
  },
  {
    "startMs": 629880,
    "text": "slop sites. Um so, this is one of the"
  },
  {
    "startMs": 631880,
    "text": "ways that practically, if we can"
  },
  {
    "startMs": 633160,
    "text": "intentionally produce something that's"
  },
  {
    "startMs": 634520,
    "text": "out of distribution, you can improve"
  },
  {
    "startMs": 636480,
    "text": "this like overall uh quality. And"
  },
  {
    "startMs": 639560,
    "text": "by the way, I I don't think that this"
  },
  {
    "startMs": 641280,
    "text": "can be something just like randomness."
  },
  {
    "startMs": 643320,
    "text": "So, it's not just about like turning up"
  },
  {
    "startMs": 644320,
    "text": "the temperature of the model and and"
  },
  {
    "startMs": 645520,
    "text": "kind of"
  },
  {
    "startMs": 646520,
    "text": "fingers crossed hoping for the best. I"
  },
  {
    "startMs": 647680,
    "text": "think it's much more like how do we"
  },
  {
    "startMs": 648760,
    "text": "understand um even like what are rules"
  },
  {
    "startMs": 652000,
    "text": "or expectations in specific domains?"
  },
  {
    "startMs": 653839,
    "text": "Like let's say that you asked for a"
  },
  {
    "startMs": 655720,
    "text": "slide deck for for the pitch of your"
  },
  {
    "startMs": 657480,
    "text": "startup. Like"
  },
  {
    "startMs": 658720,
    "text": "what is a what does a good pitch deck"
  },
  {
    "startMs": 660240,
    "text": "look like? And then how do you almost"
  },
  {
    "startMs": 661520,
    "text": "like intentionally break rules uh to"
  },
  {
    "startMs": 664160,
    "text": "create things that are more creative,"
  },
  {
    "startMs": 665280,
    "text": "right? Because usually creativity isn't"
  },
  {
    "startMs": 666960,
    "text": "like randomness, isn't doing something"
  },
  {
    "startMs": 668400,
    "text": "that completely feels off for that"
  },
  {
    "startMs": 670760,
    "text": "situation. It's like you intentionally"
  },
  {
    "startMs": 672280,
    "text": "maybe diverge on a couple of things"
  },
  {
    "startMs": 674240,
    "text": "while maintaining kind of"
  },
  {
    "startMs": 676160,
    "text": "um"
  },
  {
    "startMs": 676960,
    "text": "adherence to to expectations of that"
  },
  {
    "startMs": 679160,
    "text": "category, let's say for for others. So,"
  },
  {
    "startMs": 681000,
    "text": "that's one of the things we're working"
  },
  {
    "startMs": 681960,
    "text": "on. The second one of this problem of"
  },
  {
    "startMs": 683520,
    "text": "fit, I think um it's interesting, but"
  },
  {
    "startMs": 686320,
    "text": "brands, as probably a lot of you who are"
  },
  {
    "startMs": 688080,
    "text": "designers know, takes so much effort to"
  },
  {
    "startMs": 690240,
    "text": "create great brands. Like great brands"
  },
  {
    "startMs": 692440,
    "text": "are the work of"
  },
  {
    "startMs": 694240,
    "text": "dozens of designers putting in a lot of"
  },
  {
    "startMs": 696640,
    "text": "like craft and thought and care."
  },
  {
    "startMs": 699080,
    "text": "And so we've almost like already"
  },
  {
    "startMs": 700480,
    "text": "pre-done the work of defining what is"
  },
  {
    "startMs": 701960,
    "text": "great for that specific company and then"
  },
  {
    "startMs": 703520,
    "text": "we're not using it well. So this like"
  },
  {
    "startMs": 705520,
    "text": "brand endurance actually think is a huge"
  },
  {
    "startMs": 706920,
    "text": "problem and one of the things that can"
  },
  {
    "startMs": 708320,
    "text": "very"
  },
  {
    "startMs": 709400,
    "text": "more easily let's say like raise that"
  },
  {
    "startMs": 711040,
    "text": "bar of quality. So I'll I'll touch on an"
  },
  {
    "startMs": 712760,
    "text": "example on this one specifically. And"
  },
  {
    "startMs": 714640,
    "text": "then same with like intent and judgment"
  },
  {
    "startMs": 716160,
    "text": "I think the baby classifiers was a good"
  },
  {
    "startMs": 717600,
    "text": "example. Um"
  },
  {
    "startMs": 719160,
    "text": "like how it how we can actually like use"
  },
  {
    "startMs": 721680,
    "text": "this to be even become a gate for slop"
  },
  {
    "startMs": 723440,
    "text": "and and not let your agent ship slop."
  },
  {
    "startMs": 726640,
    "text": "But so the brand API is the first"
  },
  {
    "startMs": 728080,
    "text": "product that we're releasing to to the"
  },
  {
    "startMs": 729400,
    "text": "public. This is already in in beta"
  },
  {
    "startMs": 731200,
    "text": "testing with a bunch of our design"
  },
  {
    "startMs": 733080,
    "text": "partners. And essentially what it does"
  },
  {
    "startMs": 734840,
    "text": "is it can take let's say a brand URL and"
  },
  {
    "startMs": 738080,
    "text": "extract this into like very specific"
  },
  {
    "startMs": 740080,
    "text": "components that are good for an agent to"
  },
  {
    "startMs": 741800,
    "text": "follow. So basically how do we turn"
  },
  {
    "startMs": 743320,
    "text": "something as fuzzy as a brand into"
  },
  {
    "startMs": 745040,
    "text": "something so structured that it becomes"
  },
  {
    "startMs": 746720,
    "text": "easy to"
  },
  {
    "startMs": 748400,
    "text": "for your agent to follow that but also"
  },
  {
    "startMs": 749760,
    "text": "for you to judge against it, right?"
  },
  {
    "startMs": 750920,
    "text": "Because I think the piece that we can't"
  },
  {
    "startMs": 752280,
    "text": "forget here is this judgment and"
  },
  {
    "startMs": 753960,
    "text": "verification. So yes, this goes and"
  },
  {
    "startMs": 756560,
    "text": "helps your agent to produce something"
  },
  {
    "startMs": 757720,
    "text": "better."
  },
  {
    "startMs": 758720,
    "text": "But how can we also add a way for you to"
  },
  {
    "startMs": 761080,
    "text": "judge okay, is the agent actually"
  },
  {
    "startMs": 762600,
    "text": "staying on track? Is it actually"
  },
  {
    "startMs": 763720,
    "text": "performing well to adhere to this brand"
  },
  {
    "startMs": 765560,
    "text": "or how is it failing or where is it"
  },
  {
    "startMs": 766960,
    "text": "failing? So this is the first flow I"
  },
  {
    "startMs": 769080,
    "text": "would say that we we're seeing that is"
  },
  {
    "startMs": 771360,
    "text": "really helping to improve quality."
  },
  {
    "startMs": 773400,
    "text": "And what's cool is of course we're"
  },
  {
    "startMs": 774640,
    "text": "talking here about an example of a brand"
  },
  {
    "startMs": 776040,
    "text": "that already exists. But let's say you"
  },
  {
    "startMs": 778000,
    "text": "have an agent or you have an app and the"
  },
  {
    "startMs": 781360,
    "text": "person that is using your app actually"
  },
  {
    "startMs": 782480,
    "text": "doesn't have a brand. Let's say they're"
  },
  {
    "startMs": 783560,
    "text": "an average consumer. Can we actually One"
  },
  {
    "startMs": 785760,
    "text": "of the things that we're creating is"
  },
  {
    "startMs": 786600,
    "text": "basically like a repository, like an"
  },
  {
    "startMs": 788440,
    "text": "index of brands"
  },
  {
    "startMs": 790160,
    "text": "of pre almost like pre-created brand"
  },
  {
    "startMs": 792000,
    "text": "system so that if they want something"
  },
  {
    "startMs": 793480,
    "text": "that feels dreamy, why not retrieve a"
  },
  {
    "startMs": 795600,
    "text": "dreamy brand system that already has"
  },
  {
    "startMs": 797160,
    "text": "been thought out to be cohesive instead"
  },
  {
    "startMs": 799200,
    "text": "of doing like a generative approach the"
  },
  {
    "startMs": 801120,
    "text": "moment of that might end up not so great"
  },
  {
    "startMs": 803520,
    "text": "or might end up again in those pillars"
  },
  {
    "startMs": 805000,
    "text": "of slop."
  },
  {
    "startMs": 806800,
    "text": "And I want to show you a real example of"
  },
  {
    "startMs": 808080,
    "text": "this in action. So, um"
  },
  {
    "startMs": 809840,
    "text": "there's this company that I think is"
  },
  {
    "startMs": 810640,
    "text": "awesome called the General Intelligence"
  },
  {
    "startMs": 811800,
    "text": "Company of New York. They have a sick"
  },
  {
    "startMs": 812920,
    "text": "website, you guys should check it out."
  },
  {
    "startMs": 814440,
    "text": "Um but basically, if you ask Cloud"
  },
  {
    "startMs": 815840,
    "text": "Design to create a slide deck uh in"
  },
  {
    "startMs": 818720,
    "text": "their branding,"
  },
  {
    "startMs": 820040,
    "text": "the the middle one is basically what it"
  },
  {
    "startMs": 821360,
    "text": "comes up with. So, the one on the left"
  },
  {
    "startMs": 822520,
    "text": "is is the original brand. Uh this is the"
  },
  {
    "startMs": 825200,
    "text": "kind of the the default. And if you kind"
  },
  {
    "startMs": 827400,
    "text": "of use this extraction actually in the"
  },
  {
    "startMs": 829440,
    "text": "process, it creates something that's way"
  },
  {
    "startMs": 830960,
    "text": "more high fidelity with the original. Um"
  },
  {
    "startMs": 833280,
    "text": "and that even like in the details, I"
  },
  {
    "startMs": 835160,
    "text": "would say like feels right. So, this is"
  },
  {
    "startMs": 836920,
    "text": "just to show an example of it in in"
  },
  {
    "startMs": 838320,
    "text": "action. Um"
  },
  {
    "startMs": 840320,
    "text": "but yeah, I think we"
  },
  {
    "startMs": 844200,
    "text": "I think all of us would agree that like"
  },
  {
    "startMs": 845800,
    "text": "human human taste and kind of the peak"
  },
  {
    "startMs": 847600,
    "text": "of human craft is always going to be"
  },
  {
    "startMs": 850360,
    "text": "like deeply valuable. And that right"
  },
  {
    "startMs": 853080,
    "text": "now, I think the challenge is we are"
  },
  {
    "startMs": 855280,
    "text": "almost even not earning the right to"
  },
  {
    "startMs": 857000,
    "text": "debate this like how can we have"
  },
  {
    "startMs": 859880,
    "text": "uh models like reach this like pinnacle"
  },
  {
    "startMs": 861480,
    "text": "of taste. I don't think it's about that"
  },
  {
    "startMs": 862520,
    "text": "at all. It's like how do we first just"
  },
  {
    "startMs": 864040,
    "text": "like"
  },
  {
    "startMs": 864800,
    "text": "raise the bar. Like the bar is"
  },
  {
    "startMs": 866160,
    "text": "currently, I would say, on the ground."
  },
  {
    "startMs": 867600,
    "text": "And so, I think all of this work that"
  },
  {
    "startMs": 869760,
    "text": "we're putting into like how do we"
  },
  {
    "startMs": 871360,
    "text": "decompose a problem and how do we"
  },
  {
    "startMs": 872600,
    "text": "measure it is exactly so that we can at"
  },
  {
    "startMs": 874240,
    "text": "least like improve this bar um of"
  },
  {
    "startMs": 876440,
    "text": "quality. And I think we have to start"
  },
  {
    "startMs": 877640,
    "text": "with that."
  },
  {
    "startMs": 881240,
    "text": "That's it. Uh thank you very much for"
  },
  {
    "startMs": 883280,
    "text": "for the time. Uh this is this is"
  },
  {
    "startMs": 885120,
    "text": "awesome."
  },
  {
    "startMs": 886225,
    "text": ">> [applause]"
  },
  {
    "startMs": 903910,
    "text": "[music]"
  }
]