[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12719,
    "text": "Hello everyone. Uh I am Adita and we"
  },
  {
    "startMs": 15599,
    "text": "will be talking about uh two main"
  },
  {
    "startMs": 17359,
    "text": "problems that happen on the short form"
  },
  {
    "startMs": 19119,
    "text": "surfaces. We are going to talk about how"
  },
  {
    "startMs": 21359,
    "text": "do we deal with these things at a scale."
  },
  {
    "startMs": 23680,
    "text": "Uh"
  },
  {
    "startMs": 25199,
    "text": "so to get started first thing is we will"
  },
  {
    "startMs": 27279,
    "text": "understand what are the characteristics"
  },
  {
    "startMs": 28640,
    "text": "of the data that we are trying to deal"
  },
  {
    "startMs": 30240,
    "text": "with what are the two main problems that"
  },
  {
    "startMs": 32719,
    "text": "we are actually working on uh and what"
  },
  {
    "startMs": 35840,
    "text": "are the multi- aent systems to solve"
  },
  {
    "startMs": 37680,
    "text": "those problems at a scale what are the"
  },
  {
    "startMs": 40079,
    "text": "different specialized small scale WM"
  },
  {
    "startMs": 42800,
    "text": "that we can uh build uh to solve those"
  },
  {
    "startMs": 45680,
    "text": "individual agentic problem and how do we"
  },
  {
    "startMs": 48239,
    "text": "build those agents what are the"
  },
  {
    "startMs": 49680,
    "text": "different ways to actually optimize it"
  },
  {
    "startMs": 51520,
    "text": "like to make it like scalable at a very"
  },
  {
    "startMs": 53760,
    "text": "big scale and then we will be looking"
  },
  {
    "startMs": 56160,
    "text": "into the evaluation in a very holistic"
  },
  {
    "startMs": 58160,
    "text": "360 manner not just like a precision"
  },
  {
    "startMs": 60000,
    "text": "recall what is out there how do we"
  },
  {
    "startMs": 62079,
    "text": "understand the entire multi-agenting"
  },
  {
    "startMs": 64239,
    "text": "pipeline both from LLM level tools MCP"
  },
  {
    "startMs": 67040,
    "text": "and whatn not everything out there and"
  },
  {
    "startMs": 69520,
    "text": "then we will be getting into the"
  },
  {
    "startMs": 70960,
    "text": "optimization techniques which this which"
  },
  {
    "startMs": 72720,
    "text": "are very vision specific and metadata"
  },
  {
    "startMs": 75119,
    "text": "specific which will allow us and give us"
  },
  {
    "startMs": 77600,
    "text": "some intelligence to tell that we really"
  },
  {
    "startMs": 80080,
    "text": "don't need to do intell like this"
  },
  {
    "startMs": 81759,
    "text": "intelligent like workflow So for all the"
  },
  {
    "startMs": 83759,
    "text": "videos we can figure out what are the"
  },
  {
    "startMs": 85680,
    "text": "small set of videos which should be the"
  },
  {
    "startMs": 87759,
    "text": "candidate for doing these things and we"
  },
  {
    "startMs": 89759,
    "text": "will conclude with the takeaways."
  },
  {
    "startMs": 92640,
    "text": "So the real life data is very messy. It"
  },
  {
    "startMs": 95280,
    "text": "is we are talking about at a scale of"
  },
  {
    "startMs": 97119,
    "text": "100 million plus and a lot more viral"
  },
  {
    "startMs": 99600,
    "text": "content. There's a lot of adversal"
  },
  {
    "startMs": 101280,
    "text": "content people are trying to gain the"
  },
  {
    "startMs": 103200,
    "text": "system. There are a lot of uh"
  },
  {
    "startMs": 104560,
    "text": "multilingual text on the screens and in"
  },
  {
    "startMs": 106880,
    "text": "their videos and images. And the uh data"
  },
  {
    "startMs": 110079,
    "text": "is very dynamic. it keeps on changing"
  },
  {
    "startMs": 112000,
    "text": "from like month to month different AI"
  },
  {
    "startMs": 113600,
    "text": "tools are coming everything out there so"
  },
  {
    "startMs": 115759,
    "text": "it's it's really really dynamic that"
  },
  {
    "startMs": 117439,
    "text": "means there is a lot of drift issues and"
  },
  {
    "startMs": 118960,
    "text": "other thing that comes with a video data"
  },
  {
    "startMs": 121040,
    "text": "and then we know that there is no clear"
  },
  {
    "startMs": 123600,
    "text": "ground truth for the problem that we're"
  },
  {
    "startMs": 125040,
    "text": "trying to solve so these are like"
  },
  {
    "startMs": 126479,
    "text": "existing problem which happens in a real"
  },
  {
    "startMs": 128640,
    "text": "world video data set we will be the"
  },
  {
    "startMs": 131680,
    "text": "first problem that we will be talking"
  },
  {
    "startMs": 133040,
    "text": "about is the modality misalignment where"
  },
  {
    "startMs": 135440,
    "text": "the first is intramodality which is a"
  },
  {
    "startMs": 138400,
    "text": "very solved problem you can have a clip"
  },
  {
    "startMs": 140480,
    "text": "model you can have a modality on uh uh"
  },
  {
    "startMs": 143040,
    "text": "images, videos, uh audio, text and"
  },
  {
    "startMs": 145840,
    "text": "figure out what exactly is the cosign"
  },
  {
    "startMs": 147440,
    "text": "similarity on those embedding and figure"
  },
  {
    "startMs": 148959,
    "text": "it out. So this is a much simpler"
  },
  {
    "startMs": 150720,
    "text": "problem. What we will be talking about"
  },
  {
    "startMs": 153280,
    "text": "how we are doing intraodality u issues."
  },
  {
    "startMs": 157280,
    "text": "So this is consider that like a video we"
  },
  {
    "startMs": 159519,
    "text": "have a big video and then uh suddenly"
  },
  {
    "startMs": 161519,
    "text": "you are seeing something some ad agenda"
  },
  {
    "startMs": 163920,
    "text": "some political things or something out"
  },
  {
    "startMs": 165760,
    "text": "there which shouldn't be there in the"
  },
  {
    "startMs": 167040,
    "text": "video as when you clicked on it. So how"
  },
  {
    "startMs": 169840,
    "text": "do we actually look into the small"
  },
  {
    "startMs": 171760,
    "text": "segments of the uh video understand this"
  },
  {
    "startMs": 174400,
    "text": "problem and figure out where there is an"
  },
  {
    "startMs": 176640,
    "text": "anomaly or kind of like adversarial"
  },
  {
    "startMs": 178319,
    "text": "behavior out there which shouldn't be"
  },
  {
    "startMs": 179920,
    "text": "there in the first place. So this is the"
  },
  {
    "startMs": 181680,
    "text": "first problem which requires a lot of"
  },
  {
    "startMs": 183360,
    "text": "granular understanding vision"
  },
  {
    "startMs": 185120,
    "text": "understanding and a video understanding"
  },
  {
    "startMs": 186640,
    "text": "to see what is happening at a very clip"
  },
  {
    "startMs": 188560,
    "text": "and a very frame level. The second"
  },
  {
    "startMs": 190959,
    "text": "problem is uh to understand the"
  },
  {
    "startMs": 192800,
    "text": "unoriginal content. In today's economy"
  },
  {
    "startMs": 195200,
    "text": "with the AI tools available, it's really"
  },
  {
    "startMs": 197040,
    "text": "easy to actually duplicate the content."
  },
  {
    "startMs": 199840,
    "text": "When someone uh upload a video or"
  },
  {
    "startMs": 202319,
    "text": "something, we see that okay, it is being"
  },
  {
    "startMs": 204319,
    "text": "copied, it is being transformed and with"
  },
  {
    "startMs": 206159,
    "text": "tools, it is getting much easier to"
  },
  {
    "startMs": 208159,
    "text": "transform these videos. U so how do we"
  },
  {
    "startMs": 210959,
    "text": "actually detect this kind of unoriginal"
  },
  {
    "startMs": 212799,
    "text": "content? How do we figure out the source"
  },
  {
    "startMs": 214239,
    "text": "of the video? And um this has actually"
  },
  {
    "startMs": 217680,
    "text": "caused a problem in attribution credit u"
  },
  {
    "startMs": 220560,
    "text": "ecosystem imbalance and lot of like user"
  },
  {
    "startMs": 222640,
    "text": "fatigue where we are seeing a lot of"
  },
  {
    "startMs": 224319,
    "text": "repetitive videos which shouldn't be"
  },
  {
    "startMs": 225840,
    "text": "there in the first place. So uh coming"
  },
  {
    "startMs": 229280,
    "text": "to the multi- aent system uh the first"
  },
  {
    "startMs": 231280,
    "text": "thing is that uh why we are going to"
  },
  {
    "startMs": 233680,
    "text": "multi-agent not a single agent or single"
  },
  {
    "startMs": 235360,
    "text": "lenm because the problem is really"
  },
  {
    "startMs": 236959,
    "text": "complex. It requires a really"
  },
  {
    "startMs": 238720,
    "text": "specialized uh nodes and uh kind of like"
  },
  {
    "startMs": 241840,
    "text": "an understanding at each part of"
  },
  {
    "startMs": 243439,
    "text": "retrieval content understanding and then"
  },
  {
    "startMs": 245920,
    "text": "reasoning and if you can solve a problem"
  },
  {
    "startMs": 248480,
    "text": "with one agent one LLM you we don't need"
  },
  {
    "startMs": 250799,
    "text": "to actually do it on multi- aent"
  },
  {
    "startMs": 253040,
    "text": "systems. So this is where the"
  },
  {
    "startMs": 256160,
    "text": "centralized brain how we are actually"
  },
  {
    "startMs": 258079,
    "text": "thinking kind of like how the uh"
  },
  {
    "startMs": 259759,
    "text": "decomposition happens of this problem."
  },
  {
    "startMs": 262079,
    "text": "The first is basically you have a video"
  },
  {
    "startMs": 264320,
    "text": "and kind of like a clip video ID and"
  },
  {
    "startMs": 266560,
    "text": "everything given to a reviewer agent."
  },
  {
    "startMs": 268560,
    "text": "This is a centralized agent. This is"
  },
  {
    "startMs": 270240,
    "text": "basically the orchestrator. Consider"
  },
  {
    "startMs": 271840,
    "text": "that as an API gateway of uh doing a"
  },
  {
    "startMs": 274479,
    "text": "sing signal decomposition and finding"
  },
  {
    "startMs": 276400,
    "text": "what is happening around in the image."
  },
  {
    "startMs": 278800,
    "text": "So what this guy do is basically this"
  },
  {
    "startMs": 280960,
    "text": "guy takes ask the perceiver agent and"
  },
  {
    "startMs": 283840,
    "text": "perceiver agent is the one which is"
  },
  {
    "startMs": 285840,
    "text": "basically considered that a very"
  },
  {
    "startMs": 287440,
    "text": "sophisticated VLM expert with lot of"
  },
  {
    "startMs": 289840,
    "text": "image and the video tools at uh"
  },
  {
    "startMs": 292400,
    "text": "disposal. Uh perceiver agent will take"
  },
  {
    "startMs": 295520,
    "text": "the ID from the retriever reviewer agent"
  },
  {
    "startMs": 298800,
    "text": "and it fetches the video from the"
  },
  {
    "startMs": 301199,
    "text": "database. It decomposes into smaller"
  },
  {
    "startMs": 303280,
    "text": "parts using different tools, different"
  },
  {
    "startMs": 305040,
    "text": "like semantic embeddings and different"
  },
  {
    "startMs": 306720,
    "text": "kind of like temporal change that"
  },
  {
    "startMs": 308160,
    "text": "happens which is a little bit beyond our"
  },
  {
    "startMs": 311039,
    "text": "scope but of this talk but it's uh a"
  },
  {
    "startMs": 314000,
    "text": "technique which is making sure that we"
  },
  {
    "startMs": 315600,
    "text": "are not doing at a fixed frame rate but"
  },
  {
    "startMs": 318160,
    "text": "we are finding where the temporal change"
  },
  {
    "startMs": 319840,
    "text": "happen and compressing those similar"
  },
  {
    "startMs": 322080,
    "text": "frames into one or two frames and then"
  },
  {
    "startMs": 325120,
    "text": "the perceiver agent get all the data"
  },
  {
    "startMs": 326800,
    "text": "from a clip level. The video embedding"
  },
  {
    "startMs": 328720,
    "text": "level get all the information about a"
  },
  {
    "startMs": 331039,
    "text": "clip embedding tags OCR whatever is"
  },
  {
    "startMs": 333440,
    "text": "present out there what is the natural"
  },
  {
    "startMs": 335280,
    "text": "language description and what are the"
  },
  {
    "startMs": 336800,
    "text": "time stamp from which frame to which"
  },
  {
    "startMs": 338400,
    "text": "frame what are those this metadata looks"
  },
  {
    "startMs": 340320,
    "text": "like it gives that data to the reviewer"
  },
  {
    "startMs": 343120,
    "text": "reviewer will look into all the temporal"
  },
  {
    "startMs": 345199,
    "text": "this JSON object provided by the uh"
  },
  {
    "startMs": 347440,
    "text": "perceiver agent in a raw form along with"
  },
  {
    "startMs": 349919,
    "text": "embeddings along with semantic ids tags"
  },
  {
    "startMs": 352800,
    "text": "OCR and everything and it does a"
  },
  {
    "startMs": 354720,
    "text": "temporal analysis it looks into okay you"
  },
  {
    "startMs": 357039,
    "text": "know for from first frame to frame or"
  },
  {
    "startMs": 360160,
    "text": "frame number 360 or like till 6 second"
  },
  {
    "startMs": 362800,
    "text": "it was a video talking about a sports"
  },
  {
    "startMs": 365440,
    "text": "and suddenly we are seeing from 6 second"
  },
  {
    "startMs": 368160,
    "text": "to 6.5 second or frame number this to"
  },
  {
    "startMs": 370240,
    "text": "this we are seeing that like this is"
  },
  {
    "startMs": 371759,
    "text": "changing into some political thing. So"
  },
  {
    "startMs": 374960,
    "text": "just by looking into the metadata the"
  },
  {
    "startMs": 377120,
    "text": "reviewer agent is able to understand"
  },
  {
    "startMs": 379120,
    "text": "comprehend and figure out what exactly"
  },
  {
    "startMs": 380960,
    "text": "is the anomaly coming in the this"
  },
  {
    "startMs": 382560,
    "text": "temporal space and once it do that it"
  },
  {
    "startMs": 385600,
    "text": "figures out whether this is actually a"
  },
  {
    "startMs": 387120,
    "text": "modality misalignment or there is some"
  },
  {
    "startMs": 388880,
    "text": "bigger issue out there on there. So"
  },
  {
    "startMs": 390800,
    "text": "review agent will look and talk to the"
  },
  {
    "startMs": 393199,
    "text": "retriever agent and it tells that hey"
  },
  {
    "startMs": 395039,
    "text": "this is the video I'm looking into."
  },
  {
    "startMs": 396639,
    "text": "These are the some of the metadata that"
  },
  {
    "startMs": 398479,
    "text": "I have already uh kind of like ddup and"
  },
  {
    "startMs": 400560,
    "text": "kind of like post-processed. Now give me"
  },
  {
    "startMs": 402880,
    "text": "some understanding of this similar clips"
  },
  {
    "startMs": 405120,
    "text": "which are available in the corpus. So"
  },
  {
    "startMs": 407600,
    "text": "retriever agent will look into all the"
  },
  {
    "startMs": 409680,
    "text": "signals provided by perceiver agent. All"
  },
  {
    "startMs": 411840,
    "text": "the metadata of a clip level and a whole"
  },
  {
    "startMs": 413919,
    "text": "video level and index it into different"
  },
  {
    "startMs": 416479,
    "text": "databases adaptively. For example the"
  },
  {
    "startMs": 418720,
    "text": "topics has to be maybe like a inverted"
  },
  {
    "startMs": 420960,
    "text": "index where we have topics and a lot of"
  },
  {
    "startMs": 422639,
    "text": "videos. For the embedding it will be a"
  },
  {
    "startMs": 424720,
    "text": "vector databases which is available out"
  },
  {
    "startMs": 426800,
    "text": "there. And for different kind of"
  },
  {
    "startMs": 428720,
    "text": "entities which I've extracted we have a"
  },
  {
    "startMs": 430720,
    "text": "graph databases kind of like where you"
  },
  {
    "startMs": 432560,
    "text": "review your agent will figure out okay"
  },
  {
    "startMs": 434160,
    "text": "these are the multiple databases these"
  },
  {
    "startMs": 435759,
    "text": "are the entities and these are the"
  },
  {
    "startMs": 436960,
    "text": "metadata let me index it so that in the"
  },
  {
    "startMs": 439520,
    "text": "online inference time I can figure out"
  },
  {
    "startMs": 441599,
    "text": "for a particular clip what are the"
  },
  {
    "startMs": 442960,
    "text": "similar clips what are the similar"
  },
  {
    "startMs": 444319,
    "text": "entities and topics which are available"
  },
  {
    "startMs": 446319,
    "text": "to fetch and improve the recall."
  },
  {
    "startMs": 449039,
    "text": "So going into individual agent we talked"
  },
  {
    "startMs": 451520,
    "text": "about perceiver it looked into the"
  },
  {
    "startMs": 453120,
    "text": "entire video do a temporal decomposition"
  },
  {
    "startMs": 455599,
    "text": "into small clips based on the semantic"
  },
  {
    "startMs": 457919,
    "text": "embedding and couple of algorithms and"
  },
  {
    "startMs": 460080,
    "text": "it fine-tuned the VLM to actually emit"
  },
  {
    "startMs": 462400,
    "text": "all the real uh concrete and a very um"
  },
  {
    "startMs": 466639,
    "text": "granular information about like each"
  },
  {
    "startMs": 468880,
    "text": "clips and the entire video. Uh because"
  },
  {
    "startMs": 471680,
    "text": "we are dealing at a very big scale that"
  },
  {
    "startMs": 473759,
    "text": "means we cannot just go with the"
  },
  {
    "startMs": 475199,
    "text": "standard VLM which are out there. There"
  },
  {
    "startMs": 476879,
    "text": "has to be a compression. There has to be"
  },
  {
    "startMs": 478639,
    "text": "a cost effective way to actually serve"
  },
  {
    "startMs": 480560,
    "text": "this these models to do it at like"
  },
  {
    "startMs": 483840,
    "text": "billions of frames. That's where we are"
  },
  {
    "startMs": 486160,
    "text": "getting into the pre-training"
  },
  {
    "startMs": 487680,
    "text": "fine-tuning and knowledge distillation"
  },
  {
    "startMs": 489520,
    "text": "quantization to actually deploy and"
  },
  {
    "startMs": 491919,
    "text": "solve the very very specialized VLM"
  },
  {
    "startMs": 494720,
    "text": "based on this particular problem and we"
  },
  {
    "startMs": 496720,
    "text": "will be talking a little bit brief about"
  },
  {
    "startMs": 498080,
    "text": "that. U the retriever agent is the one"
  },
  {
    "startMs": 500800,
    "text": "which we talked at a very high level and"
  },
  {
    "startMs": 502560,
    "text": "then what it does in an offline"
  },
  {
    "startMs": 504080,
    "text": "processing is that all the signal which"
  },
  {
    "startMs": 506560,
    "text": "is decomposed by the perceiver agent in"
  },
  {
    "startMs": 508560,
    "text": "an offline fashion consider that rather"
  },
  {
    "startMs": 510319,
    "text": "an agent you are actually taking that"
  },
  {
    "startMs": 511840,
    "text": "library and doing an offline analysis on"
  },
  {
    "startMs": 513760,
    "text": "a big scale let's say ray cluster or"
  },
  {
    "startMs": 516560,
    "text": "something like that and then once you"
  },
  {
    "startMs": 518719,
    "text": "have all those metadata it is basically"
  },
  {
    "startMs": 520800,
    "text": "indexing it into different databases"
  },
  {
    "startMs": 523039,
    "text": "doing an uh periodic offline clustering"
  },
  {
    "startMs": 525360,
    "text": "and finding what are the similar content"
  },
  {
    "startMs": 527120,
    "text": "what should be the embedding ID what"
  },
  {
    "startMs": 528399,
    "text": "should be cluster ID for each clips and"
  },
  {
    "startMs": 530240,
    "text": "entire video. So that is a data which is"
  },
  {
    "startMs": 532560,
    "text": "basically used by online inference to"
  },
  {
    "startMs": 535040,
    "text": "find similar videos and in online manner"
  },
  {
    "startMs": 537920,
    "text": "it given a query a clip ID and every all"
  },
  {
    "startMs": 540720,
    "text": "the metadata details it's figured out"
  },
  {
    "startMs": 542480,
    "text": "what is out there in the corpus which"
  },
  {
    "startMs": 544320,
    "text": "this guy has is similar to what like"
  },
  {
    "startMs": 546880,
    "text": "what are the different things which has"
  },
  {
    "startMs": 548160,
    "text": "a high similarity. So what it does it"
  },
  {
    "startMs": 550959,
    "text": "looks into databases is it find similar"
  },
  {
    "startMs": 552800,
    "text": "clips and like similar authors and"
  },
  {
    "startMs": 554399,
    "text": "different other metadata information"
  },
  {
    "startMs": 556399,
    "text": "rerank those candidates find uh and use"
  },
  {
    "startMs": 559120,
    "text": "some of the tools about like for example"
  },
  {
    "startMs": 561200,
    "text": "the traditional model span score like a"
  },
  {
    "startMs": 563040,
    "text": "classifier score to see what are the"
  },
  {
    "startMs": 565120,
    "text": "different candidates which might be spam"
  },
  {
    "startMs": 567200,
    "text": "which might not be high quality and"
  },
  {
    "startMs": 568640,
    "text": "those things out there and once it has"
  },
  {
    "startMs": 570880,
    "text": "those top end candidates is give back it"
  },
  {
    "startMs": 573120,
    "text": "to the reviewer and reviewer is where it"
  },
  {
    "startMs": 575440,
    "text": "actually process this is my main content"
  },
  {
    "startMs": 577680,
    "text": "signals and clip embeddings These are a"
  },
  {
    "startMs": 580240,
    "text": "similar video which is given by"
  },
  {
    "startMs": 581600,
    "text": "retriever. Let me think over it and"
  },
  {
    "startMs": 583920,
    "text": "whether I need to retraate and get more"
  },
  {
    "startMs": 587279,
    "text": "data from retriever. So uh that's where"
  },
  {
    "startMs": 590240,
    "text": "reviewer have all the temporal signals"
  },
  {
    "startMs": 591839,
    "text": "from a single clip. It has all the"
  },
  {
    "startMs": 594000,
    "text": "information of similar clips"
  },
  {
    "startMs": 595360,
    "text": "understanding and what is similar"
  },
  {
    "startMs": 596800,
    "text": "authors and other thing. It also has a"
  },
  {
    "startMs": 599360,
    "text": "realtime information about how the users"
  },
  {
    "startMs": 601440,
    "text": "are actually interacting with this"
  },
  {
    "startMs": 602959,
    "text": "video. What are the different kind of"
  },
  {
    "startMs": 605120,
    "text": "like a reports or like likes, dislike,"
  },
  {
    "startMs": 607440,
    "text": "the comments? what is the u sentiment of"
  },
  {
    "startMs": 611279,
    "text": "those comments out there right so a lot"
  },
  {
    "startMs": 613519,
    "text": "of these signals which are actually"
  },
  {
    "startMs": 614880,
    "text": "missed by the offline uh signals and the"
  },
  {
    "startMs": 616959,
    "text": "VLMs and other kind of agents are"
  },
  {
    "startMs": 618720,
    "text": "actually also incorporated to see if"
  },
  {
    "startMs": 620880,
    "text": "there is a change happen in the"
  },
  {
    "startMs": 622160,
    "text": "sentiment what is the response I'm"
  },
  {
    "startMs": 624079,
    "text": "getting in the real time so there are"
  },
  {
    "startMs": 625920,
    "text": "different tools available for this to"
  },
  {
    "startMs": 628000,
    "text": "actually understand the video not just"
  },
  {
    "startMs": 630079,
    "text": "from the content and from the semantic"
  },
  {
    "startMs": 632480,
    "text": "perspective but understanding from the"
  },
  {
    "startMs": 634640,
    "text": "user interaction perspective those"
  },
  {
    "startMs": 636399,
    "text": "signals are really really important"
  },
  {
    "startMs": 638560,
    "text": "So once we have this judge and"
  },
  {
    "startMs": 640640,
    "text": "everything we build this uh agentic"
  },
  {
    "startMs": 643120,
    "text": "framework and each of these agentic"
  },
  {
    "startMs": 645519,
    "text": "frameworks uh all all these three agents"
  },
  {
    "startMs": 648480,
    "text": "are actually powered by specialized uh"
  },
  {
    "startMs": 650800,
    "text": "VLMs and small scale VLMs in a way. So"
  },
  {
    "startMs": 654800,
    "text": "we will be talking about uh pre-training"
  },
  {
    "startMs": 657519,
    "text": "first. Now uh in most of the cases you"
  },
  {
    "startMs": 660320,
    "text": "see that okay you have a pre-training"
  },
  {
    "startMs": 661920,
    "text": "you fine-tune your vision transform a"
  },
  {
    "startMs": 663519,
    "text": "little bit here and there and basically"
  },
  {
    "startMs": 664880,
    "text": "tune it for your specific purpose."
  },
  {
    "startMs": 667600,
    "text": "Uh the thing is that these VLMs what we"
  },
  {
    "startMs": 670720,
    "text": "have outside and available the"
  },
  {
    "startMs": 672480,
    "text": "foundational model the front end models"
  },
  {
    "startMs": 674320,
    "text": "they are trained on a very clean very"
  },
  {
    "startMs": 676720,
    "text": "nice data set very web data which is"
  },
  {
    "startMs": 678640,
    "text": "very well tuned cleaned and everything"
  },
  {
    "startMs": 680640,
    "text": "but uh data inhouse for a specific"
  },
  {
    "startMs": 683360,
    "text": "purpose is not having the same data"
  },
  {
    "startMs": 685600,
    "text": "characteristics. It's it is messy. It is"
  },
  {
    "startMs": 687839,
    "text": "generated by users. It is for your"
  },
  {
    "startMs": 689600,
    "text": "specific workflow. That means you need"
  },
  {
    "startMs": 692000,
    "text": "to pre-train on those image tokens and"
  },
  {
    "startMs": 694320,
    "text": "language to fine-tune your vision"
  },
  {
    "startMs": 696399,
    "text": "transformer from scratch and see whether"
  },
  {
    "startMs": 698720,
    "text": "there is a delta in fine-tuning it and"
  },
  {
    "startMs": 700720,
    "text": "tuning it from the scratch. That's where"
  },
  {
    "startMs": 702560,
    "text": "the pre-training is helpful. It's little"
  },
  {
    "startMs": 704720,
    "text": "expensive but um it if if it can get a"
  },
  {
    "startMs": 707600,
    "text": "delta that that actually works really"
  },
  {
    "startMs": 709839,
    "text": "well. The second is instruction"
  },
  {
    "startMs": 711839,
    "text": "fine-tuning where we have the two"
  },
  {
    "startMs": 714399,
    "text": "problems. We have a certain policies and"
  },
  {
    "startMs": 716399,
    "text": "certain guidelines. We know the content"
  },
  {
    "startMs": 718399,
    "text": "out there, signal out there and it"
  },
  {
    "startMs": 720240,
    "text": "understand and tune it on that"
  },
  {
    "startMs": 723200,
    "text": "particular data set with it with a"
  },
  {
    "startMs": 724959,
    "text": "particular output which is schema like a"
  },
  {
    "startMs": 727440,
    "text": "JSON schema to understand what is the"
  },
  {
    "startMs": 729040,
    "text": "modality uh misalignment or duplication"
  },
  {
    "startMs": 731839,
    "text": "of scores and other chain of thought"
  },
  {
    "startMs": 734000,
    "text": "reasoning provided by reviewer agent. So"
  },
  {
    "startMs": 736560,
    "text": "here we have a video clip we have a"
  },
  {
    "startMs": 738160,
    "text": "vision encoder which we are already"
  },
  {
    "startMs": 740399,
    "text": "pre-trained a little bit like now we are"
  },
  {
    "startMs": 741920,
    "text": "doing a little bit more training on"
  },
  {
    "startMs": 743040,
    "text": "that. There's a projector which is"
  },
  {
    "startMs": 744880,
    "text": "sitting between the vision transformer"
  },
  {
    "startMs": 746560,
    "text": "and the L language models and it is"
  },
  {
    "startMs": 748800,
    "text": "actually basically a bridge between them"
  },
  {
    "startMs": 750880,
    "text": "and then we have our output depending on"
  },
  {
    "startMs": 752639,
    "text": "what is the instruction finetuning data"
  },
  {
    "startMs": 754639,
    "text": "we have on this side. So this is the"
  },
  {
    "startMs": 757680,
    "text": "critical part of actually making your uh"
  },
  {
    "startMs": 761519,
    "text": "model performance like go up for your"
  },
  {
    "startMs": 764079,
    "text": "domain specific. So uh the context is"
  },
  {
    "startMs": 767040,
    "text": "basically you have a role, you have a"
  },
  {
    "startMs": 768720,
    "text": "policy, what are the tools available to"
  },
  {
    "startMs": 770639,
    "text": "ground that into some of the metadata or"
  },
  {
    "startMs": 772959,
    "text": "and what are the other things available"
  },
  {
    "startMs": 774480,
    "text": "out there give it entire things into uh"
  },
  {
    "startMs": 777120,
    "text": "in in a very brief manner into the"
  },
  {
    "startMs": 779120,
    "text": "context and let it figure out like what"
  },
  {
    "startMs": 781440,
    "text": "the structured label you have for these"
  },
  {
    "startMs": 783200,
    "text": "labels are actually in-house label."
  },
  {
    "startMs": 785279,
    "text": "These are the one where we have created"
  },
  {
    "startMs": 787279,
    "text": "like what exactly modality things are,"
  },
  {
    "startMs": 790240,
    "text": "what are the different uh issues we are"
  },
  {
    "startMs": 792240,
    "text": "seeing, what are the different chain of"
  },
  {
    "startMs": 793680,
    "text": "thought reasoning there should be there"
  },
  {
    "startMs": 795040,
    "text": "in the model and how does the output"
  },
  {
    "startMs": 797279,
    "text": "looks like for a human reviewer. So this"
  },
  {
    "startMs": 799440,
    "text": "is like a very high quality data data"
  },
  {
    "startMs": 801680,
    "text": "set that we are fine-tuning it on for"
  },
  {
    "startMs": 803920,
    "text": "different agents. Right? So once we have"
  },
  {
    "startMs": 807360,
    "text": "done with the pre-training just to"
  },
  {
    "startMs": 808800,
    "text": "understand the vision aspect of or the"
  },
  {
    "startMs": 810800,
    "text": "other modality aspects of the videos"
  },
  {
    "startMs": 813360,
    "text": "then we go into the finetuning to make"
  },
  {
    "startMs": 815200,
    "text": "it understand and provide the context"
  },
  {
    "startMs": 817440,
    "text": "and the output in the manner that we"
  },
  {
    "startMs": 819200,
    "text": "would want to actually process the data"
  },
  {
    "startMs": 820800,
    "text": "set on. So u the next phase is the DPO"
  },
  {
    "startMs": 824800,
    "text": "phase where uh which is basically this"
  },
  {
    "startMs": 827120,
    "text": "technique is used a lot in the uh post-"
  },
  {
    "startMs": 829519,
    "text": "training to actually fine-tune our"
  },
  {
    "startMs": 831040,
    "text": "models into a specific uh like realm"
  },
  {
    "startMs": 834639,
    "text": "domain policy understanding or"
  },
  {
    "startMs": 836639,
    "text": "something. But this can also be used a"
  },
  {
    "startMs": 839040,
    "text": "lot in the prediction for actually"
  },
  {
    "startMs": 840720,
    "text": "understanding what are the samples which"
  },
  {
    "startMs": 842480,
    "text": "are actually getting uh not so good by"
  },
  {
    "startMs": 845120,
    "text": "the by your multi-ent systems and by"
  },
  {
    "startMs": 847519,
    "text": "your u llms. So what uh can be done is"
  },
  {
    "startMs": 852800,
    "text": "that like uh you have this uh production"
  },
  {
    "startMs": 855199,
    "text": "data set which is coming out you have a"
  },
  {
    "startMs": 857199,
    "text": "lot of inference happening you take a"
  },
  {
    "startMs": 859279,
    "text": "subsample of these uh data which is from"
  },
  {
    "startMs": 862320,
    "text": "the production you pass it through LLM"
  },
  {
    "startMs": 864639,
    "text": "as a judge which is trained inhouse on"
  },
  {
    "startMs": 866720,
    "text": "the human label data set and then you"
  },
  {
    "startMs": 869600,
    "text": "have the human review Q to see what is"
  },
  {
    "startMs": 871519,
    "text": "the performance coming up on on the"
  },
  {
    "startMs": 873440,
    "text": "actual system. Once you have this thing"
  },
  {
    "startMs": 876320,
    "text": "if your performance is coming amazing"
  },
  {
    "startMs": 878320,
    "text": "and it is above like your whatever is"
  },
  {
    "startMs": 880000,
    "text": "the prediction threshold you have 95%"
  },
  {
    "startMs": 882560,
    "text": "for each of the problem it's great but"
  },
  {
    "startMs": 885279,
    "text": "if it is not that means there is a way"
  },
  {
    "startMs": 887199,
    "text": "there has to be a way to loan on these"
  },
  {
    "startMs": 889279,
    "text": "samples where the model did not do well"
  },
  {
    "startMs": 892079,
    "text": "that's where you have a human in the"
  },
  {
    "startMs": 893680,
    "text": "queue it understand those all the traces"
  },
  {
    "startMs": 896240,
    "text": "which is there from all the agents the"
  },
  {
    "startMs": 898079,
    "text": "LLM call MCP and it figured out where"
  },
  {
    "startMs": 900240,
    "text": "the problem happened is it like a chain"
  },
  {
    "startMs": 902240,
    "text": "of thought reasoning or is it like some"
  },
  {
    "startMs": 904480,
    "text": "of the wrong tool tools are called the"
  },
  {
    "startMs": 905839,
    "text": "retrieval did not work. So that entire"
  },
  {
    "startMs": 908480,
    "text": "validations and basically understanding"
  },
  {
    "startMs": 910560,
    "text": "of each and every hook and node both at"
  },
  {
    "startMs": 913279,
    "text": "a model intelligence level plus at a"
  },
  {
    "startMs": 916320,
    "text": "harness level is what you figure out and"
  },
  {
    "startMs": 919199,
    "text": "say that okay these are the improvement"
  },
  {
    "startMs": 920800,
    "text": "that we I need to do on these samples"
  },
  {
    "startMs": 923040,
    "text": "which was incorrectly um sampled by our"
  },
  {
    "startMs": 926880,
    "text": "um system. So once you have this thing"
  },
  {
    "startMs": 929600,
    "text": "you have a positive sample negative"
  },
  {
    "startMs": 931360,
    "text": "sample what needs to be updated and"
  },
  {
    "startMs": 933519,
    "text": "that's where you retrain your model to"
  },
  {
    "startMs": 936000,
    "text": "see what uh how can we improve it better"
  },
  {
    "startMs": 938480,
    "text": "and this is the continuous improvement"
  },
  {
    "startMs": 940480,
    "text": "where we are looking into these samples"
  },
  {
    "startMs": 942399,
    "text": "reiterating re um improving the models"
  },
  {
    "startMs": 945199,
    "text": "and having a new data set out there from"
  },
  {
    "startMs": 947519,
    "text": "the production u like samples and trying"
  },
  {
    "startMs": 950560,
    "text": "to understand whether the drift happen"
  },
  {
    "startMs": 952079,
    "text": "or what exactly the model is doing. So"
  },
  {
    "startMs": 954560,
    "text": "this human in the loop is like always a"
  },
  {
    "startMs": 956480,
    "text": "continuous thing where you have a daily"
  },
  {
    "startMs": 958240,
    "text": "sampling from the production and trying"
  },
  {
    "startMs": 960160,
    "text": "to understand like how the model and the"
  },
  {
    "startMs": 962480,
    "text": "L&M as a judge are doing."
  },
  {
    "startMs": 965279,
    "text": "So uh because of the cost and ability to"
  },
  {
    "startMs": 970079,
    "text": "solve at this scale we cannot go with"
  },
  {
    "startMs": 972399,
    "text": "the standard like a like a VLM models"
  },
  {
    "startMs": 975040,
    "text": "from frontier sizes because it's not"
  },
  {
    "startMs": 977600,
    "text": "scalable. It requires a lot of"
  },
  {
    "startMs": 979040,
    "text": "inference. It's a lot of complexity and"
  },
  {
    "startMs": 980639,
    "text": "we are solving a very specific problem"
  },
  {
    "startMs": 983040,
    "text": "like I really don't care if the model"
  },
  {
    "startMs": 985040,
    "text": "can solve a coding problem. I only care"
  },
  {
    "startMs": 987040,
    "text": "about my domain specific problem. That"
  },
  {
    "startMs": 989440,
    "text": "is all I care. This is not exposed to"
  },
  {
    "startMs": 991120,
    "text": "the customer. This is an internal thing."
  },
  {
    "startMs": 993120,
    "text": "So I would actually do a off policy and"
  },
  {
    "startMs": 995680,
    "text": "on policy knowledge distillation and do"
  },
  {
    "startMs": 997839,
    "text": "a little bit of quantizations depending"
  },
  {
    "startMs": 999440,
    "text": "on like some experimentation"
  },
  {
    "startMs": 1000880,
    "text": "quantization to understand whether a"
  },
  {
    "startMs": 1002399,
    "text": "forbit work braining float what actually"
  },
  {
    "startMs": 1005199,
    "text": "work really well. Then I would have a"
  },
  {
    "startMs": 1006800,
    "text": "table of different sizes of"
  },
  {
    "startMs": 1008959,
    "text": "distillations and quantization and"
  },
  {
    "startMs": 1010800,
    "text": "understand and see where exactly is my"
  },
  {
    "startMs": 1013519,
    "text": "performance like up to the mark and"
  },
  {
    "startMs": 1015920,
    "text": "where I'm gaining a lot of compute and"
  },
  {
    "startMs": 1018079,
    "text": "the cost resources um savings from on"
  },
  {
    "startMs": 1020639,
    "text": "inference production by doing these"
  },
  {
    "startMs": 1022560,
    "text": "optimization. So this is really critical"
  },
  {
    "startMs": 1025679,
    "text": "because as a problem what we have solved"
  },
  {
    "startMs": 1028799,
    "text": "it's all good but in production it has"
  },
  {
    "startMs": 1031839,
    "text": "to be scalable. It has to be cost"
  },
  {
    "startMs": 1033600,
    "text": "effective. It cannot just be like"
  },
  {
    "startMs": 1035520,
    "text": "something which is um out of the um"
  },
  {
    "startMs": 1040079,
    "text": "which is just out in the market because"
  },
  {
    "startMs": 1041600,
    "text": "we are solving a very specific problem"
  },
  {
    "startMs": 1043360,
    "text": "out here. Coming back to the evaluation"
  },
  {
    "startMs": 1046799,
    "text": "basically first is the task of success."
  },
  {
    "startMs": 1048880,
    "text": "Of course these are the binary things"
  },
  {
    "startMs": 1050160,
    "text": "the modality happen it does not happen"
  },
  {
    "startMs": 1052000,
    "text": "like there is alignment or disalignment."
  },
  {
    "startMs": 1054480,
    "text": "So precision, recall, F1 are obvious"
  },
  {
    "startMs": 1056799,
    "text": "metrics to understand what the task is"
  },
  {
    "startMs": 1059200,
    "text": "success is. But we want to look at a"
  },
  {
    "startMs": 1061280,
    "text": "system in a very holistic manner. Not"
  },
  {
    "startMs": 1063120,
    "text": "just like the end goal but what is"
  },
  {
    "startMs": 1064720,
    "text": "happening at each nodes. How well like"
  },
  {
    "startMs": 1067120,
    "text": "the retrieval system is working. What is"
  },
  {
    "startMs": 1068799,
    "text": "the latency and recall of the system and"
  },
  {
    "startMs": 1071600,
    "text": "how well we are able to reason it? What"
  },
  {
    "startMs": 1073520,
    "text": "is the chain of thought reasoning coming"
  },
  {
    "startMs": 1074880,
    "text": "from these models at each agentic level"
  },
  {
    "startMs": 1077520,
    "text": "or wherever the its reasonings are"
  },
  {
    "startMs": 1079679,
    "text": "applicable which is basically a reviewer"
  },
  {
    "startMs": 1081919,
    "text": "agent in our case. And what is the"
  },
  {
    "startMs": 1084080,
    "text": "quality of that? Is it like"
  },
  {
    "startMs": 1085360,
    "text": "overthinking? Can we reduce a budget"
  },
  {
    "startMs": 1086960,
    "text": "somewhere like to make sure that like"
  },
  {
    "startMs": 1088559,
    "text": "the models is actually doing a good job"
  },
  {
    "startMs": 1090880,
    "text": "with less uh budget or can we increase"
  },
  {
    "startMs": 1093440,
    "text": "it the budget of the planning or the"
  },
  {
    "startMs": 1095600,
    "text": "reasoning things to have it a higher"
  },
  {
    "startMs": 1097760,
    "text": "budget and make sure that the complex"
  },
  {
    "startMs": 1100320,
    "text": "problem that we are solving is maybe"
  },
  {
    "startMs": 1102640,
    "text": "having a much higher performance and"
  },
  {
    "startMs": 1104160,
    "text": "accuracy. So having an adaptive adaptive"
  },
  {
    "startMs": 1106960,
    "text": "reasoning budgeting is also uh quite"
  },
  {
    "startMs": 1109760,
    "text": "important. And then we have the"
  },
  {
    "startMs": 1111280,
    "text": "robustness. what are the edge cases we"
  },
  {
    "startMs": 1112960,
    "text": "are seeing? Uh what are the error rates"
  },
  {
    "startMs": 1114720,
    "text": "we are having? What are the different"
  },
  {
    "startMs": 1116400,
    "text": "nodes and hooks where they are"
  },
  {
    "startMs": 1118000,
    "text": "happening? Is it like a tool call is not"
  },
  {
    "startMs": 1119520,
    "text": "working really well? It's a retrieval"
  },
  {
    "startMs": 1121280,
    "text": "part or LLM is not doing a good job"
  },
  {
    "startMs": 1123840,
    "text": "where or those kind of thing. And then"
  },
  {
    "startMs": 1126080,
    "text": "we have a system efficiency where we are"
  },
  {
    "startMs": 1127840,
    "text": "not looking uh just at the performance"
  },
  {
    "startMs": 1130400,
    "text": "of the output and everything but we are"
  },
  {
    "startMs": 1132480,
    "text": "looking at each and every aspects of the"
  },
  {
    "startMs": 1134640,
    "text": "token cost which LLM are calling can you"
  },
  {
    "startMs": 1137120,
    "text": "optimize LLM little bit more to actually"
  },
  {
    "startMs": 1138960,
    "text": "save the cost. What is the efficiency"
  },
  {
    "startMs": 1140880,
    "text": "which is we are saying and the latency"
  },
  {
    "startMs": 1142559,
    "text": "of each agents and the entire system"
  },
  {
    "startMs": 1145360,
    "text": "combined together. Then we have a LLM as"
  },
  {
    "startMs": 1147600,
    "text": "a judge where we see that like in any of"
  },
  {
    "startMs": 1150400,
    "text": "the prediction system there is always a"
  },
  {
    "startMs": 1152480,
    "text": "data drift happening especially when you"
  },
  {
    "startMs": 1154160,
    "text": "have the user generated content. So how"
  },
  {
    "startMs": 1157280,
    "text": "our LLM as a judge which is used for"
  },
  {
    "startMs": 1159120,
    "text": "evaluation and everything is doing with"
  },
  {
    "startMs": 1160799,
    "text": "respect to the human queue. Is there a"
  },
  {
    "startMs": 1162559,
    "text": "drift happening? Do we need to retrain"
  },
  {
    "startMs": 1164480,
    "text": "our LLM as a judge on new data set which"
  },
  {
    "startMs": 1166480,
    "text": "is coming from labeling team or how do"
  },
  {
    "startMs": 1168880,
    "text": "we actually do on those parts in"
  },
  {
    "startMs": 1170799,
    "text": "optimization? We have the spatial"
  },
  {
    "startMs": 1172559,
    "text": "temporal reduction optimization which"
  },
  {
    "startMs": 1174320,
    "text": "actually look into the frame which are"
  },
  {
    "startMs": 1175679,
    "text": "similar and just compress them into the"
  },
  {
    "startMs": 1177280,
    "text": "one aspect. This reduces the total"
  },
  {
    "startMs": 1181039,
    "text": "processing of the videos by a huge"
  },
  {
    "startMs": 1183919,
    "text": "extent. The second one is basically you"
  },
  {
    "startMs": 1186480,
    "text": "have a caching you have a viral content"
  },
  {
    "startMs": 1188000,
    "text": "which is basically coming up and you do"
  },
  {
    "startMs": 1189919,
    "text": "not want to have the same content out"
  },
  {
    "startMs": 1192160,
    "text": "there which is doing going through the"
  },
  {
    "startMs": 1194000,
    "text": "entire pipeline and that's where you"
  },
  {
    "startMs": 1195600,
    "text": "have a high similarity score and just"
  },
  {
    "startMs": 1197200,
    "text": "skip the multi- aent system and just"
  },
  {
    "startMs": 1198960,
    "text": "make a call on that. The third one which"
  },
  {
    "startMs": 1201280,
    "text": "is really important is the metadata"
  },
  {
    "startMs": 1202720,
    "text": "pruning. This is where we actually"
  },
  {
    "startMs": 1204640,
    "text": "shrink the space from lot of candidates"
  },
  {
    "startMs": 1206720,
    "text": "based on some of the metadata. For"
  },
  {
    "startMs": 1208240,
    "text": "example, some topics and some creator"
  },
  {
    "startMs": 1210559,
    "text": "which already have a really good record."
  },
  {
    "startMs": 1212320,
    "text": "That means we really don't need to"
  },
  {
    "startMs": 1213760,
    "text": "process these all these images all all"
  },
  {
    "startMs": 1217039,
    "text": "the videos from the creator which has a"
  },
  {
    "startMs": 1218960,
    "text": "really good high authenticity score"
  },
  {
    "startMs": 1220400,
    "text": "which are really doing really well on"
  },
  {
    "startMs": 1221919,
    "text": "this where the video quality is really"
  },
  {
    "startMs": 1223600,
    "text": "high engagement is good those kind of"
  },
  {
    "startMs": 1225840,
    "text": "things. A metadata is something which"
  },
  {
    "startMs": 1227679,
    "text": "can be used to filter out the video"
  },
  {
    "startMs": 1230480,
    "text": "which shouldn't even go to the systems."
  },
  {
    "startMs": 1232960,
    "text": "So some of these flags u would be"
  },
  {
    "startMs": 1235280,
    "text": "helpful. Um so the last takeaway that u"
  },
  {
    "startMs": 1239520,
    "text": "u we basically we can take out of this"
  },
  {
    "startMs": 1242320,
    "text": "room is that like decomposition t is a"
  },
  {
    "startMs": 1244720,
    "text": "key. Uh decompose uh a problem as and"
  },
  {
    "startMs": 1247919,
    "text": "when necessary. Adaptive optimization is"
  },
  {
    "startMs": 1250559,
    "text": "really important for uh ROI and cost"
  },
  {
    "startMs": 1252880,
    "text": "feasibility. Good evaluation is"
  },
  {
    "startMs": 1255039,
    "text": "paramount. Everything in and out depends"
  },
  {
    "startMs": 1257120,
    "text": "on this. This is the foundation of your"
  },
  {
    "startMs": 1258880,
    "text": "entire system and entire VLMs and"
  },
  {
    "startMs": 1261840,
    "text": "production monitoring and qualitative"
  },
  {
    "startMs": 1263840,
    "text": "improvement are really essential to make"
  },
  {
    "startMs": 1265919,
    "text": "sure this is sustainable in long term."
  },
  {
    "startMs": 1268480,
    "text": "With that, I'll end it and thank you"
  },
  {
    "startMs": 1270240,
    "text": "very much for listening. Thank"
  },
  {
    "startMs": 1272679,
    "text": "[applause]"
  }
]