[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12719,
    "text": "I'm Ashwin Paranspe. I am uh the"
  },
  {
    "startMs": 14870,
    "text": "I'm Ashwin Paranspe. I am uh the"
  },
  {
    "startMs": 14880,
    "text": "I'm Ashwin Paranspe. I am uh the founding AI lead at Somaya. Uh and today"
  },
  {
    "startMs": 17670,
    "text": "founding AI lead at Somaya. Uh and today"
  },
  {
    "startMs": 17680,
    "text": "founding AI lead at Somaya. Uh and today I'm going to talk about uh you know if"
  },
  {
    "startMs": 19910,
    "text": "I'm going to talk about uh you know if"
  },
  {
    "startMs": 19920,
    "text": "I'm going to talk about uh you know if agents can rewrite compilers why can't"
  },
  {
    "startMs": 21830,
    "text": "agents can rewrite compilers why can't"
  },
  {
    "startMs": 21840,
    "text": "agents can rewrite compilers why can't they pick stocks? Uh so I give I had"
  },
  {
    "startMs": 24150,
    "text": "they pick stocks? Uh so I give I had"
  },
  {
    "startMs": 24160,
    "text": "they pick stocks? Uh so I give I had this title um couple of months ago but"
  },
  {
    "startMs": 27189,
    "text": "this title um couple of months ago but"
  },
  {
    "startMs": 27199,
    "text": "this title um couple of months ago but AI moves so fast that I now need to"
  },
  {
    "startMs": 29269,
    "text": "AI moves so fast that I now need to"
  },
  {
    "startMs": 29279,
    "text": "AI moves so fast that I now need to change this title to agents can solve"
  },
  {
    "startMs": 31910,
    "text": "change this title to agents can solve"
  },
  {
    "startMs": 31920,
    "text": "change this title to agents can solve Navier Stokes uh and why can't they uh"
  },
  {
    "startMs": 34549,
    "text": "Navier Stokes uh and why can't they uh"
  },
  {
    "startMs": 34559,
    "text": "Navier Stokes uh and why can't they uh pick stocks. So um just a brief"
  },
  {
    "startMs": 37670,
    "text": "pick stocks. So um just a brief"
  },
  {
    "startMs": 37680,
    "text": "pick stocks. So um just a brief background about me uh so I did my PhD"
  },
  {
    "startMs": 40310,
    "text": "background about me uh so I did my PhD"
  },
  {
    "startMs": 40320,
    "text": "background about me uh so I did my PhD from Stanford um one of the highlights"
  },
  {
    "startMs": 43190,
    "text": "from Stanford um one of the highlights"
  },
  {
    "startMs": 43200,
    "text": "from Stanford um one of the highlights was you know fine-tuning GPT2 in 2019"
  },
  {
    "startMs": 46069,
    "text": "was you know fine-tuning GPT2 in 2019"
  },
  {
    "startMs": 46079,
    "text": "was you know fine-tuning GPT2 in 2019 and putting it in production with real"
  },
  {
    "startMs": 48150,
    "text": "and putting it in production with real"
  },
  {
    "startMs": 48160,
    "text": "and putting it in production with real users with tens of thousands of"
  },
  {
    "startMs": 49670,
    "text": "users with tens of thousands of"
  },
  {
    "startMs": 49680,
    "text": "users with tens of thousands of interactions every day. uh when I joined"
  },
  {
    "startMs": 52229,
    "text": "interactions every day. uh when I joined"
  },
  {
    "startMs": 52239,
    "text": "interactions every day. uh when I joined Somaya I was focused on training our"
  },
  {
    "startMs": 54389,
    "text": "Somaya I was focused on training our"
  },
  {
    "startMs": 54399,
    "text": "Somaya I was focused on training our retrieval stack uh make working with"
  },
  {
    "startMs": 56470,
    "text": "retrieval stack uh make working with"
  },
  {
    "startMs": 56480,
    "text": "retrieval stack uh make working with long context models in fact uh you might"
  },
  {
    "startMs": 58310,
    "text": "long context models in fact uh you might"
  },
  {
    "startMs": 58320,
    "text": "long context models in fact uh you might have seen a paper called lost in the"
  },
  {
    "startMs": 59990,
    "text": "have seen a paper called lost in the"
  },
  {
    "startMs": 60000,
    "text": "have seen a paper called lost in the middle that was led by my intern Nelson"
  },
  {
    "startMs": 62310,
    "text": "middle that was led by my intern Nelson"
  },
  {
    "startMs": 62320,
    "text": "middle that was led by my intern Nelson when he was interning with us um but"
  },
  {
    "startMs": 64630,
    "text": "when he was interning with us um but"
  },
  {
    "startMs": 64640,
    "text": "when he was interning with us um but yeah so this allowed us to have high"
  },
  {
    "startMs": 66149,
    "text": "yeah so this allowed us to have high"
  },
  {
    "startMs": 66159,
    "text": "yeah so this allowed us to have high quality retrieval over messy customer"
  },
  {
    "startMs": 67990,
    "text": "quality retrieval over messy customer"
  },
  {
    "startMs": 68000,
    "text": "quality retrieval over messy customer data last year uh I kind of changed my"
  },
  {
    "startMs": 71190,
    "text": "data last year uh I kind of changed my"
  },
  {
    "startMs": 71200,
    "text": "data last year uh I kind of changed my focus into designing our financial agent"
  },
  {
    "startMs": 73750,
    "text": "focus into designing our financial agent"
  },
  {
    "startMs": 73760,
    "text": "focus into designing our financial agent harness so this allows contact context"
  },
  {
    "startMs": 75590,
    "text": "harness so this allows contact context"
  },
  {
    "startMs": 75600,
    "text": "harness so this allows contact context efficient unified access to data sources"
  },
  {
    "startMs": 78310,
    "text": "efficient unified access to data sources"
  },
  {
    "startMs": 78320,
    "text": "efficient unified access to data sources and it's rolled out to like 12,000 users"
  },
  {
    "startMs": 80870,
    "text": "and it's rolled out to like 12,000 users"
  },
  {
    "startMs": 80880,
    "text": "and it's rolled out to like 12,000 users users across Morgan Stanley and hundreds"
  },
  {
    "startMs": 82870,
    "text": "users across Morgan Stanley and hundreds"
  },
  {
    "startMs": 82880,
    "text": "users across Morgan Stanley and hundreds of other users in hedge funds and asset"
  },
  {
    "startMs": 84789,
    "text": "of other users in hedge funds and asset"
  },
  {
    "startMs": 84799,
    "text": "of other users in hedge funds and asset management firms. Uh finally this year"
  },
  {
    "startMs": 88149,
    "text": "management firms. Uh finally this year"
  },
  {
    "startMs": 88159,
    "text": "management firms. Uh finally this year we released a benchmark which is the"
  },
  {
    "startMs": 89990,
    "text": "we released a benchmark which is the"
  },
  {
    "startMs": 90000,
    "text": "we released a benchmark which is the largest and the hardest open-source"
  },
  {
    "startMs": 91990,
    "text": "largest and the hardest open-source"
  },
  {
    "startMs": 92000,
    "text": "largest and the hardest open-source benchmark for you know deep research"
  },
  {
    "startMs": 94149,
    "text": "benchmark for you know deep research"
  },
  {
    "startMs": 94159,
    "text": "benchmark for you know deep research financial workflows. Um I led the design"
  },
  {
    "startMs": 97350,
    "text": "financial workflows. Um I led the design"
  },
  {
    "startMs": 97360,
    "text": "financial workflows. Um I led the design of this benchmark a couple of years ago"
  },
  {
    "startMs": 99270,
    "text": "of this benchmark a couple of years ago"
  },
  {
    "startMs": 99280,
    "text": "of this benchmark a couple of years ago when we started collecting it. Please"
  },
  {
    "startMs": 101030,
    "text": "when we started collecting it. Please"
  },
  {
    "startMs": 101040,
    "text": "when we started collecting it. Please check it out if that's of interest to"
  },
  {
    "startMs": 103190,
    "text": "check it out if that's of interest to"
  },
  {
    "startMs": 103200,
    "text": "check it out if that's of interest to you. Um so yeah so I was thinking well"
  },
  {
    "startMs": 105749,
    "text": "you. Um so yeah so I was thinking well"
  },
  {
    "startMs": 105759,
    "text": "you. Um so yeah so I was thinking well you know worked on training the"
  },
  {
    "startMs": 107190,
    "text": "you know worked on training the"
  },
  {
    "startMs": 107200,
    "text": "you know worked on training the retrieval stack the harness eval so"
  },
  {
    "startMs": 109350,
    "text": "retrieval stack the harness eval so"
  },
  {
    "startMs": 109360,
    "text": "retrieval stack the harness eval so what's next right um so in math you have"
  },
  {
    "startMs": 113510,
    "text": "what's next right um so in math you have"
  },
  {
    "startMs": 113520,
    "text": "what's next right um so in math you have like this marquee problem like the"
  },
  {
    "startMs": 115429,
    "text": "like this marquee problem like the"
  },
  {
    "startMs": 115439,
    "text": "like this marquee problem like the millennium price problem can you get a"
  },
  {
    "startMs": 117510,
    "text": "millennium price problem can you get a"
  },
  {
    "startMs": 117520,
    "text": "millennium price problem can you get a solution to navia stokes there are"
  },
  {
    "startMs": 119429,
    "text": "solution to navia stokes there are"
  },
  {
    "startMs": 119439,
    "text": "solution to navia stokes there are hundreds of open math problems like that"
  },
  {
    "startMs": 121990,
    "text": "hundreds of open math problems like that"
  },
  {
    "startMs": 122000,
    "text": "hundreds of open math problems like that so what could something like this look"
  },
  {
    "startMs": 123749,
    "text": "so what could something like this look"
  },
  {
    "startMs": 123759,
    "text": "so what could something like this look for finance right um so like the kind of"
  },
  {
    "startMs": 128790,
    "text": "for finance right um so like the kind of"
  },
  {
    "startMs": 128800,
    "text": "for finance right um so like the kind of zoomed out answer is can we forecast the"
  },
  {
    "startMs": 131190,
    "text": "zoomed out answer is can we forecast the"
  },
  {
    "startMs": 131200,
    "text": "zoomed out answer is can we forecast the future can we forecast it better than"
  },
  {
    "startMs": 133430,
    "text": "future can we forecast it better than"
  },
  {
    "startMs": 133440,
    "text": "future can we forecast it better than what we do today and that's really like"
  },
  {
    "startMs": 135350,
    "text": "what we do today and that's really like"
  },
  {
    "startMs": 135360,
    "text": "what we do today and that's really like the the highest level challenge we can"
  },
  {
    "startMs": 137750,
    "text": "the the highest level challenge we can"
  },
  {
    "startMs": 137760,
    "text": "the the highest level challenge we can focus on here. So you know in my naivity"
  },
  {
    "startMs": 140869,
    "text": "focus on here. So you know in my naivity"
  },
  {
    "startMs": 140879,
    "text": "focus on here. So you know in my naivity I went around asking people well what if"
  },
  {
    "startMs": 143910,
    "text": "I went around asking people well what if"
  },
  {
    "startMs": 143920,
    "text": "I went around asking people well what if you try to predict stocks. Um so just a"
  },
  {
    "startMs": 146869,
    "text": "you try to predict stocks. Um so just a"
  },
  {
    "startMs": 146879,
    "text": "you try to predict stocks. Um so just a show of hands here like how many people"
  },
  {
    "startMs": 148949,
    "text": "show of hands here like how many people"
  },
  {
    "startMs": 148959,
    "text": "show of hands here like how many people try to predict stocks or are adjacent to"
  },
  {
    "startMs": 151110,
    "text": "try to predict stocks or are adjacent to"
  },
  {
    "startMs": 151120,
    "text": "try to predict stocks or are adjacent to it in this audience?"
  },
  {
    "startMs": 154070,
    "text": "it in this audience?"
  },
  {
    "startMs": 154080,
    "text": "it in this audience? Wow that's like over over half of us. Um"
  },
  {
    "startMs": 157589,
    "text": "Wow that's like over over half of us. Um"
  },
  {
    "startMs": 157599,
    "text": "Wow that's like over over half of us. Um and so um so I was like okay well you"
  },
  {
    "startMs": 159750,
    "text": "and so um so I was like okay well you"
  },
  {
    "startMs": 159760,
    "text": "and so um so I was like okay well you know how hard could it be like"
  },
  {
    "startMs": 161270,
    "text": "know how hard could it be like"
  },
  {
    "startMs": 161280,
    "text": "know how hard could it be like everybody's trying to do this. So uh I"
  },
  {
    "startMs": 164470,
    "text": "everybody's trying to do this. So uh I"
  },
  {
    "startMs": 164480,
    "text": "everybody's trying to do this. So uh I went to Arash who's a colleague here at"
  },
  {
    "startMs": 166790,
    "text": "went to Arash who's a colleague here at"
  },
  {
    "startMs": 166800,
    "text": "went to Arash who's a colleague here at Somaya. He was previously at Bridgewater"
  },
  {
    "startMs": 169350,
    "text": "Somaya. He was previously at Bridgewater"
  },
  {
    "startMs": 169360,
    "text": "Somaya. He was previously at Bridgewater and Varition and he says like you know"
  },
  {
    "startMs": 171030,
    "text": "and Varition and he says like you know"
  },
  {
    "startMs": 171040,
    "text": "and Varition and he says like you know you just don't simply start with stock"
  },
  {
    "startMs": 172710,
    "text": "you just don't simply start with stock"
  },
  {
    "startMs": 172720,
    "text": "you just don't simply start with stock prediction. It's very noisy. It um"
  },
  {
    "startMs": 175750,
    "text": "prediction. It's very noisy. It um"
  },
  {
    "startMs": 175760,
    "text": "prediction. It's very noisy. It um doesn't tell you why a certain model is"
  },
  {
    "startMs": 178309,
    "text": "doesn't tell you why a certain model is"
  },
  {
    "startMs": 178319,
    "text": "doesn't tell you why a certain model is behaving in certain ways. It doesn't"
  },
  {
    "startMs": 179910,
    "text": "behaving in certain ways. It doesn't"
  },
  {
    "startMs": 179920,
    "text": "behaving in certain ways. It doesn't tell you if you know the reasoning is"
  },
  {
    "startMs": 181270,
    "text": "tell you if you know the reasoning is"
  },
  {
    "startMs": 181280,
    "text": "tell you if you know the reasoning is right or not. Nobody really knows why"
  },
  {
    "startMs": 183350,
    "text": "right or not. Nobody really knows why"
  },
  {
    "startMs": 183360,
    "text": "right or not. Nobody really knows why stock moves a lot of times. So then u by"
  },
  {
    "startMs": 187270,
    "text": "stock moves a lot of times. So then u by"
  },
  {
    "startMs": 187280,
    "text": "stock moves a lot of times. So then u by the way he's sitting here uh close"
  },
  {
    "startMs": 188869,
    "text": "the way he's sitting here uh close"
  },
  {
    "startMs": 188879,
    "text": "the way he's sitting here uh close collaborator"
  },
  {
    "startMs": 190597,
    "text": "collaborator"
  },
  {
    "startMs": 190607,
    "text": "collaborator [laughter]"
  },
  {
    "startMs": 192229,
    "text": "[laughter]"
  },
  {
    "startMs": 192239,
    "text": "[laughter] come with us afterwards. So um yeah so"
  },
  {
    "startMs": 196149,
    "text": "come with us afterwards. So um yeah so"
  },
  {
    "startMs": 196159,
    "text": "come with us afterwards. So um yeah so he suggested let's instead do earnings"
  },
  {
    "startMs": 198229,
    "text": "he suggested let's instead do earnings"
  },
  {
    "startMs": 198239,
    "text": "he suggested let's instead do earnings prediction. So what is earnings right?"
  },
  {
    "startMs": 200390,
    "text": "prediction. So what is earnings right?"
  },
  {
    "startMs": 200400,
    "text": "prediction. So what is earnings right? So every quarter a company uh comes out"
  },
  {
    "startMs": 203350,
    "text": "So every quarter a company uh comes out"
  },
  {
    "startMs": 203360,
    "text": "So every quarter a company uh comes out and reports its earnings. It gives"
  },
  {
    "startMs": 205270,
    "text": "and reports its earnings. It gives"
  },
  {
    "startMs": 205280,
    "text": "and reports its earnings. It gives revenue, gross margin, operating income,"
  },
  {
    "startMs": 208790,
    "text": "revenue, gross margin, operating income,"
  },
  {
    "startMs": 208800,
    "text": "revenue, gross margin, operating income, uh EPS, a bunch of other metrics and"
  },
  {
    "startMs": 211270,
    "text": "uh EPS, a bunch of other metrics and"
  },
  {
    "startMs": 211280,
    "text": "uh EPS, a bunch of other metrics and then it also gives guidance for the next"
  },
  {
    "startMs": 212789,
    "text": "then it also gives guidance for the next"
  },
  {
    "startMs": 212799,
    "text": "then it also gives guidance for the next quarter. Um analysts uh across buy side"
  },
  {
    "startMs": 216149,
    "text": "quarter. Um analysts uh across buy side"
  },
  {
    "startMs": 216159,
    "text": "quarter. Um analysts uh across buy side and sell side try to predict these"
  },
  {
    "startMs": 217750,
    "text": "and sell side try to predict these"
  },
  {
    "startMs": 217760,
    "text": "and sell side try to predict these numbers in the coming quarter. Uh a lot"
  },
  {
    "startMs": 220070,
    "text": "numbers in the coming quarter. Uh a lot"
  },
  {
    "startMs": 220080,
    "text": "numbers in the coming quarter. Uh a lot of the sellside analysts publish this"
  },
  {
    "startMs": 222149,
    "text": "of the sellside analysts publish this"
  },
  {
    "startMs": 222159,
    "text": "of the sellside analysts publish this right. So it gets pulled as what is"
  },
  {
    "startMs": 224149,
    "text": "right. So it gets pulled as what is"
  },
  {
    "startMs": 224159,
    "text": "right. So it gets pulled as what is called a consensus estimate. And then"
  },
  {
    "startMs": 226229,
    "text": "called a consensus estimate. And then"
  },
  {
    "startMs": 226239,
    "text": "called a consensus estimate. And then the next quarter the company reports."
  },
  {
    "startMs": 228309,
    "text": "the next quarter the company reports."
  },
  {
    "startMs": 228319,
    "text": "the next quarter the company reports. Now if it's above the consensus it's a"
  },
  {
    "startMs": 230390,
    "text": "Now if it's above the consensus it's a"
  },
  {
    "startMs": 230400,
    "text": "Now if it's above the consensus it's a beat and it's a surprise. If it's below"
  },
  {
    "startMs": 232550,
    "text": "beat and it's a surprise. If it's below"
  },
  {
    "startMs": 232560,
    "text": "beat and it's a surprise. If it's below the consensus it's a miss. Uh and then"
  },
  {
    "startMs": 235030,
    "text": "the consensus it's a miss. Uh and then"
  },
  {
    "startMs": 235040,
    "text": "the consensus it's a miss. Uh and then during the call and after the call in"
  },
  {
    "startMs": 236789,
    "text": "during the call and after the call in"
  },
  {
    "startMs": 236799,
    "text": "during the call and after the call in the commentary they also talk about why"
  },
  {
    "startMs": 238710,
    "text": "the commentary they also talk about why"
  },
  {
    "startMs": 238720,
    "text": "the commentary they also talk about why these uh like you know surprises"
  },
  {
    "startMs": 240869,
    "text": "these uh like you know surprises"
  },
  {
    "startMs": 240879,
    "text": "these uh like you know surprises happened. So this is from a from a AI"
  },
  {
    "startMs": 243670,
    "text": "happened. So this is from a from a AI"
  },
  {
    "startMs": 243680,
    "text": "happened. So this is from a from a AI and ML perspective this is a great task."
  },
  {
    "startMs": 246390,
    "text": "and ML perspective this is a great task."
  },
  {
    "startMs": 246400,
    "text": "and ML perspective this is a great task. It has qual quantitative ground truth."
  },
  {
    "startMs": 249030,
    "text": "It has qual quantitative ground truth."
  },
  {
    "startMs": 249040,
    "text": "It has qual quantitative ground truth. You know what exactly happened"
  },
  {
    "startMs": 250550,
    "text": "You know what exactly happened"
  },
  {
    "startMs": 250560,
    "text": "You know what exactly happened afterwards. you have an expert human"
  },
  {
    "startMs": 252789,
    "text": "afterwards. you have an expert human"
  },
  {
    "startMs": 252799,
    "text": "afterwards. you have an expert human baseline to compare against and you also"
  },
  {
    "startMs": 255429,
    "text": "baseline to compare against and you also"
  },
  {
    "startMs": 255439,
    "text": "baseline to compare against and you also have the qualitative ground truth that"
  },
  {
    "startMs": 257270,
    "text": "have the qualitative ground truth that"
  },
  {
    "startMs": 257280,
    "text": "have the qualitative ground truth that tells you you know why the agent was"
  },
  {
    "startMs": 260229,
    "text": "tells you you know why the agent was"
  },
  {
    "startMs": 260239,
    "text": "tells you you know why the agent was right if it was right and so the then"
  },
  {
    "startMs": 263510,
    "text": "right if it was right and so the then"
  },
  {
    "startMs": 263520,
    "text": "right if it was right and so the then the question is well can an agent do"
  },
  {
    "startMs": 265909,
    "text": "the question is well can an agent do"
  },
  {
    "startMs": 265919,
    "text": "the question is well can an agent do this task can they make good predictions"
  },
  {
    "startMs": 268070,
    "text": "this task can they make good predictions"
  },
  {
    "startMs": 268080,
    "text": "this task can they make good predictions and this is this is going to be the"
  },
  {
    "startMs": 269670,
    "text": "and this is this is going to be the"
  },
  {
    "startMs": 269680,
    "text": "and this is this is going to be the focus of today's talk"
  },
  {
    "startMs": 272629,
    "text": "focus of today's talk"
  },
  {
    "startMs": 272639,
    "text": "focus of today's talk so to be able to do this we had to work"
  },
  {
    "startMs": 275350,
    "text": "so to be able to do this we had to work"
  },
  {
    "startMs": 275360,
    "text": "so to be able to do this we had to work on building an environment and a harness"
  },
  {
    "startMs": 277670,
    "text": "on building an environment and a harness"
  },
  {
    "startMs": 277680,
    "text": "on building an environment and a harness so just to give you background on"
  },
  {
    "startMs": 279350,
    "text": "so just to give you background on"
  },
  {
    "startMs": 279360,
    "text": "so just to give you background on environments whenever you have these"
  },
  {
    "startMs": 280950,
    "text": "environments whenever you have these"
  },
  {
    "startMs": 280960,
    "text": "environments whenever you have these agent intake models they need to be in"
  },
  {
    "startMs": 283030,
    "text": "agent intake models they need to be in"
  },
  {
    "startMs": 283040,
    "text": "agent intake models they need to be in an environment where you know they are"
  },
  {
    "startMs": 284390,
    "text": "an environment where you know they are"
  },
  {
    "startMs": 284400,
    "text": "an environment where you know they are given the right data they are given a"
  },
  {
    "startMs": 285909,
    "text": "given the right data they are given a"
  },
  {
    "startMs": 285919,
    "text": "given the right data they are given a task and then you can evaluate it at the"
  },
  {
    "startMs": 287830,
    "text": "task and then you can evaluate it at the"
  },
  {
    "startMs": 287840,
    "text": "task and then you can evaluate it at the end and you might have heard it in the"
  },
  {
    "startMs": 289670,
    "text": "end and you might have heard it in the"
  },
  {
    "startMs": 289680,
    "text": "end and you might have heard it in the context of training so RL training but"
  },
  {
    "startMs": 292150,
    "text": "context of training so RL training but"
  },
  {
    "startMs": 292160,
    "text": "context of training so RL training but you need the same environment for"
  },
  {
    "startMs": 293430,
    "text": "you need the same environment for"
  },
  {
    "startMs": 293440,
    "text": "you need the same environment for evaluation as well and then uh that"
  },
  {
    "startMs": 296310,
    "text": "evaluation as well and then uh that"
  },
  {
    "startMs": 296320,
    "text": "evaluation as well and then uh that environment needs to provide a harness"
  },
  {
    "startMs": 298150,
    "text": "environment needs to provide a harness"
  },
  {
    "startMs": 298160,
    "text": "environment needs to provide a harness to the agent so here's how the"
  },
  {
    "startMs": 300950,
    "text": "to the agent so here's how the"
  },
  {
    "startMs": 300960,
    "text": "to the agent so here's how the environment schematically looks like so"
  },
  {
    "startMs": 302950,
    "text": "environment schematically looks like so"
  },
  {
    "startMs": 302960,
    "text": "environment schematically looks like so you have an AI model that's sitting"
  },
  {
    "startMs": 305189,
    "text": "you have an AI model that's sitting"
  },
  {
    "startMs": 305199,
    "text": "you have an AI model that's sitting inside a a harness so we provided the"
  },
  {
    "startMs": 308070,
    "text": "inside a a harness so we provided the"
  },
  {
    "startMs": 308080,
    "text": "inside a a harness so we provided the harness this harness we constructed such"
  },
  {
    "startMs": 310790,
    "text": "harness this harness we constructed such"
  },
  {
    "startMs": 310800,
    "text": "harness this harness we constructed such that it brings point in time"
  },
  {
    "startMs": 312790,
    "text": "that it brings point in time"
  },
  {
    "startMs": 312800,
    "text": "that it brings point in time information. Um so if we are trying to"
  },
  {
    "startMs": 315430,
    "text": "information. Um so if we are trying to"
  },
  {
    "startMs": 315440,
    "text": "information. Um so if we are trying to predict say 7 days or five sessions"
  },
  {
    "startMs": 318230,
    "text": "predict say 7 days or five sessions"
  },
  {
    "startMs": 318240,
    "text": "predict say 7 days or five sessions before earnings the context only"
  },
  {
    "startMs": 320390,
    "text": "before earnings the context only"
  },
  {
    "startMs": 320400,
    "text": "before earnings the context only includes information up to that point up"
  },
  {
    "startMs": 322469,
    "text": "includes information up to that point up"
  },
  {
    "startMs": 322479,
    "text": "includes information up to that point up to this uh minute and okay so with this"
  },
  {
    "startMs": 325749,
    "text": "to this uh minute and okay so with this"
  },
  {
    "startMs": 325759,
    "text": "to this uh minute and okay so with this harness the AI agent researches and"
  },
  {
    "startMs": 327430,
    "text": "harness the AI agent researches and"
  },
  {
    "startMs": 327440,
    "text": "harness the AI agent researches and makes a prediction and come the earnings"
  },
  {
    "startMs": 329270,
    "text": "makes a prediction and come the earnings"
  },
  {
    "startMs": 329280,
    "text": "makes a prediction and come the earnings day we compare the differences and then"
  },
  {
    "startMs": 331110,
    "text": "day we compare the differences and then"
  },
  {
    "startMs": 331120,
    "text": "day we compare the differences and then see how good its prediction was."
  },
  {
    "startMs": 334070,
    "text": "see how good its prediction was."
  },
  {
    "startMs": 334080,
    "text": "see how good its prediction was. Now to build this harness what we did"
  },
  {
    "startMs": 335909,
    "text": "Now to build this harness what we did"
  },
  {
    "startMs": 335919,
    "text": "Now to build this harness what we did was we started from Somaya's production"
  },
  {
    "startMs": 337990,
    "text": "was we started from Somaya's production"
  },
  {
    "startMs": 338000,
    "text": "was we started from Somaya's production grade harness. So this includes all the"
  },
  {
    "startMs": 339990,
    "text": "grade harness. So this includes all the"
  },
  {
    "startMs": 340000,
    "text": "grade harness. So this includes all the real world data that we provide through"
  },
  {
    "startMs": 341749,
    "text": "real world data that we provide through"
  },
  {
    "startMs": 341759,
    "text": "real world data that we provide through our system. Uh it includes uh company"
  },
  {
    "startMs": 344310,
    "text": "our system. Uh it includes uh company"
  },
  {
    "startMs": 344320,
    "text": "our system. Uh it includes uh company documents, prices, uh consensus also"
  },
  {
    "startMs": 347749,
    "text": "documents, prices, uh consensus also"
  },
  {
    "startMs": 347759,
    "text": "documents, prices, uh consensus also news. Then we have an uh point in time"
  },
  {
    "startMs": 351430,
    "text": "news. Then we have an uh point in time"
  },
  {
    "startMs": 351440,
    "text": "news. Then we have an uh point in time gate. So it's a programmatic gate that"
  },
  {
    "startMs": 353510,
    "text": "gate. So it's a programmatic gate that"
  },
  {
    "startMs": 353520,
    "text": "gate. So it's a programmatic gate that enforces no access to future information"
  },
  {
    "startMs": 356629,
    "text": "enforces no access to future information"
  },
  {
    "startMs": 356639,
    "text": "enforces no access to future information and it's enforced with an authentication"
  },
  {
    "startMs": 358469,
    "text": "and it's enforced with an authentication"
  },
  {
    "startMs": 358479,
    "text": "and it's enforced with an authentication token. So there is no way for the agent"
  },
  {
    "startMs": 360150,
    "text": "token. So there is no way for the agent"
  },
  {
    "startMs": 360160,
    "text": "token. So there is no way for the agent to hack around this. uh we modify all of"
  },
  {
    "startMs": 363749,
    "text": "to hack around this. uh we modify all of"
  },
  {
    "startMs": 363759,
    "text": "to hack around this. uh we modify all of our document retrieval and structured"
  },
  {
    "startMs": 365510,
    "text": "our document retrieval and structured"
  },
  {
    "startMs": 365520,
    "text": "our document retrieval and structured data tools to support this point in time"
  },
  {
    "startMs": 367590,
    "text": "data tools to support this point in time"
  },
  {
    "startMs": 367600,
    "text": "data tools to support this point in time gate and web is one of the trickier ones"
  },
  {
    "startMs": 370629,
    "text": "gate and web is one of the trickier ones"
  },
  {
    "startMs": 370639,
    "text": "gate and web is one of the trickier ones where it's very hard to have this gate."
  },
  {
    "startMs": 372790,
    "text": "where it's very hard to have this gate."
  },
  {
    "startMs": 372800,
    "text": "where it's very hard to have this gate. So instead we get a reasonable proxy by"
  },
  {
    "startMs": 376230,
    "text": "So instead we get a reasonable proxy by"
  },
  {
    "startMs": 376240,
    "text": "So instead we get a reasonable proxy by adding more new streams and making sure"
  },
  {
    "startMs": 378070,
    "text": "adding more new streams and making sure"
  },
  {
    "startMs": 378080,
    "text": "adding more new streams and making sure that they are timestamped. So uh finally"
  },
  {
    "startMs": 380950,
    "text": "that they are timestamped. So uh finally"
  },
  {
    "startMs": 380960,
    "text": "that they are timestamped. So uh finally we add some expert guidance that helps"
  },
  {
    "startMs": 383430,
    "text": "we add some expert guidance that helps"
  },
  {
    "startMs": 383440,
    "text": "we add some expert guidance that helps with um making better use of the context"
  },
  {
    "startMs": 386629,
    "text": "with um making better use of the context"
  },
  {
    "startMs": 386639,
    "text": "with um making better use of the context and also helps with the predictive task."
  },
  {
    "startMs": 391189,
    "text": "and also helps with the predictive task."
  },
  {
    "startMs": 391199,
    "text": "and also helps with the predictive task. So this is the headline. So we found"
  },
  {
    "startMs": 393270,
    "text": "So this is the headline. So we found"
  },
  {
    "startMs": 393280,
    "text": "So this is the headline. So we found that AI is able starting to outperform"
  },
  {
    "startMs": 395590,
    "text": "that AI is able starting to outperform"
  },
  {
    "startMs": 395600,
    "text": "that AI is able starting to outperform consensus in predicting earning"
  },
  {
    "startMs": 397510,
    "text": "consensus in predicting earning"
  },
  {
    "startMs": 397520,
    "text": "consensus in predicting earning surprises. Uh here we looked at more"
  },
  {
    "startMs": 400390,
    "text": "surprises. Uh here we looked at more"
  },
  {
    "startMs": 400400,
    "text": "surprises. Uh here we looked at more than 450 earning sprints. This had to be"
  },
  {
    "startMs": 403510,
    "text": "than 450 earning sprints. This had to be"
  },
  {
    "startMs": 403520,
    "text": "than 450 earning sprints. This had to be from July onwards because the latest"
  },
  {
    "startMs": 406070,
    "text": "from July onwards because the latest"
  },
  {
    "startMs": 406080,
    "text": "from July onwards because the latest model that we were testing uh was Fable"
  },
  {
    "startMs": 408550,
    "text": "model that we were testing uh was Fable"
  },
  {
    "startMs": 408560,
    "text": "model that we were testing uh was Fable 5.1 which had a knowledge cut off of"
  },
  {
    "startMs": 410390,
    "text": "5.1 which had a knowledge cut off of"
  },
  {
    "startMs": 410400,
    "text": "5.1 which had a knowledge cut off of June. Um so we looked at seven frontier"
  },
  {
    "startMs": 413270,
    "text": "June. Um so we looked at seven frontier"
  },
  {
    "startMs": 413280,
    "text": "June. Um so we looked at seven frontier models to compare and here are some of"
  },
  {
    "startMs": 415670,
    "text": "models to compare and here are some of"
  },
  {
    "startMs": 415680,
    "text": "models to compare and here are some of the results. Um so the first thing we"
  },
  {
    "startMs": 418710,
    "text": "the results. Um so the first thing we"
  },
  {
    "startMs": 418720,
    "text": "the results. Um so the first thing we looked at was prediction error. So this"
  },
  {
    "startMs": 420629,
    "text": "looked at was prediction error. So this"
  },
  {
    "startMs": 420639,
    "text": "looked at was prediction error. So this is um when your agent is making a"
  },
  {
    "startMs": 423430,
    "text": "is um when your agent is making a"
  },
  {
    "startMs": 423440,
    "text": "is um when your agent is making a prediction how far is it from the actual"
  },
  {
    "startMs": 425830,
    "text": "prediction how far is it from the actual"
  },
  {
    "startMs": 425840,
    "text": "prediction how far is it from the actual normalized by the uh company's"
  },
  {
    "startMs": 428230,
    "text": "normalized by the uh company's"
  },
  {
    "startMs": 428240,
    "text": "normalized by the uh company's historical volatility. Uh as a"
  },
  {
    "startMs": 431270,
    "text": "historical volatility. Uh as a"
  },
  {
    "startMs": 431280,
    "text": "historical volatility. Uh as a comparison point we looked at consensus."
  },
  {
    "startMs": 433990,
    "text": "comparison point we looked at consensus."
  },
  {
    "startMs": 434000,
    "text": "comparison point we looked at consensus. It had an error of 1.5"
  },
  {
    "startMs": 436710,
    "text": "It had an error of 1.5"
  },
  {
    "startMs": 436720,
    "text": "It had an error of 1.5 uh times historical volatility. But we"
  },
  {
    "startMs": 440150,
    "text": "uh times historical volatility. But we"
  },
  {
    "startMs": 440160,
    "text": "uh times historical volatility. But we also know that a lot of uh consensus is"
  },
  {
    "startMs": 444150,
    "text": "also know that a lot of uh consensus is"
  },
  {
    "startMs": 444160,
    "text": "also know that a lot of uh consensus is under reportported and uh at earnings"
  },
  {
    "startMs": 447510,
    "text": "under reportported and uh at earnings"
  },
  {
    "startMs": 447520,
    "text": "under reportported and uh at earnings you know the companies outperform more"
  },
  {
    "startMs": 449189,
    "text": "you know the companies outperform more"
  },
  {
    "startMs": 449199,
    "text": "you know the companies outperform more often than not. So we corrected for that"
  },
  {
    "startMs": 451110,
    "text": "often than not. So we corrected for that"
  },
  {
    "startMs": 451120,
    "text": "often than not. So we corrected for that bias and uh we have a much better bias"
  },
  {
    "startMs": 454870,
    "text": "bias and uh we have a much better bias"
  },
  {
    "startMs": 454880,
    "text": "bias and uh we have a much better bias corrected error which is 1.18"
  },
  {
    "startMs": 457990,
    "text": "corrected error which is 1.18"
  },
  {
    "startMs": 458000,
    "text": "corrected error which is 1.18 and here's how the progression of models"
  },
  {
    "startMs": 460629,
    "text": "and here's how the progression of models"
  },
  {
    "startMs": 460639,
    "text": "and here's how the progression of models looks like. So you start from cla set"
  },
  {
    "startMs": 463430,
    "text": "looks like. So you start from cla set"
  },
  {
    "startMs": 463440,
    "text": "looks like. So you start from cla set which is you know Kim K3 some of the"
  },
  {
    "startMs": 465110,
    "text": "which is you know Kim K3 some of the"
  },
  {
    "startMs": 465120,
    "text": "which is you know Kim K3 some of the weaker models but then as you go to the"
  },
  {
    "startMs": 467110,
    "text": "weaker models but then as you go to the"
  },
  {
    "startMs": 467120,
    "text": "weaker models but then as you go to the latest models opus 5.5 fable 5.1 GP6"
  },
  {
    "startMs": 471029,
    "text": "latest models opus 5.5 fable 5.1 GP6"
  },
  {
    "startMs": 471039,
    "text": "latest models opus 5.5 fable 5.1 GP6 Astra you start seeing the error go down"
  },
  {
    "startMs": 473909,
    "text": "Astra you start seeing the error go down"
  },
  {
    "startMs": 473919,
    "text": "Astra you start seeing the error go down uh and in fact this is the graph with"
  },
  {
    "startMs": 476309,
    "text": "uh and in fact this is the graph with"
  },
  {
    "startMs": 476319,
    "text": "uh and in fact this is the graph with all the metrics together but if you look"
  },
  {
    "startMs": 477909,
    "text": "all the metrics together but if you look"
  },
  {
    "startMs": 477919,
    "text": "all the metrics together but if you look at revenue which has the least amount of"
  },
  {
    "startMs": 479909,
    "text": "at revenue which has the least amount of"
  },
  {
    "startMs": 479919,
    "text": "at revenue which has the least amount of adjustments you actually start seeing uh"
  },
  {
    "startMs": 482710,
    "text": "adjustments you actually start seeing uh"
  },
  {
    "startMs": 482720,
    "text": "adjustments you actually start seeing uh GBD6 Ara is significantly better than"
  },
  {
    "startMs": 485110,
    "text": "GBD6 Ara is significantly better than"
  },
  {
    "startMs": 485120,
    "text": "GBD6 Ara is significantly better than even the bias corrected consensus."
  },
  {
    "startMs": 488070,
    "text": "even the bias corrected consensus."
  },
  {
    "startMs": 488080,
    "text": "even the bias corrected consensus. We also looked at hit rate. So uh in"
  },
  {
    "startMs": 491029,
    "text": "We also looked at hit rate. So uh in"
  },
  {
    "startMs": 491039,
    "text": "We also looked at hit rate. So uh in simple terms, let's say if the consensus"
  },
  {
    "startMs": 492710,
    "text": "simple terms, let's say if the consensus"
  },
  {
    "startMs": 492720,
    "text": "simple terms, let's say if the consensus is $114,"
  },
  {
    "startMs": 495029,
    "text": "is $114,"
  },
  {
    "startMs": 495039,
    "text": "is $114, the print is $19. So did the model call"
  },
  {
    "startMs": 498230,
    "text": "the print is $19. So did the model call"
  },
  {
    "startMs": 498240,
    "text": "the print is $19. So did the model call it above consensus? If so, that's a hit."
  },
  {
    "startMs": 501189,
    "text": "it above consensus? If so, that's a hit."
  },
  {
    "startMs": 501199,
    "text": "it above consensus? If so, that's a hit. Otherwise, it's a miss. So here also we"
  },
  {
    "startMs": 503670,
    "text": "Otherwise, it's a miss. So here also we"
  },
  {
    "startMs": 503680,
    "text": "Otherwise, it's a miss. So here also we see the latest models are starting to do"
  },
  {
    "startMs": 505510,
    "text": "see the latest models are starting to do"
  },
  {
    "startMs": 505520,
    "text": "see the latest models are starting to do better. And interestingly, uh looking at"
  },
  {
    "startMs": 508309,
    "text": "better. And interestingly, uh looking at"
  },
  {
    "startMs": 508319,
    "text": "better. And interestingly, uh looking at just the below consensus calls, which"
  },
  {
    "startMs": 510070,
    "text": "just the below consensus calls, which"
  },
  {
    "startMs": 510080,
    "text": "just the below consensus calls, which tend to be harder typically to make, uh"
  },
  {
    "startMs": 512550,
    "text": "tend to be harder typically to make, uh"
  },
  {
    "startMs": 512560,
    "text": "tend to be harder typically to make, uh we see that newer models are starting to"
  },
  {
    "startMs": 515110,
    "text": "we see that newer models are starting to"
  },
  {
    "startMs": 515120,
    "text": "we see that newer models are starting to outperform uh historical majority"
  },
  {
    "startMs": 517670,
    "text": "outperform uh historical majority"
  },
  {
    "startMs": 517680,
    "text": "outperform uh historical majority baseline. So just to talk about the"
  },
  {
    "startMs": 519110,
    "text": "baseline. So just to talk about the"
  },
  {
    "startMs": 519120,
    "text": "baseline. So just to talk about the baseline we looked at last eight"
  },
  {
    "startMs": 520550,
    "text": "baseline we looked at last eight"
  },
  {
    "startMs": 520560,
    "text": "baseline we looked at last eight quarters and saw what was the majority"
  },
  {
    "startMs": 523269,
    "text": "quarters and saw what was the majority"
  },
  {
    "startMs": 523279,
    "text": "quarters and saw what was the majority did it on the whole beat or miss"
  },
  {
    "startMs": 525269,
    "text": "did it on the whole beat or miss"
  },
  {
    "startMs": 525279,
    "text": "did it on the whole beat or miss consensus in the last eight quarters. Um"
  },
  {
    "startMs": 528070,
    "text": "consensus in the last eight quarters. Um"
  },
  {
    "startMs": 528080,
    "text": "consensus in the last eight quarters. Um so yeah so Astra again doing"
  },
  {
    "startMs": 530230,
    "text": "so yeah so Astra again doing"
  },
  {
    "startMs": 530240,
    "text": "so yeah so Astra again doing significantly better and then uh we also"
  },
  {
    "startMs": 533590,
    "text": "significantly better and then uh we also"
  },
  {
    "startMs": 533600,
    "text": "significantly better and then uh we also have a new metric which is you know um"
  },
  {
    "startMs": 536150,
    "text": "have a new metric which is you know um"
  },
  {
    "startMs": 536160,
    "text": "have a new metric which is you know um surprise correlation. So was a big uh"
  },
  {
    "startMs": 539750,
    "text": "surprise correlation. So was a big uh"
  },
  {
    "startMs": 539760,
    "text": "surprise correlation. So was a big uh predicted beat also a big actual beat"
  },
  {
    "startMs": 543430,
    "text": "predicted beat also a big actual beat"
  },
  {
    "startMs": 543440,
    "text": "predicted beat also a big actual beat versus a big um predicted miss a big"
  },
  {
    "startMs": 546949,
    "text": "versus a big um predicted miss a big"
  },
  {
    "startMs": 546959,
    "text": "versus a big um predicted miss a big actual miss. So that allows us to"
  },
  {
    "startMs": 549110,
    "text": "actual miss. So that allows us to"
  },
  {
    "startMs": 549120,
    "text": "actual miss. So that allows us to combine both the um magnitude as well as"
  },
  {
    "startMs": 552150,
    "text": "combine both the um magnitude as well as"
  },
  {
    "startMs": 552160,
    "text": "combine both the um magnitude as well as the direction together and gives us more"
  },
  {
    "startMs": 554710,
    "text": "the direction together and gives us more"
  },
  {
    "startMs": 554720,
    "text": "the direction together and gives us more statistical power uh over all of these"
  },
  {
    "startMs": 557030,
    "text": "statistical power uh over all of these"
  },
  {
    "startMs": 557040,
    "text": "statistical power uh over all of these models. And so here we see again Astra,"
  },
  {
    "startMs": 560070,
    "text": "models. And so here we see again Astra,"
  },
  {
    "startMs": 560080,
    "text": "models. And so here we see again Astra, Opus and Fable are significantly"
  },
  {
    "startMs": 562230,
    "text": "Opus and Fable are significantly"
  },
  {
    "startMs": 562240,
    "text": "Opus and Fable are significantly outperforming the baseline. And uh there"
  },
  {
    "startMs": 565190,
    "text": "outperforming the baseline. And uh there"
  },
  {
    "startMs": 565200,
    "text": "outperforming the baseline. And uh there is uh you know with they are within"
  },
  {
    "startMs": 568150,
    "text": "is uh you know with they are within"
  },
  {
    "startMs": 568160,
    "text": "is uh you know with they are within statistical significance of each other."
  },
  {
    "startMs": 570470,
    "text": "statistical significance of each other."
  },
  {
    "startMs": 570480,
    "text": "statistical significance of each other. Okay. So we looked at a bunch of"
  },
  {
    "startMs": 571910,
    "text": "Okay. So we looked at a bunch of"
  },
  {
    "startMs": 571920,
    "text": "Okay. So we looked at a bunch of numbers. You know the numbers tell one"
  },
  {
    "startMs": 573670,
    "text": "numbers. You know the numbers tell one"
  },
  {
    "startMs": 573680,
    "text": "numbers. You know the numbers tell one part of the story. But we also want to"
  },
  {
    "startMs": 575590,
    "text": "part of the story. But we also want to"
  },
  {
    "startMs": 575600,
    "text": "part of the story. But we also want to know what's really happening uh under"
  },
  {
    "startMs": 577750,
    "text": "know what's really happening uh under"
  },
  {
    "startMs": 577760,
    "text": "know what's really happening uh under the hood, right? So um here's here's an"
  },
  {
    "startMs": 581670,
    "text": "the hood, right? So um here's here's an"
  },
  {
    "startMs": 581680,
    "text": "the hood, right? So um here's here's an example. Um what we find is that agents"
  },
  {
    "startMs": 585350,
    "text": "example. Um what we find is that agents"
  },
  {
    "startMs": 585360,
    "text": "example. Um what we find is that agents win when they are gathering evidence and"
  },
  {
    "startMs": 587670,
    "text": "win when they are gathering evidence and"
  },
  {
    "startMs": 587680,
    "text": "win when they are gathering evidence and they are being a they are able to adjust"
  },
  {
    "startMs": 589829,
    "text": "they are being a they are able to adjust"
  },
  {
    "startMs": 589839,
    "text": "they are being a they are able to adjust away from consensus. So in this example"
  },
  {
    "startMs": 592710,
    "text": "away from consensus. So in this example"
  },
  {
    "startMs": 592720,
    "text": "away from consensus. So in this example we have MACM uh which is a"
  },
  {
    "startMs": 595110,
    "text": "we have MACM uh which is a"
  },
  {
    "startMs": 595120,
    "text": "we have MACM uh which is a semi-manufacturer that supplies data"
  },
  {
    "startMs": 597190,
    "text": "semi-manufacturer that supplies data"
  },
  {
    "startMs": 597200,
    "text": "semi-manufacturer that supplies data centers and then the agent found one of"
  },
  {
    "startMs": 599910,
    "text": "centers and then the agent found one of"
  },
  {
    "startMs": 599920,
    "text": "centers and then the agent found one of its suppliers a small UK wafer uh"
  },
  {
    "startMs": 602949,
    "text": "its suppliers a small UK wafer uh"
  },
  {
    "startMs": 602959,
    "text": "its suppliers a small UK wafer uh manufacturer uh it uh it revealed sort"
  },
  {
    "startMs": 607190,
    "text": "manufacturer uh it uh it revealed sort"
  },
  {
    "startMs": 607200,
    "text": "manufacturer uh it uh it revealed sort of MACM's data center improvements like"
  },
  {
    "startMs": 610230,
    "text": "of MACM's data center improvements like"
  },
  {
    "startMs": 610240,
    "text": "of MACM's data center improvements like in revenue and it adjusted for it uh"
  },
  {
    "startMs": 613590,
    "text": "in revenue and it adjusted for it uh"
  },
  {
    "startMs": 613600,
    "text": "in revenue and it adjusted for it uh went all the way down to the P&L and"
  },
  {
    "startMs": 616150,
    "text": "went all the way down to the P&L and"
  },
  {
    "startMs": 616160,
    "text": "went all the way down to the P&L and found uh and predicted a forecast that"
  },
  {
    "startMs": 618710,
    "text": "found uh and predicted a forecast that"
  },
  {
    "startMs": 618720,
    "text": "found uh and predicted a forecast that was significantly higher than consensus"
  },
  {
    "startMs": 621269,
    "text": "was significantly higher than consensus"
  },
  {
    "startMs": 621279,
    "text": "was significantly higher than consensus or bias corrected consensus and very"
  },
  {
    "startMs": 623509,
    "text": "or bias corrected consensus and very"
  },
  {
    "startMs": 623519,
    "text": "or bias corrected consensus and very close to actual. So this is the kind of"
  },
  {
    "startMs": 625670,
    "text": "close to actual. So this is the kind of"
  },
  {
    "startMs": 625680,
    "text": "close to actual. So this is the kind of reasoning uh these models are doing"
  },
  {
    "startMs": 627430,
    "text": "reasoning uh these models are doing"
  },
  {
    "startMs": 627440,
    "text": "reasoning uh these models are doing where they're able to find data uh that"
  },
  {
    "startMs": 630150,
    "text": "where they're able to find data uh that"
  },
  {
    "startMs": 630160,
    "text": "where they're able to find data uh that is very timely uh over a large scale and"
  },
  {
    "startMs": 632870,
    "text": "is very timely uh over a large scale and"
  },
  {
    "startMs": 632880,
    "text": "is very timely uh over a large scale and then are able to adjust for it. Uh on"
  },
  {
    "startMs": 635990,
    "text": "then are able to adjust for it. Uh on"
  },
  {
    "startMs": 636000,
    "text": "then are able to adjust for it. Uh on the other hand uh we also find that they"
  },
  {
    "startMs": 638550,
    "text": "the other hand uh we also find that they"
  },
  {
    "startMs": 638560,
    "text": "the other hand uh we also find that they miss when they see the evidence but they"
  },
  {
    "startMs": 640550,
    "text": "miss when they see the evidence but they"
  },
  {
    "startMs": 640560,
    "text": "miss when they see the evidence but they don't have enough conviction to act on"
  },
  {
    "startMs": 642069,
    "text": "don't have enough conviction to act on"
  },
  {
    "startMs": 642079,
    "text": "don't have enough conviction to act on it. So this is Ross stores uh the dress"
  },
  {
    "startMs": 645269,
    "text": "it. So this is Ross stores uh the dress"
  },
  {
    "startMs": 645279,
    "text": "it. So this is Ross stores uh the dress for less Ross and um here the agent"
  },
  {
    "startMs": 649269,
    "text": "for less Ross and um here the agent"
  },
  {
    "startMs": 649279,
    "text": "for less Ross and um here the agent found that one of its peers had booked a"
  },
  {
    "startMs": 651430,
    "text": "found that one of its peers had booked a"
  },
  {
    "startMs": 651440,
    "text": "found that one of its peers had booked a tariff refund but then it decided to"
  },
  {
    "startMs": 653509,
    "text": "tariff refund but then it decided to"
  },
  {
    "startMs": 653519,
    "text": "tariff refund but then it decided to leave it out of its um prediction for"
  },
  {
    "startMs": 658230,
    "text": "leave it out of its um prediction for"
  },
  {
    "startMs": 658240,
    "text": "leave it out of its um prediction for um gross margin and then the prediction"
  },
  {
    "startMs": 661269,
    "text": "um gross margin and then the prediction"
  },
  {
    "startMs": 661279,
    "text": "um gross margin and then the prediction uh like very was very close to the"
  },
  {
    "startMs": 662870,
    "text": "uh like very was very close to the"
  },
  {
    "startMs": 662880,
    "text": "uh like very was very close to the consensus but the actual came out much"
  },
  {
    "startMs": 664870,
    "text": "consensus but the actual came out much"
  },
  {
    "startMs": 664880,
    "text": "consensus but the actual came out much higher and so we see sort of them"
  },
  {
    "startMs": 667910,
    "text": "higher and so we see sort of them"
  },
  {
    "startMs": 667920,
    "text": "higher and so we see sort of them missing when they don't work with"
  },
  {
    "startMs": 669509,
    "text": "missing when they don't work with"
  },
  {
    "startMs": 669519,
    "text": "missing when they don't work with conviction. These are two examples."
  },
  {
    "startMs": 671430,
    "text": "conviction. These are two examples."
  },
  {
    "startMs": 671440,
    "text": "conviction. These are two examples. There are more examples on the online"
  },
  {
    "startMs": 673430,
    "text": "There are more examples on the online"
  },
  {
    "startMs": 673440,
    "text": "There are more examples on the online blog post. U but yeah, this is just to"
  },
  {
    "startMs": 675670,
    "text": "blog post. U but yeah, this is just to"
  },
  {
    "startMs": 675680,
    "text": "blog post. U but yeah, this is just to give a flavor of sort of what's"
  },
  {
    "startMs": 676949,
    "text": "give a flavor of sort of what's"
  },
  {
    "startMs": 676959,
    "text": "give a flavor of sort of what's happening behind the scenes. U okay so"
  },
  {
    "startMs": 680870,
    "text": "happening behind the scenes. U okay so"
  },
  {
    "startMs": 680880,
    "text": "happening behind the scenes. U okay so uh here's the last interesting thing"
  },
  {
    "startMs": 682870,
    "text": "uh here's the last interesting thing"
  },
  {
    "startMs": 682880,
    "text": "uh here's the last interesting thing before we kind of jump into some of the"
  },
  {
    "startMs": 684870,
    "text": "before we kind of jump into some of the"
  },
  {
    "startMs": 684880,
    "text": "before we kind of jump into some of the uplations. So uh if you look at you know"
  },
  {
    "startMs": 688790,
    "text": "uplations. So uh if you look at you know"
  },
  {
    "startMs": 688800,
    "text": "uplations. So uh if you look at you know these predictions which are"
  },
  {
    "startMs": 690230,
    "text": "these predictions which are"
  },
  {
    "startMs": 690240,
    "text": "these predictions which are significantly above or below consensus"
  },
  {
    "startMs": 692630,
    "text": "significantly above or below consensus"
  },
  {
    "startMs": 692640,
    "text": "significantly above or below consensus we find that the models tend to"
  },
  {
    "startMs": 694870,
    "text": "we find that the models tend to"
  },
  {
    "startMs": 694880,
    "text": "we find that the models tend to outperform when the surprises are bigger"
  },
  {
    "startMs": 697190,
    "text": "outperform when the surprises are bigger"
  },
  {
    "startMs": 697200,
    "text": "outperform when the surprises are bigger which is also very interesting. like you"
  },
  {
    "startMs": 698710,
    "text": "which is also very interesting. like you"
  },
  {
    "startMs": 698720,
    "text": "which is also very interesting. like you can look at the things that these models"
  },
  {
    "startMs": 700790,
    "text": "can look at the things that these models"
  },
  {
    "startMs": 700800,
    "text": "can look at the things that these models tend to uh have on the extremes and then"
  },
  {
    "startMs": 703670,
    "text": "tend to uh have on the extremes and then"
  },
  {
    "startMs": 703680,
    "text": "tend to uh have on the extremes and then learn sort of what they are doing there."
  },
  {
    "startMs": 707190,
    "text": "learn sort of what they are doing there."
  },
  {
    "startMs": 707200,
    "text": "learn sort of what they are doing there. Okay. So we looked at how a model can"
  },
  {
    "startMs": 709829,
    "text": "Okay. So we looked at how a model can"
  },
  {
    "startMs": 709839,
    "text": "Okay. So we looked at how a model can change our predictions. But what about"
  },
  {
    "startMs": 711750,
    "text": "change our predictions. But what about"
  },
  {
    "startMs": 711760,
    "text": "change our predictions. But what about the data and the hardness that goes into"
  },
  {
    "startMs": 714069,
    "text": "the data and the hardness that goes into"
  },
  {
    "startMs": 714079,
    "text": "the data and the hardness that goes into it, right?"
  },
  {
    "startMs": 715910,
    "text": "it, right?"
  },
  {
    "startMs": 715920,
    "text": "it, right? So um we found that access to life data"
  },
  {
    "startMs": 718949,
    "text": "So um we found that access to life data"
  },
  {
    "startMs": 718959,
    "text": "So um we found that access to life data is the most important factor in accurate"
  },
  {
    "startMs": 721269,
    "text": "is the most important factor in accurate"
  },
  {
    "startMs": 721279,
    "text": "is the most important factor in accurate financial predictions. So uh we ran two"
  },
  {
    "startMs": 724710,
    "text": "financial predictions. So uh we ran two"
  },
  {
    "startMs": 724720,
    "text": "financial predictions. So uh we ran two comparisons. one is with just the"
  },
  {
    "startMs": 727269,
    "text": "comparisons. one is with just the"
  },
  {
    "startMs": 727279,
    "text": "comparisons. one is with just the parametric knowledge. So no data access"
  },
  {
    "startMs": 729910,
    "text": "parametric knowledge. So no data access"
  },
  {
    "startMs": 729920,
    "text": "parametric knowledge. So no data access just the model itself. And this gives us"
  },
  {
    "startMs": 731990,
    "text": "just the model itself. And this gives us"
  },
  {
    "startMs": 732000,
    "text": "just the model itself. And this gives us a good comparison point because uh Fable"
  },
  {
    "startMs": 734710,
    "text": "a good comparison point because uh Fable"
  },
  {
    "startMs": 734720,
    "text": "a good comparison point because uh Fable was trained until uh June 2026. And so"
  },
  {
    "startMs": 738949,
    "text": "was trained until uh June 2026. And so"
  },
  {
    "startMs": 738959,
    "text": "was trained until uh June 2026. And so technically it should have memorized all"
  },
  {
    "startMs": 740790,
    "text": "technically it should have memorized all"
  },
  {
    "startMs": 740800,
    "text": "technically it should have memorized all the documents that came out before that."
  },
  {
    "startMs": 742790,
    "text": "the documents that came out before that."
  },
  {
    "startMs": 742800,
    "text": "the documents that came out before that. But even uh with all of that in its"
  },
  {
    "startMs": 745350,
    "text": "But even uh with all of that in its"
  },
  {
    "startMs": 745360,
    "text": "But even uh with all of that in its pre-training, it's not able to be like"
  },
  {
    "startMs": 747829,
    "text": "pre-training, it's not able to be like"
  },
  {
    "startMs": 747839,
    "text": "pre-training, it's not able to be like it's not any good at all. Okay. So"
  },
  {
    "startMs": 750710,
    "text": "it's not any good at all. Okay. So"
  },
  {
    "startMs": 750720,
    "text": "it's not any good at all. Okay. So that's the parametric version. What"
  },
  {
    "startMs": 752870,
    "text": "that's the parametric version. What"
  },
  {
    "startMs": 752880,
    "text": "that's the parametric version. What about if we give it stale data? So if"
  },
  {
    "startMs": 755430,
    "text": "about if we give it stale data? So if"
  },
  {
    "startMs": 755440,
    "text": "about if we give it stale data? So if you give it data that is 11 weeks before"
  },
  {
    "startMs": 757910,
    "text": "you give it data that is 11 weeks before"
  },
  {
    "startMs": 757920,
    "text": "you give it data that is 11 weeks before the earnings call uh it still has the"
  },
  {
    "startMs": 760310,
    "text": "the earnings call uh it still has the"
  },
  {
    "startMs": 760320,
    "text": "the earnings call uh it still has the previous earnings result and it's like"
  },
  {
    "startMs": 762550,
    "text": "previous earnings result and it's like"
  },
  {
    "startMs": 762560,
    "text": "previous earnings result and it's like you know the guidance and other analyst"
  },
  {
    "startMs": 764629,
    "text": "you know the guidance and other analyst"
  },
  {
    "startMs": 764639,
    "text": "you know the guidance and other analyst consensus but it's around 12 to 15"
  },
  {
    "startMs": 768150,
    "text": "consensus but it's around 12 to 15"
  },
  {
    "startMs": 768160,
    "text": "consensus but it's around 12 to 15 points lower and to put this in context"
  },
  {
    "startMs": 770790,
    "text": "points lower and to put this in context"
  },
  {
    "startMs": 770800,
    "text": "points lower and to put this in context this is the same gap you would see"
  },
  {
    "startMs": 772150,
    "text": "this is the same gap you would see"
  },
  {
    "startMs": 772160,
    "text": "this is the same gap you would see between sonnet and fable 5.1 in terms of"
  },
  {
    "startMs": 775350,
    "text": "between sonnet and fable 5.1 in terms of"
  },
  {
    "startMs": 775360,
    "text": "between sonnet and fable 5.1 in terms of the quality you would get out of these"
  },
  {
    "startMs": 776710,
    "text": "the quality you would get out of these"
  },
  {
    "startMs": 776720,
    "text": "the quality you would get out of these two models. Uh yeah so latest data is"
  },
  {
    "startMs": 780230,
    "text": "two models. Uh yeah so latest data is"
  },
  {
    "startMs": 780240,
    "text": "two models. Uh yeah so latest data is important also another proof point that"
  },
  {
    "startMs": 782470,
    "text": "important also another proof point that"
  },
  {
    "startMs": 782480,
    "text": "important also another proof point that they are actually you know finding"
  },
  {
    "startMs": 783910,
    "text": "they are actually you know finding"
  },
  {
    "startMs": 783920,
    "text": "they are actually you know finding information and adjusting for it in the"
  },
  {
    "startMs": 786550,
    "text": "information and adjusting for it in the"
  },
  {
    "startMs": 786560,
    "text": "information and adjusting for it in the uh for the latest information."
  },
  {
    "startMs": 789509,
    "text": "uh for the latest information."
  },
  {
    "startMs": 789519,
    "text": "uh for the latest information. Then uh we looked at the effect of"
  },
  {
    "startMs": 791269,
    "text": "Then uh we looked at the effect of"
  },
  {
    "startMs": 791279,
    "text": "Then uh we looked at the effect of expert guidance. So without expert"
  },
  {
    "startMs": 793590,
    "text": "expert guidance. So without expert"
  },
  {
    "startMs": 793600,
    "text": "expert guidance. So without expert guidance uh fewer tokens used fewer"
  },
  {
    "startMs": 797829,
    "text": "guidance uh fewer tokens used fewer"
  },
  {
    "startMs": 797839,
    "text": "guidance uh fewer tokens used fewer minutes spent on research but with"
  },
  {
    "startMs": 800230,
    "text": "minutes spent on research but with"
  },
  {
    "startMs": 800240,
    "text": "minutes spent on research but with expert guidance we see significantly"
  },
  {
    "startMs": 802470,
    "text": "expert guidance we see significantly"
  },
  {
    "startMs": 802480,
    "text": "expert guidance we see significantly more research being put into things. And"
  },
  {
    "startMs": 805030,
    "text": "more research being put into things. And"
  },
  {
    "startMs": 805040,
    "text": "more research being put into things. And in the case of Astra, we have a"
  },
  {
    "startMs": 807829,
    "text": "in the case of Astra, we have a"
  },
  {
    "startMs": 807839,
    "text": "in the case of Astra, we have a significant difference improvement in"
  },
  {
    "startMs": 810230,
    "text": "significant difference improvement in"
  },
  {
    "startMs": 810240,
    "text": "significant difference improvement in the predicted error."
  },
  {
    "startMs": 812949,
    "text": "the predicted error."
  },
  {
    "startMs": 812959,
    "text": "the predicted error. Finally, uh we wanted to see what"
  },
  {
    "startMs": 815190,
    "text": "Finally, uh we wanted to see what"
  },
  {
    "startMs": 815200,
    "text": "Finally, uh we wanted to see what happens if you don't give any consensus"
  },
  {
    "startMs": 817829,
    "text": "happens if you don't give any consensus"
  },
  {
    "startMs": 817839,
    "text": "happens if you don't give any consensus to these models. So we created a setting"
  },
  {
    "startMs": 819750,
    "text": "to these models. So we created a setting"
  },
  {
    "startMs": 819760,
    "text": "to these models. So we created a setting where we removed access to all the"
  },
  {
    "startMs": 821430,
    "text": "where we removed access to all the"
  },
  {
    "startMs": 821440,
    "text": "where we removed access to all the consensus. We even had like document"
  },
  {
    "startMs": 824550,
    "text": "consensus. We even had like document"
  },
  {
    "startMs": 824560,
    "text": "consensus. We even had like document extraction tools redact any mentions of"
  },
  {
    "startMs": 827110,
    "text": "extraction tools redact any mentions of"
  },
  {
    "startMs": 827120,
    "text": "extraction tools redact any mentions of consensus from any sort of news sources"
  },
  {
    "startMs": 829910,
    "text": "consensus from any sort of news sources"
  },
  {
    "startMs": 829920,
    "text": "consensus from any sort of news sources as well. And what we see here is that"
  },
  {
    "startMs": 832310,
    "text": "as well. And what we see here is that"
  },
  {
    "startMs": 832320,
    "text": "as well. And what we see here is that the the best models uh don't really need"
  },
  {
    "startMs": 835190,
    "text": "the the best models uh don't really need"
  },
  {
    "startMs": 835200,
    "text": "the the best models uh don't really need consensus anymore. So they are able to"
  },
  {
    "startMs": 837189,
    "text": "consensus anymore. So they are able to"
  },
  {
    "startMs": 837199,
    "text": "consensus anymore. So they are able to reconstruct the consensus view. Uh in"
  },
  {
    "startMs": 839990,
    "text": "reconstruct the consensus view. Uh in"
  },
  {
    "startMs": 840000,
    "text": "reconstruct the consensus view. Uh in fact in the case of Astra there is no"
  },
  {
    "startMs": 842310,
    "text": "fact in the case of Astra there is no"
  },
  {
    "startMs": 842320,
    "text": "fact in the case of Astra there is no significant difference between with and"
  },
  {
    "startMs": 844230,
    "text": "significant difference between with and"
  },
  {
    "startMs": 844240,
    "text": "significant difference between with and without consensus whereas for the older"
  },
  {
    "startMs": 846310,
    "text": "without consensus whereas for the older"
  },
  {
    "startMs": 846320,
    "text": "without consensus whereas for the older models you did like see a big jump with"
  },
  {
    "startMs": 848710,
    "text": "models you did like see a big jump with"
  },
  {
    "startMs": 848720,
    "text": "models you did like see a big jump with access to consensus. Um so yeah so we"
  },
  {
    "startMs": 852470,
    "text": "access to consensus. Um so yeah so we"
  },
  {
    "startMs": 852480,
    "text": "access to consensus. Um so yeah so we are seeing frontier models outperforming"
  },
  {
    "startMs": 854310,
    "text": "are seeing frontier models outperforming"
  },
  {
    "startMs": 854320,
    "text": "are seeing frontier models outperforming consensus. Uh so the question is you"
  },
  {
    "startMs": 856629,
    "text": "consensus. Uh so the question is you"
  },
  {
    "startMs": 856639,
    "text": "consensus. Uh so the question is you know what's next? Uh this is available"
  },
  {
    "startMs": 858629,
    "text": "know what's next? Uh this is available"
  },
  {
    "startMs": 858639,
    "text": "know what's next? Uh this is available for alpha testing in the product. But"
  },
  {
    "startMs": 860230,
    "text": "for alpha testing in the product. But"
  },
  {
    "startMs": 860240,
    "text": "for alpha testing in the product. But we're also working with customers to see"
  },
  {
    "startMs": 862310,
    "text": "we're also working with customers to see"
  },
  {
    "startMs": 862320,
    "text": "we're also working with customers to see if their internal data is useful for"
  },
  {
    "startMs": 864150,
    "text": "if their internal data is useful for"
  },
  {
    "startMs": 864160,
    "text": "if their internal data is useful for some of these predictive tasks. So feel"
  },
  {
    "startMs": 865829,
    "text": "some of these predictive tasks. So feel"
  },
  {
    "startMs": 865839,
    "text": "some of these predictive tasks. So feel free to get in touch. We are doing"
  },
  {
    "startMs": 867590,
    "text": "free to get in touch. We are doing"
  },
  {
    "startMs": 867600,
    "text": "free to get in touch. We are doing research on model reasoning. So I'm"
  },
  {
    "startMs": 870069,
    "text": "research on model reasoning. So I'm"
  },
  {
    "startMs": 870079,
    "text": "research on model reasoning. So I'm trying to understand if this reasoning"
  },
  {
    "startMs": 871670,
    "text": "trying to understand if this reasoning"
  },
  {
    "startMs": 871680,
    "text": "trying to understand if this reasoning is the same way as humans do their"
  },
  {
    "startMs": 873670,
    "text": "is the same way as humans do their"
  },
  {
    "startMs": 873680,
    "text": "is the same way as humans do their reasoning. If not, is there a way to"
  },
  {
    "startMs": 875990,
    "text": "reasoning. If not, is there a way to"
  },
  {
    "startMs": 876000,
    "text": "reasoning. If not, is there a way to complement human reasoning to try and"
  },
  {
    "startMs": 877750,
    "text": "complement human reasoning to try and"
  },
  {
    "startMs": 877760,
    "text": "complement human reasoning to try and you know get the best of both worlds. Um"
  },
  {
    "startMs": 880230,
    "text": "you know get the best of both worlds. Um"
  },
  {
    "startMs": 880240,
    "text": "you know get the best of both worlds. Um and then uh we're looking at"
  },
  {
    "startMs": 881910,
    "text": "and then uh we're looking at"
  },
  {
    "startMs": 881920,
    "text": "and then uh we're looking at post-raining and continual learning. So"
  },
  {
    "startMs": 884150,
    "text": "post-raining and continual learning. So"
  },
  {
    "startMs": 884160,
    "text": "post-raining and continual learning. So these agents just like humans right you"
  },
  {
    "startMs": 886629,
    "text": "these agents just like humans right you"
  },
  {
    "startMs": 886639,
    "text": "these agents just like humans right you can learn through experience. you can"
  },
  {
    "startMs": 888150,
    "text": "can learn through experience. you can"
  },
  {
    "startMs": 888160,
    "text": "can learn through experience. you can learn by the mistakes you've made in the"
  },
  {
    "startMs": 889750,
    "text": "learn by the mistakes you've made in the"
  },
  {
    "startMs": 889760,
    "text": "learn by the mistakes you've made in the past and so can the models keep learning"
  },
  {
    "startMs": 892150,
    "text": "past and so can the models keep learning"
  },
  {
    "startMs": 892160,
    "text": "past and so can the models keep learning from the mistakes in the past I think"
  },
  {
    "startMs": 893509,
    "text": "from the mistakes in the past I think"
  },
  {
    "startMs": 893519,
    "text": "from the mistakes in the past I think that's the other interesting question uh"
  },
  {
    "startMs": 896470,
    "text": "that's the other interesting question uh"
  },
  {
    "startMs": 896480,
    "text": "that's the other interesting question uh you have a couple of QR codes uh we have"
  },
  {
    "startMs": 898710,
    "text": "you have a couple of QR codes uh we have"
  },
  {
    "startMs": 898720,
    "text": "you have a couple of QR codes uh we have the blog post uh there's a lot more"
  },
  {
    "startMs": 900790,
    "text": "the blog post uh there's a lot more"
  },
  {
    "startMs": 900800,
    "text": "the blog post uh there's a lot more details there also a link to Frontier"
  },
  {
    "startMs": 903590,
    "text": "details there also a link to Frontier"
  },
  {
    "startMs": 903600,
    "text": "details there also a link to Frontier Finance it might be of interest if you"
  },
  {
    "startMs": 905430,
    "text": "Finance it might be of interest if you"
  },
  {
    "startMs": 905440,
    "text": "Finance it might be of interest if you are trying to evaluate your own sort of"
  },
  {
    "startMs": 907590,
    "text": "are trying to evaluate your own sort of"
  },
  {
    "startMs": 907600,
    "text": "are trying to evaluate your own sort of hardnesses um yeah and uh Arash and I"
  },
  {
    "startMs": 912150,
    "text": "hardnesses um yeah and uh Arash and I"
  },
  {
    "startMs": 912160,
    "text": "hardnesses um yeah and uh Arash and I will be here would love to chat with you"
  },
  {
    "startMs": 913750,
    "text": "will be here would love to chat with you"
  },
  {
    "startMs": 913760,
    "text": "will be here would love to chat with you thank"
  }
]