Trust, but Verify: High-Fidelity Reasoning in Agentic Workflows

Talk transcript

1. 0:00

   [on-hold music] I'm Mike Conover.

2. 0:17

   I am founder and CEO of Brightwave. Uh, we build a research agent that digests very large corpuses of content in the financial domain. So you can think of due diligence in a competitive deal process.

3. 0:30

   You are pre-term sheet, you step into a data room with thousands of pages of content. Uh, you need to get to conviction quickly ahead of, uh, other teams. You need to spot, uh, critical risk factors that would, would diminish asset performance.

4. 0:44

   Um, it's a fairly non-trivial task. Um, you think about mutual fund analysts. It's earnings season. You've got a universe of coverage of eighty, one hundred and twenty names. There are calls, transcripts, filings.

5. 0:56

   It, it's, um, a fairly non-trivial problem to understand, uh, at a sector level, but also at the individual, uh, ticker level what's, what's happening in the market. Um, or goodness, you get into confirmatory diligence, and you've got eight-- eighty, eight hundred vendor contracts, and you need to spot, uh, t- early termination clauses.

6. 1:16

   You need to understand thematically, how is my entire portfolio, uh, negotiating their vendor contracts? It's, um, frankly not a human-level intelligence task. And the reality, as we've stepped into this space, um, is that these, uh, these professionals, uh, just get put in a meat grinder.

7. 1:36

   Junior analysts are, um, tasked to do the impossible on extremely tight deadlines. Um, I come from a, a technical background. Um, prior to Brightwave, I was at Databricks and created a language model called Dolly, uh, that was one of the earlier models to demonstrate the power of instruction tuning, um, for eliciting, uh, instruction following behavior from, from

8. 1:57

   open source technologies. And, um, as I have met with these professionals, I have developed a deep sense of empathy for, um, the stakes and the human cost of doing this work, uh, manually.

9. 2:12

   When it comes to the role of the individual in, uh, finance workflows and financial research, um, we think of the parallels to early, early spreadsheets. You go to an accountant or finance professional in 1978 before the advent of computational spreadsheets.

10. 2:29

    You say, "What's your job?" "Well, I run the numbers." It's cognitively demanding. These people write this stuff out by hand on literally wide pieces of paper called spreadsheets. It's cognitively demanding.

11. 2:38

    It's important to the business, and it's time-intensive. It feels like real work. And now, nobody wants that job. And it's not because there aren't finance professionals. It's not because nobody's doing analysis.

12. 2:50

    It's the sophistication of the thought that you can bring to bear on the problem has increased so substantially because there are tools that allow us to think more effectively, more efficiently.

13. 3:01

    What we're seeing, what we're hearing from our customers is that a system like Brightwave that is able to diges-- and not just Brightwave. These-- this class of knowledge agents is able to digest volumes of content and perform meaningful work that accelerates by orders of magnitude, um, the efficiency and also, uh, time to, to value in, in these

14. 3:23

    markets. And so the purpose of this talk is to relate, mm,

15. 3:30

    sort of the intelligence that we've developed, uh, in, in the course of building this high-fidelity research agent, um, and just things that we're seeing both technically but also in terms of product affordances.

16. 3:40

    I mean, the, the design problem that you have to solve is how do you reveal the thought process of something that's considered ten thousand pages of content to a human in a way that's useful and legible?

17. 3:51

    That is not a UI/UX problem. It's not a product architecture problem that existed three years ago. And the final form factor has not been determined. Chat, everybody's very target fixated on chat.

18. 4:04

    Um, that's probably not enough. So the, the first thing that I'll observe is that non-reasoning models are performing greedy local search. So the, the Bloomberg talk highlighted that sort of fidelity issue.

19. 4:17

    Like a really concrete example, you put a Reuters article in Foro, and you ask it to extract all the organizations, goodness if it's not gonna give you products. And if you have an, a ten, five percent error rate, and you chain calls like that, you're gonna introduce, um, sort of in an exponential way, uh, the likelihood of,

20. 4:33

    of error being in these models. And so the, the winning systems will perform end-to-end RL over tool-use calls, where the results of the API call are in fact part of the RLled, um, sequence of decisions so that you can make locally suboptimal decisions in order to get globally optimal outputs.

21. 4:51

    Um, the reality is that that's still an open research problem. You know, how do I avail myself of a knowledge graph or--

22. 4:59

    I did not do that. [audience laughing] Um, uh, how do you avail yourself of these tools in an intelligent way, um, so that you get globally optimal outputs? It, it does seem like that that is not a solved question.

23. 5:12

    So the reality, and I think it's, like, heartening to see, um, this is a theme, and I think everybody in this room can be sort of comforted by this.

24. 5:20

    You got, you gotta build a product today, and, like, you're g- you're gonna-- there's gonna be this talk of the bitter lesson that more data, more compute, better models dominate all other approaches.

25. 5:28

    Like, nobody wants an expert system. Nobody, nobody wants to use spaCy to do named entity recognition. Um, the-- Sorry. Um, that was not in the speaker notes. Uh, it-- you can think of being more circumspect about what is the scope of behaviors that the system, the agent is going to engage in, sort of like a regularization parameter,

26. 5:48

    which constrains the complexity of the model, and that limits the likelihood, reduces the likelihood that it will go truly off the rails and begin to produce, uh, degenerate output.

27. 5:57

    You can think of it sort of like a-

28. 6:01

    Like multi-turn-- The most interesting interactions I've had with language models are deep into a conversational tree, where you can think of selecting at each branch, each response. There are a set of, uh, reactions that I can have to the model output, and I'm steering, I'm choosing.

29. 6:15

    This is what knowing how to use language models, it's, it's a skill. Um, and many people who have real full-time jobs may not invest in developing that skill. This is not dissimilar to what these RL systems are doing.

30. 6:28

    And if you can think of a multi-turn conversation as not just establishing a, a human-orchestrated chain of thought, but really that set of tokens defines the activations of the model.

31. 6:41

    And if you think of the activations of the model as defining a program, what you are doing when you respond to the model and say, "No, not quite like that, more like this," is if you think of the, the activation weights at, or the activations as a point in a vector space, you are n-nudging the activations to

32. 7:00

    a place where they can finally solve the problem at hand. And I think that's what the chain-of-thought process or the sort of reasoning monologue is performing. It's, it's getting the activations to a position where it can actually solve the problem.

33. 7:11

    So it's actually not-- I don't-- It's cute that it, you can interpret it, but I would prefer if it just got to the right set of activations automatically. Um, and so from a product affordance standpoint, [sighs]

34. 7:23

    people are not going to want to really become prompting experts in a deep way. And, and frankly, it takes, you know, easily a thousand hours. Um, and so the scaffolding that products put in place in order to orchestrate these workflows and, and shape and the, the, the behavior of these systems, um, I think ha-- you know, these

35. 7:42

    verticalized product workflows are, are probably going to be enduring because they specify intent. They take that weight off the user. Um, so some of the things that we see with respect to archetypal design patterns in this space, consider a, a basic autonomous agent.

36. 7:58

    You really wanna mimic the human decision-making process and decompose what is it that a person would do. Well, if I need to understand how this, uh, poly, uh, polypropylene resin, uh, manufacturer, um, is, is managing costs, I might look for public-market comparables, and that would inc-- that would, you know, maybe entail going to the SEC filings or

37. 8:18

    earnings-call transcripts. And I would assess content potentially from a knowledge graph constructed from, uh, p-previous deals that, that, that, you know, I as a, as a private equity investor have done.

38. 8:28

    Um, news corpuses assess which, which document sets are relevant to me, distill down from those documents findings that substantiate, mm, premises or hypotheses that I might have about this question or this investment thesis.

39. 8:42

    Um, and then enrich and error-correct those findings. And so a couple points on this. One is that, um, it is actually-- So I forget who it was, but they were talking about-- It was the deep research team talking about, um, on that next step, what are my intermediary notes?

40. 8:58

    What is it that I believed on the basis of what I found? That's actually an extremely useful think out loud about what do we believe given the facts as they, uh, have materialized on that first pass through the n- the, the dataset.

41. 9:11

    Um, enriching individual findings that are distilled down from documents is an extremely powerful, um, design pattern. Likewise, um, it's, it's ki-- you can ask these models, you know, is this accurate?

42. 9:24

    For that Reuters example, you can say, uh, is this factually entailed by this document, or is this actually an organization? Um, and the model can frequently self-correct. And what we've noticed is that it is-- You can do that in the JSON, um, as sort of like a chain-of-thought behavior, but it's also-- it's actually more powerful to do

43. 9:41

    it as a secondary call because the model's kind of primed to be credulous. It says, "Well, you know, I, I told you it was, and so, yeah, I'm probably right."

44. 9:50

    Um, so it's interesting how you can tease apart some of these steps into multiple different, uh, calls. And then through this process of synthesis, you're able to weave together fact patterns across many, many, many documents into a coherent narrative.

45. 10:02

    Um, and that control loop, we think that obviously human oversight is extremely important. Um, the ability to nudge the model, um, with directives or, or sort of selecting, "The-- This is an interesting thread I want you to pull," that is extremely important.

46. 10:18

    And that's because the human analyst always is going to have access to information that has not been digitized. That's that conversation with management. That's, uh, your portfolio manager thinks b-- this class of biotech is just harebrained.

47. 10:30

    Um, that taste-making, I think, is going to be where you see, um, the most powerful, uh, products lean. I firmly believe with respect to the nodes in that knowledge graph, and we probab-- many people in this room probably have reached this un-this conclusion as well, but you still see this, oh, we got a portfolio manager agent, and

48. 10:47

    this is the fact checker. And, and that sort of it needless like anthropomorphizing of these systems, um, it constrains your flexibility if the design needs of your compute graph change.

49. 10:59

    And this is, this nineteen seventy-- I think it was 1978, and Bell, you know, the Unix philosophy, it's like you think about piping and teeing on the Bash command line, or I guess I date myself, I still use Bash, not Zshale.

50. 11:10

    Um, y- just simple tools that do one thing and that work together well. And text is the universal interface. Um, it's forty years ago. Fifty years ago. Geez. Um, so our friends at Layton Space put together this plot.

51. 11:24

    With respect to the structure of these graphs, I'd-- obviously, that Pareto frontier, which is the sort of efficiency frontier. [phone ringing] It's two. Batting a thousand today. Um, the efficiency frontier for compute and performance trade-off or price-performance trade-off, um, that frontier is going to continue to move out, but there will, I believe there will, for at least enduring time,

52. 11:43

    be a frontier. And what's notable about that is that you have to select then which tool, which system, which model am I going to use for each node in the compute graph.

53. 11:53

    And the reason that this is important is

54. 11:59

    what I call the latency trap If you think about the plot of time to value and realized value for agentic systems, and I think this is extremely important, it's very easy to think, "Oh man, it's gonna do all of these things.

55. 12:11

    It's going to, you know, I'm gonna check it and error-correct, and then it, you know, in, in twenty-five minutes it's gonna be banger." And I think even with high-quality products like OpenAI's Deep Research, it's, hmm, you're not always sure that what you're going to get out is high quality.

56. 12:26

    So there's, there's kind of like a question of like which side of the diagonal, it's probably not a straight line, but is that product on? But also from a rep's standpoint, the impulse response for the user, how well, how well refined is...

57. 12:38

    You can think of, like, my expectation for what the report is going to look like and what the report actually looks like is the loss. And the user's mental model is developing a sense for how do, how do my prompts elicit behaviors from these models?

58. 12:49

    If it's a eight-minute feedback loop, it's a twenty-minute feedback loop, goodness, you're not gonna do many of those in the course of a day. And your faculty with the system or the product is going to be low.

59. 13:00

    So synthesis is, is really where a lot of the magic happens in these systems, and, um, y- a couple observations. So n- notice that it-- I don't know, has anybody in this room ever had a fifty thousand token response from any model?

60. 13:17

    No? They say that it's, you know, o one is a hundred thousand context, output context length. Um, I'm not so sure, and it's because the instruction tuning demonstrations, these human-gener- synthetic or human-generated outputs that are used to post-train the models, have a characteristic output length.

61. 13:34

    It's hard to write fifty thousand coherent, uh, novel words. And so the likelihood that the models are able to produce, I mean, even, um, o one still is about two thousand, three thousand tokens.

62. 13:47

    It's better than four oh. And so what happens, it's kind of like a com-- there's a compression problem. If I have a very, very large context window for input, I'm compressing that information into a set of tokens.

63. 13:57

    And so it's the, like the difference between writing a book report and a synopsis of each chapter. You can dis-- you can be more focused and, um, specific about what is it that I want those n thousand tokens to be focused on.

64. 14:11

    Um, here we have, uh, you know, I said write, write, uh, an analysis of the global financial crisis. Goodness if I don't think the rise of the shadow banking system warrants more than three sentences.

65. 14:23

    And so if you, if you can be more granular and more specific, um, you can get higher quality, higher fidelity, more information-dense outputs out of these systems by decomposing your researched instructions into multiple subthemes.

66. 14:36

    Um, additionally, the last point I'll make on this problem is that of, uh, the, the presence of recombinant of reasoning demonstrations in the instruction tuning and post-training corpuses is low.

67. 14:48

    So it is, uh, easy to say, here, you know, given the text of The Great Gatsby, this is the epilogue, and write a new epilogue for The Great Gatsby because the cost of internalizing that corpus is fixed, effectively.

68. 15:00

    You read the book, and then you write five epilogues, and it's like, "Goodness, I, I got it."

69. 15:06

    Synthesis really is about weaving together disparate fact patterns from multiple documents. Think about the applications to biomedical literature synthesis. I need to read all of these papers and then have something useful to say that actually brings together the facts from these documents.

70. 15:21

    Now, there's like a, a cute trick you could try, which is to say, given the bibliography of any given paper, write the abstract as an ins-- as a, as a post-training exercise.

71. 15:30

    But it's just really hard to get high-quality, intelligent, thoughtful analysis of many, many, many different documents. And so there are limitations in practice for, uh, even state-of-the-art models in terms of how they are able to manage complex real-world situ-situations, uh, factors like temporality, um, the per-plexity at a...

72. 15:53

    Well. So temporality is hard. Um, and being able to understand, you know, something like a merger and an acquisition, um, you know, this, these pro forma financial statements are different from those that came, um, before the event, um, if they're addendums to contracts.

73. 16:08

    It's important to propagate with, um, evidentiary, um, passages, metadata that contextualizes why do I care about this? What do we think about this document? Um, what-- How should I consider this in relation to the other dem-- uh, pieces of evidence in, in the cor- in the context window?

74. 16:29

    Um, so I'll now shift a little bit with some, some examples from, from the, our, the product that we've built, which is, um, how do you reveal the thought process of something that's considered ten thousand pages of text?

75. 16:42

    And I think that it is more like a surface, and one where you're able to, um...

76. 16:50

    It's kind, so it's kinda like this. Like, people you may know, the Facebook and LinkedIn recommendation algorithm for, for, um,

77. 16:59

    connections, uh, feels uncanny good in part, not because... I mean, the algorithms are okay, not great, um, have gotten a lot better over time. But in your visual cortex, there is a, a bundle of nerves that are, uh, exclusively dedicated to face recognition and the ability to say in a, in a f- you know, six by six

78. 17:20

    grid of faces, "Goodness, I know that person." And so you attend to the things that matter, even if it's actually a low-precision product experience. And so the ability to give the person, um, details on demand is extremely important.

79. 17:34

    Um, we'll see. So here we have a Brightwave report. Um, we, you know, I think the ability to click on a citation and then get additional context about this, not just what document is it from, but how should I be thinking about this?

80. 17:47

    What was the model thinking in the course of this? Um, as well as structured interactive outputs that give you the ability to pull the thread and say, "Well, tell me, tell me more about that rising CapEx spend."

81. 17:59

    In Brightwave, um, you can highlight any passage of text. So it's not just the citations, but you can highlight any passage of text and say, "Tell me more. What are the implications of this?"

82. 18:08

    I think OpenAI gestures towards this with respect to Canvas and the ability to increase the reading level of, of a passage. Having a continuous surface that's not just these citations, um, but in fact any, uh, finding should be interrogable.

83. 18:24

    Um- Likewise, you can think of... I'm actually gonna pause. Is that gonna-- It's not gonna pause. I'm gonna go back and do this again. Um,

84. 18:36

    you can think of the set of things that the model has discovered. It reads all of these documents. It develops a view. It weaves the facts together, um, as a h- as a high-dimensional data structure, and the report is one view on that data structure.

85. 18:49

    It's kind of a low c- low-effort point of entry into the, the space of ideas. You wanna be able to turn over that cube and see, especially in finance, um, the receipts.

86. 19:01

    What's the audit trail for this system that's read all of these materials? And so being able to, in this example, click into the documents is one level, but having all of the findings laid out for you, whether it's a fundraising timeline, um, ongoing litigation, I'm able to, if something catches my attention, click on it, and this is

87. 19:18

    where that, the investor... Hello? [chuckles] Investor or analyst taste comes into play. I'm able to say, "Tell me more about that." It's like a magnifying glass for text. Something catches my eye, this patent litigation.

88. 19:30

    Goodness, that seems important. Um, you had a factory fire in Mexico that wiped out a m- you know, critical single-source supplier. Um, what are you gonna do about that?

89. 19:39

    That ability to drill in and get additional details on demand is extremely important in these systems, and I think candidly, um, we, we do not yet have the final version, the final form factor of this class of products.

90. 19:52

    Um, but it, it's an extremely interesting design problem, and I will say, uh, we are, we are hiring. So these QR codes, not only is it a great place to work, we've got, uh, people from Goldman Sachs and UBS and Meta and Instagram and Ana Plan, and we just hired a senior staff software engineer from Brave.

91. 20:09

    Um, goodness, we got a stacked team. We also have a $10,000 referral bonus, so I'm gonna see a lot more phones come out now. [laughs]

92. 20:17

    Um, $10,000 referral bonus for all of these roles, primarily the product designer and the front-end engineer. We're hiring staff and senior staff-level professionals. We t- we have a small team of extremely experienced individuals, um, and this is structured like the DARPA Red Balloon Challenge, if you're familiar.

93. 20:34

    Um, so if you refer the person that refers the person that we hire, you get 1,000 bucks, and so on, and so on, and so on, all along that, uh, exponentially exploding, uh, referral tree.

94. 20:43

    So we're Brightwave. We build knowledge agents for finance workflows. I appreciate your time today. [upbeat music]