Why Off-the-Shelf AI Doesn't Understand Money

Talk transcript

1. 0:00

   [on hold music] So I have a [REDACTED:age] daughter, and she's absol-absolutely adorable, and the parents here in the room know how insightful that age can be.

2. 0:23

   And she has a complete theory about money by now, and I'll give you an example. So a couple weeks ago, I was driving the car, I was coming in to park, and there was something in my dead end, and I scratched the car.

3. 0:36

   I went out, I'm like, "Oh, man, I can't believe I scratched the car." And then I hear my daughter from the back, and she was like, "Daddy, what's happened?"

4. 0:44

   And, you know, I'm explaining it to her. And then she says, "What's the problem? Just buy another one."

5. 0:52

   So anyway, um, I wanna ask you today with a raise of hand, who uses LLMs, has used LLMs for getting financial advice, a recommendation on something in the financial world?

6. 1:08

   Great. Almost everyone. Wait, keep your hand up if you trusted the answer and you actually followed the advice.

7. 1:19

   Okay. A lot of hands are going down, and that's the core problem. So I had the same thing a couple, a couple, uh, months ago. I had a big decision I was looking to take.

8. 1:31

   Should I invest in, you know, real estate niche or in the stock market niche on a specific area? And, you know, I do AI for finance for a living, so I went the full-blown way, context, brain, um, the books, the knowledge, all my finances combined, all the latest models, and it gave me a recommendation, "You should do

9. 1:53

   A with great reasoning." And then I changed just a little bit one of the assumptions, and it completely flipped. "You should do B. Never do A." And then I tweaked one more small thing, and it went all the way back to A.

10. 2:10

    And at that moment, I understood that the advice sounds good, it sounds sound, but I can't really trust it. And by the end of the talk today, you will understand why off-the-shelf LLMs don't understand money and what you need to do about it.

11. 2:32

    So I'm gonna show you a couple of real examples from a study we're doing at Intuit on thousands and thousands of businesses, around a hundred thousand situations and time frames.

12. 2:44

    This is an example of a small business of a new landlord that is building a rental property business. His first property, and he's down. He's a, he's in a negative cash flow.

13. 2:55

    There's an open loan, and the profit is basically trending into the red. And a question comes up, "How do I improve my profit?" And a frontier model gives the following response: "Go and acquire a second rental property because that'll bring more income and compensate for the deficit."

14. 3:17

    And that model had all of the business's data. Now, that's very risky for someone in the negative, in the red to be doing. On the other hand, a model that is grounded in real outcomes, and what I mean by that is a model that has seen similar situations of such businesses, what they did and what was the

15. 3:38

    outcome, actually recommended to raise prices on the existing tenant by five to ten percent,

16. 3:47

    and to do it before the renewal. Now, some of you are thinking that it's just a matter of context. Just give it more context. And the thing is that this advice is coming based off on real situations of similar businesses and would actually move them into profitability in this case.

17. 4:10

    And this is not a one-off. I can go on and on showing you a lot of examples. This is second one. This is an egg supplier where one customer is seventy percent of the revenue and one vendor is almost all of its cost.

18. 4:23

    Same question: How do I improve my profit? A frontier model says, "Raise your prices fifteen to twenty percent on that customer."

19. 4:34

    Now, you understand that is very risky because you might lose almost all your revenue. On the other hand, the same grounded model and real outcomes went actually to the cost side and recommended to negotiate, um, negotiate the vendor cost pricing for a five to ten percent reduction.

20. 4:55

    So it went to the cost side. It took into account the constraints. So what we saw are two examples of frontiers, I'm talking about leading models in the world today that had all the business context, actually give advice that could be very harmful for the business.

21. 5:14

    And that's what I call the fluent bluff.

22. 5:19

    The fluent bluff is a generic, fluent, and confident answer that frontier LLMs can give you around money because of what they learn on the Internet, blogs, books, advice columns, what people wrote about money, but not based on what actually happened.

23. 5:42

    And I would argue that almost every answer that you see related to money and finances is such. And we're soon gonna release res-- the research I talked about, but I'll give you a highlight from there.

24. 5:59

    On-- Across these hundred thousand businesses and time frames-

25. 6:02

    40% of the time, the essence of the advice that the frontier models were giving was acquire a new customer,

26. 6:11

    which is, you know, everyone would, would wish they could do that. And 14 more additional percent were increase basically the revenue from your product. So combined, more than half of the essence of the advice that was given by the frontier models was acquire new customers and try to increase revenue from the profit.

27. 6:36

    And this is not just me saying, this is a very interesting research coming-- that just came out a couple weeks ago from researchers at Princeton. What they tried to do is simulate and answer the question, can the leading models drive long-horizon business decisions?

28. 6:54

    And what they did is they gave the models a harness with tools and data and everything they would need to take decisions across a simulation of five hundred days.

29. 7:04

    Can they turn a profit? Each model got a million dollars to start with, and guess what happened? Most of the models drove the company bankrupt, and it didn't even take five hundred days.

30. 7:20

    And the interesting part is that they also ran a simple rules-based system, and that rules-based system out-beat almost all of the models. Even the very, very few models that were able to generate some profit, it was in a specific ins-instance, and when you rerun that, they also actually drove bankruptcy.

31. 7:43

    So how can it be that a simple rule-based system out-beats the frontier models today on real business decisions?

32. 7:55

    And here's what I want you to think about. A frontier model has read about money,

33. 8:03

    but a grounded model and real outcome has actually watched what happens.

34. 8:10

    And let me be precise with my argue here. Even if you take all of a company's data, and for example, we at Intuit have all the financial data from QuickBooks, for example, the general ledger, the P&L, the cash flows, everything, the invoices of the business, and you give it to a frontier LLM, it's still just one group

35. 8:28

    of data points on a company. And that's the difference between sounding right and actually being right.

36. 8:38

    So I'm Udi Menkes, and I've been in the AI and finance world for the past fifteen years. I started in the AI science world, leading AI and data teams, and shifted into product management.

37. 8:52

    Becoming in, uh, about four years ago, an AI prins- AI product manager at Intuit long before, by the way, it was cool to become an AI PM. Um, and today, I'm a principal product manager at Intuit.

38. 9:03

    I lead financial intelligence and advisory systems that help Intuit's customers to take better decisions and grow their business. And the question that I fixate on on a daily basis is not which model is the best now that I, I can use.

39. 9:21

    It's what do we fundamentally have that no model access can replicate? And how can we-- can I transform that into AI-native, transformative, and delightful experiences for our customers?

40. 9:37

    Now, I wanna develop some intuition for-- from three different angles on why these models bluff. So the first angle is around context is not experience. And I'll give you another real example.

41. 9:50

    This is an apparel company where eighty percent of the cost is coming from this one vendor. And the textbook answer is, "Go cut your biggest cost," right? Textbook book answer.

42. 10:04

    But the issue here is you cut this cost, that same vendor was actually enabling the generation of ninety-seven percent of that company's revenue. So cut that biggest cost,

43. 10:17

    and you lose almost all the revenue, and that can be great margins on zero dollars.

44. 10:25

    And think about it. If I give you an option to work with two, two different advisors. One advisor is a very experienced one, years of experience working with businesses, guiding them.

45. 10:39

    And another advisor, which is very, very smart. They know all the textbook. They're fresh. They read everything. They know AI in and out, but they don't have experience. I would bet you would always go with the experienced one.

46. 10:54

    And that's the same thing with the models in what I just showed you.

47. 10:59

    And it turns out experience is very hard to measure,

48. 11:03

    and I'll give you an example. So let's look at a restaurant. A restaurant, let's say, raises price, and after six months becomes much more profitable. Is it because they raised prices, or is it because they're just naturally successful?

49. 11:20

    And the challenge is, obviously, you can't run the business twice, right?

50. 11:24

    So what you do is you take two groups of similar companies, similar businesses that have the same propensity to raise prices, the same likelihood to raise prices. One group raised prices while the other didn't.

51. 11:39

    And then we look after some time at the results. So the group that raised prices actually gained forty-two hundred a day profit, and the group that didn't raise prices actually gained twenty-eight hundred dollars a day.

52. 11:53

    So the question is, what is the impact of raising prices? So a naive answer would be the difference, right? Fourteen hundred is the impact. Of raising prices. But actually, you need to account for the fact that the companies that raised prices are actually naturally more successful businesses, which is also why they could raise the prices.

53. 12:14

    So the real difference is more like eleven hundred and fifty dollars for this illustration. And we measure the impact of actions on the outcome through a measure called CATE, conditional average treatment error, which looks at that connection.

54. 12:33

    And here's what I want all the AI and finance leaders here in the room to pay attention to.

55. 12:38

    So find where you can see a lot of different situations across entities that you have in your data, in our case, it's businesses, what they did, and verify the outcomes and how things turned out if you can see that in the data 'cause that's the one thing that frontier off-the-shelf models do not have.

56. 13:01

    And don't get me wrong, frontier models are amazing, and we actually use them, and I'll, I'll show you how we use them. So we use them to generate hypothesis, candidates for actions we would suggest a business to do.

57. 13:17

    But then we would use a model that we trained using reinforcement learning in order to figure out which one of those is actually a right move to do versus a mistake that could drive the business down.

58. 13:31

    Now, how we do it, a little bit into our approach, is we look at what we call, we actually create from the data, millions of business trajectories. So we have data at Intuit across our products, QuickBooks, TurboTax, Credit Karma, Mailchimp.

59. 13:48

    So think about a business and the financial data. There's the general ledger, the P&L, the cash flow statements, the invoices, like I mentioned before. So we take all of that data, and we create what we call business states.

60. 13:59

    A state of a business is, think about a very detailed summary at a given point of time. And then we derive actions. So for example, in your general ledger, I can look and see that you have invested in a campaign, in a marketing campaign, or you paid someone.

61. 14:17

    So I know you're paying payroll, I know how much, um, um, your hiring costs are, and so on. So we derive all of these actions,

62. 14:26

    and we look at what are the outcomes in different time frames. And outcomes can be increase in profit and revenue and cash flow, in time, combination of those. So we create millions of vectors of state, action, and outcome, and then we train an RL model to be able to understand, in given situations of similar businesses, which actions

63. 14:49

    lead to the best outcomes. And then the third step is we actually train an LLM to be able to generate that better advice.

64. 15:02

    That's where opinions are going in and evidence is going out. So the model you saw in the examples at the beginning were actually a model that we developed with researchers at Intuit across millions of small and medium businesses, and we actually tested it head-to-head with all of the leading models in the world.

65. 15:22

    And we were able, with a mid-sized, cheaper model, to outperform the frontier models because of the grounding that I just showed you. And the interesting part, as a product person, you would think that it's all about the model size and the bigger and better model.

66. 15:38

    Obviously, I would have a lot better chance. But it doesn't turn out to be true. And the moat here is that it's not about the model access. It's about the data itself that you have.

67. 15:53

    And then we went ahead and built an experience out of it, and this is an AI business advisor that is currently in beta with research, in a research preview with our customers, where we use the LLM that I just described that we created to proactively raise opportunities for businesses at every given point of time.

68. 16:12

    "Here's what you should do. Here is why," grounded in who is like you, who's similar to you, what they did, and why we're actually recommending you to do it, and you can drill down into it, understand, and create action plans that will lead to your business actually growing in the right direction.

69. 16:31

    Now, I wanna take a step back and zoom out because this isn't just about money. We are entering the era of outcome-driven AI. And the question stops being which model is better

70. 16:45

    and starts becoming how can we steer AI to actually make it achieve the outcomes we want in our domains? And it doesn't matter if you're building an anti-fraud system or a healthcare system, logistics, developer tools.

71. 17:02

    The winners, in my opinion, are gonna be those with the best system of records, creating unique data sets out of them, and then training the models to achieve the outcomes.

72. 17:17

    And as the product person here, it's not just about the science. The science is very important. But a great advisor, think about the great advisors and mentors that you had in your life.

73. 17:27

    They understand you, right? They understand your preferences, what you like, what you don't like. So a great advisory experience needs to have two things. It needs to have great, grounded science, the best science, but also it needs to understand you, what you prefer, and it needs to even make you feel as if you were part of the

74. 17:49

    decision to create a trusted experience. So three things I want you to remember today. Every model has read about your domain,

75. 18:02

    but none has actually watched the plays and the moves and their outcomes, and that gap is the whole game. Now, in coding agents and coding models, we're seeing it very advanced, a lot of verified outcomes and creating models that actually lead to better outcomes in coding, but it's very much unexplored in the financial domain and in other

76. 18:26

    domains as well. And you don't close the gap with bigger models. You close the gap with experience, embedding experience into the model by looking at verified outcomes in your data, what actually worked at scale.

77. 18:44

    So off-the-shelf models don't understand money, but grounded in real outcomes, it does, and that's what we were able to figure out.

78. 18:56

    So here's the one thing I want you to do tomorrow.

79. 19:00

    Well, actually, you know what? Go ahead and enjoy Fourth of July weekend. But right after that,

80. 19:05

    look in your data where you can see situations across entities and outcomes you can verify.

81. 19:15

    Get deep into that data and start grounding your AI in that. Think about those angles. And that's for you to build.

82. 19:24

    Thank you very much. Thank you for listening to me. [audience applauding] Happy to connect, LinkedIn, Twitter, in the hallway. Thank you very much. [upbeat music]