"Data readiness" is a Myth: Reliable AI with an Agentic Semantic Layer — Anushrut Gupta, PromptQL

Talk transcript

1. 0:00

   [on hold music] Hey, folks.

2. 0:16

   I am Anushrut. I lead the applied research team here at PromptQL. Uh, PromptQL, you might have seen is the sponsor for the reliability track here, uh, at the AI Engineers World's Fair.

3. 0:29

   Uh, so today I'll talk about, um, that data readiness is a myth. How many of you are trying to deploy some kind of AI system on some kind of data in a production environment?

4. 0:43

   Okay, awesome. Who would-- Who is trying to work with, uh, more than like documents and vector databases? Okay. Is-- Whose data is perfect? Clean, annotated, perfect column names, table names.

5. 0:58

   Anyone? See no hands. Okay. So, and how much time does everyone like-- Okay, I'm not gonna ask make this a question. You all spend a lot of time, uh, making your data ready, right?

6. 1:09

   So that the AI can understand it, so that the AI understands the meanings and the relationships in your data. But it's just a pipe dream that we are all chasing, right?

7. 1:19

   We are all chasing this perfect data dream so that our AI can finally work reliably on it, and that's never gonna happen. So how do we still make AI re-reliable on no matter how messy data we have?

8. 1:30

   Okay. So tell me if this is a fact. Is this what your data looks like? Your, uh, that's how you name your tables. That's what how you name your columns.

9. 1:40

   Sometimes they have null values. Sometimes they have all values, all column names. Sometimes you have shorthand, CST_NM. Is that customer name? Is it custom nomenclature? I don't know what that is.

10. 1:52

    Uh, rev_amount_USD, that's how you name your revenue. Is active. Now, is this, uh, binary? Is it Boolean field? Is it zero, one, true, false, null, not null? I don't know what it is, right?

11. 2:05

    Then you have other system. That's just one system, one table in one system. You have other system which have similar things, right? It has organization name, it has total revenue.

12. 2:13

    How does that map to your other systems, right? Is the revenue in cents? Is it in dollars? Decimal value? Floating points? What is it, right? You have no idea.

13. 2:22

    Okay. So in 2019, everyone was saying, "Let's standardize everything. Move everything in Snowflake. Let's move everything in Databricks, and finally our problems will be fixed." Forty percent complete. Uh, MDM will fix this.

14. 2:35

    Master data management team will fix this. That's their responsibility. They're still implementing their solution. So in 2023, with the rise of AI, rise of agents, we'll, we'll create semantic layers that understand our, uh, data domain.

15. 2:48

    I mean, it breaks every quarter. Your data domain changes every quarter, right? You change your tables, you change your schemas, you change your, uh, uh, workflows. So in 2025, we are saying AI needs perfect data to work, and it's still waiting, and it's never gonna happen, right?

16. 3:03

    And McKinsey, uh, said that on an average, a Fortune Five Hundred company loses two hundred and fifty million dollars because of poor data quality.

17. 3:11

    So how do we fix this? Uh, who has tried playing with semantic layers? Memzero, Atlan's Semantic Kernel, you have tried playing with it, right? How has your experience been?

18. 3:23

    We actually use Django- Okay. -to create a kind of flexible model. Okay. Gotcha. Um, yeah, it needs a lot of

19. 3:31

    pruning. A lot of pruning. Okay. And I'm assuming you're manually adding information to it, maintaining it, and stuff like that, right? We do it automatically, but- Sure. Yeah. Uh, okay.

20. 3:41

    So let's say you've added a definition, like customer acquisition cost means marketing spend divided by new customers. Okay? Now, this is some information your AI needs to answer your questions.

21. 3:53

    But that's not enough, right? Like what-- Which marketing spend? Coming from the brand team? From the performance team? What does a new customer even mean, right? First purchaser, uh, purchase, uh, customer, reactivated customer.

22. 4:04

    For what time period are we talking about? Does it include failed trials or not? Uh, does it-- Is it accounting for seasonality? There's so many things that you need to do, and you can never capture all of that in a semantic layer just by...

23. 4:16

    If you think you can manually add everything, you can't, right? So you can't predefine, uh, every edge case. Knowledge graphs. Who's played with knowledge graphs? GraphRAG? Heard of it at least?

24. 4:28

    Okay. A bunch of people. Okay, awesome. So, uh, let's take a very simple example. Assume a customer's, uh, uh, assume a sales data set, uh, where you have defined this graph that deals map to stage, a date, and an owner.

25. 4:43

    Okay? And a very simple question I ask, show me deals at risk, right? Very simple questions. The graph knows that deals map to stages, stages map to close dates.

26. 4:52

    But what does at risk mean? Is it mean that the champion has just left? Is it mean it has been stuck in that stage for two months? Like what does at risk mean to my business, right?

27. 5:03

    How do you capture that in a graph database, right? How do you capture a billion rows of Snowflake da-table in a, uh, graph database? You can't. So graph is also not the solution.

28. 5:15

    Knowledge graph is also not the solution. So the real problem here is not that we need we, we need a better semantic layer solution, we need a better GraphRAG solution, we need a, like, better, uh, named database systems.

29. 5:28

    No. The problem is that the AI does not speak your business la- business' language. Like a GM in a finance domain can mean gross margin, but in a HR domain might mean general manager, right?

30. 5:41

    What does conversion mean to you? What does quarter mean to you? What is the definition of your quarter? I'll show you an example, uh, here with an AI system not working with that.

31. 5:48

    And, uh, what is, what is, what is an active customer? Every team has their own definition, right? So how does an AI speak this tribal knowledge, this tacit knowledge that you have developed while being in your company for so many years, right?

32. 6:00

    Your AI does not know that. Your vanilla LLMs don't know that. They're super smart, incredible at doing so many g- cool stuff, uh, so much cool stuff, but they don't understand your business, your domain, right?

33. 6:11

    So traditionally, we had these analysts, these engineers, right? Whenever a, a business user or a customer had a problem, had a task, they had a question, um, they would go to this analyst or an engineer who knew about the business, who had this tribal knowledge in their head.

34. 6:26

    They knew how to write code, SQL, whatever. They can talk to your underlying data systems. SQL, NoSQL, doesn't matter, any kind of data source, right? And they have this tribal knowledge which they use to answer your question with a hundred percent reliability.

35. 6:40

    They explain what they are doing, right? And that's how you have all built trust in your colleagues, in your, uh, peers, right? That's what is missing with AI, right?

36. 6:49

    This, this tribal knowledge piece that doesn't exist today with AI, and that's the problem. So the solution. The same semantic label, let's make it agentic. What that means is, let's not try to improve it.

37. 7:02

    Let's not try to, uh, manually add context to there continuously. No. How about we make an AI system that behaves like the analyst you just hired today, day one, day zero.

38. 7:15

    The analyst comes to your company, super smart, can do a bunch of things, doesn't know a lot about your business yet. They start working with you. They mess up somewhere, you tell them, "No, no, that's not what you should have done.

39. 7:24

    This is what I mean when I say this." It learns, learns, learns, learns. Now, this analyst, ten years later, is an experienced analyst in your company. They know everything about your business, right?

40. 7:33

    Let's make an AI like that. An AI that keeps improving, keeps learning as you use it more and more, as you course-correct it, as you steer it. But assumption is your AI needs to be correctable, explainable, steerable, already accurate in what it knows, right?

41. 7:49

    So let's see how you build such an AI, right? So we are trying to replace this human part of the AI, right?

42. 7:56

    So that's what we have been trying to do with PromptQL. It's like a day zero smart analyst, right? So we take a foundational LLM, and that's the-- whatever LLM you bring.

43. 8:06

    We make it create PromptQL plans. PromptQL is basically a domain-specific language which can do three tasks: data retrieval, data compute aggregation, and semantics. And this is a deterministic domain language.

44. 8:23

    And vanilla LLMs are incredible at generating. We, um, don't have to fine-tune them, right? Now, within this DSL, I can ask the LLM to create this DSL whenever I ask-- user asks a question.

45. 8:34

    Now, I can execute this DSL in a deterministic runtime. I do not involve the LLM in actual execution, actual generation of an answer, because if I let the LLM generate the answer, it's by default hallucinating, and I'm just hoping the hallucination is correct.

46. 8:48

    That's how LLMs work, right? So I'm saying decouple it. Let the LLM generate the plan, and we will execute this plan in a deterministic runtime, and let it work on a distributed query engine, which will, uh, talk to the different data sources, pull out data, do whatever composition is required inside that DSL, show the answer directly to

47. 9:06

    the user. Don't give it back to the LLM. Let's not do RAG, right? Let's not give the LLM, uh, data back to the LLM and l-- make it generate the answer, right?

48. 9:13

    That's what, uh, the PromptQL design is. Let me show you PromptQL working in action. Uh, I have five minutes, so let's make this quick. Simple question, uh, who are my top five customers by revenue?

49. 9:25

    You'll be like, "Any AI system can answer that question, dude." It's a simple text-to-SQL question. Okay. So, uh, PromptQL is like, first of all, it understands what revenue means.

50. 9:35

    Revenue means your invoice items. Okay, cool. Uh, I'll do, uh, do the math, execute the math, and here are your top five customers. Oops, nope. And you notice we did not get any results, right?

51. 9:47

    That's what a smart analyst does. In their first attempt, they realize they messed up somewhere. Okay, I see the issue now. We're looking for succeeded status, but the actual statuses are paid and pending.

52. 9:56

    See, your data is messy. It did not know what was happening, right? It figured that out, and now these are your top five based on the actual data, uh, that's under the hood.

53. 10:04

    So that's possible. Let's run this query now. Okay. Find the unique customers we serve. The cust-- org ID data is messed up, so don't use that. Find unique orgs based on the email domains of the individual users.

54. 10:15

    Then find the org with the third highest revenue. I'll, I'll let it run because, uh, it'll take time. Uh, find the unique orgs based on the email domains of the individual users.

55. 10:23

    Then for the org with the third highest revenue, take a look at the latest thirty support tickets. So, um, multiple database, then your Zendesk support, uh, system, uh, including the comments on those tickets, then summarize each ticket.

56. 10:36

    Then use those summaries and extract their feelings towards our product. Create five categories from bad to great, and then tell me what is going well, what, what can be improved.

57. 10:44

    And then issue up to five thousand dollars to this org pre-- uh, org's project as like credits with the highest usage. Uh, which means five thousand if their feeling is bad towards us and one thousand if it's great towards us.

58. 10:57

    Now, do you think an AI can do this? Spread across your databases, your, uh, SaaS application like Zendesk, your, uh, API, uh, like Stripe to issue the credit, stuff like that.

59. 11:08

    Say, "Oh, cool. First I'll get all the users, extract their email domains." See, this is an analyst explaining their thought process. You see that tiny, uh, pencil icon? I can edit their brain.

60. 11:18

    I can tell them, "No, this is not, uh, what I wanted you to do. Don't do step three. Instead of that, do these three steps instead." Right? I can, I can be in charge of my AI, but still every single time if I have to nudge my AI, the AI is going to learn.

61. 11:32

    The AI will understand, "Okay, that's what you wanted me to do. Makes sense. That's how you do your business. I didn't know that, sorry. I'll learn now." Right? So okay, cool.

62. 11:42

    I should have said, "Can you just show me in, uh, intermediate results step by step?" But anyway, um, it says I'm about to issue three thousand dollars refund, probably got a neutral sentiment from it.

63. 11:52

    Um, and see, it's exactly what's happening. It's saying, "I got the top five domains by revenue. Uh, this is the sentiment. I summarized a bunch of tickets. I classified, extracted the sentiment out of that," and then finally, um, uh, figuring out what is happening there.

64. 12:05

    Um, okay. So prepare to show details. Perfect. Based on analysis, uh, Peters thompson.business third highest revenue customer with this much in revenue, sentiment analysis, recent support tickets, blah, blah, blah.

65. 12:17

    See, this AI just worked as an analyst, as an analyst with such a complicated prompt. So now let's look back at the learning, right? Uh, just because we are at two and a half minutes.

66. 12:28

    Okay, so that's day zero. Day zero, it had to figure stuff out, figure it out, did, did well. Day X, once it's become a veteran analyst, right? It has learned a bunch.

67. 12:38

    Let me make this bigger for you. Uh, learned a bunch, right? So as it's using-- uh, as you're working with it, right, it learns from it. There's a PromptQL learning layer, which basically improves the semantic graph and starts creating your company's business language.

68. 12:52

    This AcmeQL, Acme, assume it's a name of a company, right? And now suddenly PromptQL becomes AcmeQL. Be-becomes GoogleQL, MicrosoftQL, AppleQL, CiscoQL, whatever company you come from, right? And I'll show you that learning process in action.

69. 13:06

    Okay? So this is an example, uh, where I have purposefully named my tables extremely bad, right? And I ask the question, which employees are working in departments with more than US, uh, dollars, ten thousand dollars budget, okay?

70. 13:18

    It says, "I have no idea what you're talking about. Your data says there are three tables called morg, plug, and zorp. I have no idea what that means. I can't answer your question."

71. 13:26

    I'm like, "No worries. Can you sample a few rows from each table and figure out what table con-contains the, the employees?" Like, cool. That's what I tell man. Let's go figure it out, right?

72. 13:36

    It's like, okay, now I see zorp contain the employee information, plug contains, uh, department information, and morg is like a junction table that you have. Okay? So now this is your answer that you were as-asking for.

73. 13:46

    And I'm like, "Okay, but the data is in cents, not in dollars. The budget is in cents. So can you divide by hundred, please, and give me the right answer now?"

74. 13:54

    There shouldn't be five, uh, employees. It says, "Cool, there are two employees." Like, perfect. Now, this is, um, uh, this is manual, but this also runs agentically in the background, where all I have to do is suggest metadata improvements based on the recent trends, right?

75. 14:10

    Look at what have we spoken about, how much I had to guide you, whatever, uh, hints I had to give you. Learn from it and improve your semantic layer, right?

76. 14:19

    It's like, cool. So based on the, uh, interaction that we have, now I have this, um, improved semantic layer where it's like, okay, zorp and plug are two tables.

77. 14:29

    I need to add a lot more context to it in my own, uh, semantic layer, right? And the department budget is in cents. Like, cool. Apply the suggestion. Every single instance of your semantic layer is version controlled, so a new build is created.

78. 14:42

    You can always fall back to a previous build. And now the next time as generated by AutoGraph, that's what you call the feature, and the next time I ask the same question, which employees are working in departments with more than US ten thousand dollars in budget?

79. 14:53

    It creates the right plan, and I get the right answer,

80. 14:57

    right? So same, if I had to say something like this, uh, find accounts, um, with the maximum suspicious anti-money laundering, um, out-outgoing amounts for the first quarter. For each, uh, print the account ID and name, right?

81. 15:12

    If I let my AI do this... Okay, the internet is a little bad. But, um, I have this thread preloaded here. Okay. So it gives me the answer, and then I'm like, "No, the acc-- uh, my quarter starts in February, not in January.

82. 15:26

    So you should know that," right? So I just tell it this time. Next time, the semantic layer learns from it exactly the same way as it'll infer the meanings of these tables.

83. 15:34

    It'll find the relationships across your tables, right? And finally, with all of this, what you have is

84. 15:45

    day zero. Your AI does not know what an en-enterprise customer means. It does not know how to match customer IDs across systems. It does not know what your financial quar-- when your financial quarter starts.

85. 15:55

    Day thirty, it has figured out forty-seven business terms. It has mapped relationships across the six systems. It has discovered twelve calculation variants and hundred percent accurate on your complex tasks.

86. 16:05

    That's what an agentic semantic layer allows you to do. So it reduces, uh, months of work into immediate start. Like, just deploy your AI today. Let your AI start working on your data.

87. 16:16

    Let it improve itself. No wait time, no lag time with your AI deployments, right? It's self-improving and gets to a hundred percent accuracy. Uh, that's what we've been hearing from our customers.

88. 16:26

    It's a Fortune Five Hundred food chain company. They evaluate a hundred ven-- uh, vendors. They realize, no, none of them work. Finally, they saw PromptQL, and it worked a hundred percent reliable AI.

89. 16:36

    Same with a high, high growth fintech company. The-on their hardest questions, we were able to demonstrate a hundred percent accuracy. So try it. Pr-- um, reach out to us if you have these big problems that you want to solve and you want a hundred percent accurate AI on top of that.

90. 16:51

    Uh, we are there for you. Thank you. [upbeat music]