Let's integrate AI Agents in Event-Sourced Systems

Talk transcript

1. 0:00

   [on-hold music] Hello, everyone. Thanks for joining.

2. 0:16

   So I think, like, finally, we are at the last day of the conference. Personally, I had a great experience, learned a lot of new things. So I believe by the end of the session, you would have at least few key takeaways that you could apply in your work projects.

3. 0:33

   So what is it we are gonna learn? So we are gonna learn how to integrate AI agents in your existing system. So this system, it could be an event store system, or it could be an event-driven system, or it could be any architecture that your business has invested over the last few years, right?

4. 0:52

   Because I always believe that these AI agents are m- not just for the chatbots or the coding assistants, right? So the real value that you could bring out of these AI agents is, like, when you start to apply these into your business workflows.

5. 1:07

   And that is what we are gonna learn. And the problem space that we are gonna work on today is the real-time fraud detection. So let me start with an example.

6. 1:15

   So this is my personal experience. So, uh, exactly a month ago, I purchased, uh, like, I decided to purchase this laptop that I'm using for the presentation. So it's...

7. 1:27

   It costed me around, like, three thousand five hundred dollars. So I waited for a right moment. So I was seeing like whether I have a nice offers, and there was one moment, so I decided to buy.

8. 1:39

   I pro- provided my card details, and then I clicked on buy button, then my transaction got declined. So I thought, like, it was a network issue and some information has been misplaced.

9. 1:50

   So I tried the second attempt, and it was failed again. Before I tried the third attempt, so I got a call from the customer service asking to verify, like, if I'm doing that particular transaction.

10. 2:02

    And I was like, "Yes, I, I was trying to do this for the past few minutes." And I asked them like, "Why did you block my transaction?" Do you know what the response was?

11. 2:12

    They didn't know because-- I wouldn't blame, blame them because they didn't know, like, why it was blocked. It was somewhere in the system, either the rule-based engine or the ML-based engine would have taken that decision.

12. 2:24

    So it would have looked through my transaction history, or it would have have an average threshold beyond which, like, if it goes, like, just block the transaction. That would be the static rule that it would have had, because of which my transaction got declined.

13. 2:39

    So th-those are the key areas, like, those are the uncertain areas where we are trying to integrate the AI agents. Now, you might be thinking, like, what a great idiot, right?

14. 2:50

    Because it is already an uncertain case. Like, why do you want to introduce an AI agent? Because it is also a non-deterministic by nature, right? But the key point here that we are all trying to miss is that earlier in the rule-based engine or the ML-based engine, like, we don't have enough context.

15. 3:08

    We don't have enough real-time data that gets passed on to the system. And those are the real data, like, that we are trying to capture from different bounded contexts that we have in our domain.

16. 3:19

    And we are gonna see, like, how we could build that architecture so that the AI agent can make use of it and, uh, come up with an verdict.

17. 3:30

    So the domain that we are gonna, uh, talk about is the real-time fraud detection, as I mentioned before. So we had this rule-based engine, like, um, uh, five years before, and this rule-based engine was perfectly fine.

18. 3:42

    Like, it was working perfectly fine for a few of the cases and... But, but the problem with this rule-based engine is, like, the maintainability because the fraudsters are trying to get intrude into a system, like, by a lot of different ways, and you just need to keep on updating the static rules day by day, and it, it,

19. 4:00

    it's gonna be really difficult for you to manage. And that's when, like, we started to, um, tie up with a third-party provider, like, who helped us to, uh, develop this ML model.

20. 4:10

    So we, we have this, uh, ML-based approach, like, where, uh, we shared with them a transaction history or, uh, different features with them. Like, based on that, they trained the ML model, and we were able to get a risk score based on that which...

21. 4:25

    with which, like, we were, uh, able to block or approve the transaction. But the problem with either of these approaches, like, either, um, like we, we, we were able to handle most of the transaction because it would fall below a certain threshold, then we would approve the transaction.

22. 4:44

    And if it goes beyond a certain thresh-threshold, we would be blocking those transaction. But majority of the transaction, like, few of the transaction, like, goes under the gray zone area, and this is the area where it is really uncertain for those systems to really, um, come to a conclusion whether it is an, um, fraudulent transaction or a

23. 5:04

    legitimate transaction. So what we are trying to do is, like, we, we had, uh, build a system where we had both these tier one system, which has this rule-based or the traditional ML model, and then we also had a tier two system, which is agentic AI approach.

24. 5:23

    Like, um, most of the cases would be handled re-really well by these existing system we already had because our thought process is not to exclude the systems that we already had.

25. 5:35

    We were just trying to handle w- few of the areas, like that is the gray zone areas, with the help of agentic AI processing. So this is the approach, like, uh, that we decided, "Okay, let's move on with this approach."

26. 5:47

    But then our architecture, our, um, um, domain is really complicated. So this is our different bounded context that we have internally in our domain. So the transaction context holds all the details about your transactions.

27. 6:02

    Like, it knows about the merchants It knows about the amounts that, um, that you transact, and everything related to a transaction would be residing on this particular context. And it doesn't have any information about the customer it is handling, or it doesn't have any information about the payments or the device details.

28. 6:20

    That is what the con-- uh, bounded context means, right? Because if you are from the DDD background or, um, uh, software engineering, like you would know that these are different bounded contexts, like with-- like you wouldn't share the data among themselves.

29. 6:34

    Like you need to do an asynchronous way of communication and, um, like all sort of things like that you would do in a microservice communication would necessarily been done here as well.

30. 6:45

    So then we do have accounts context, all the details about the accounts like the KYC compliant, whether the user is KYC compliant or anything about the customer, like if you want to know about the customer, this would be the right context that we need to reach out to.

31. 7:01

    And there is a device context like we had stored, uh, all the device fingerprints, browser fingerprints, uh, the OS that they are using, um, over these device contexts, like that would be really helpful like for detecting these kinds of real-time frauds.

32. 7:16

    And finally, like we had payment context, so any sort of chargebacks or, um, uh, anything related to the payments will be residing on this particular context.

33. 7:26

    So now the problem that we have with this kind of an architecture is like, uh, we, we really, we really don't have a way or means to share these, uh, con-- uh, datas across different, um, uh, bounded contexts, right?

34. 7:40

    So that's where like we started to introduce an orchestrator layer. So earlier, like we had this orchestrator layer, but now we also have an agentic AI inside this orchestrator.

35. 7:51

    So what this orchestrator does is like similar to, um, a saga orchestration. So all the communication would go through this layer, and then it would be, um, communicated to other services, like who are interested in those events.

36. 8:05

    And that is how like we define this, uh, orchestration layer. And we also have an asynchronous way of communicating with-within other different contexts, like through a message broker.

37. 8:17

    If I zoom in a little bit on the transaction context, you could see what, what are the details that these transactions are holding. So, um, essentially like what happens is like when you have, um, a particular do-domain, you will be having different events that are emitted from that system.

38. 8:35

    So those are really called as domain events. And within our transaction context, like we do have transaction created domain event, which is the start of entire transaction, and we do have a lot of integration events.

39. 8:47

    So these are the events that comes from other contexts like payment context, uh, device context, or account context. So events like transaction rejected, payment approved, payment rejected would come from other systems to our, um, uh, transaction context.

40. 9:03

    And these entire informations are stored in our NoSQL database. Uh, we are currently using Cosmos DB as our event store. And we are following event sourcing as our methodology, uh, to store the events.

41. 9:16

    So what happens is like whenever a user initiate a command, so that goes into our event store as an event, as a business fact. So we are not mutating the state, but instead like we are appending all the events as when it arrives.

42. 9:30

    And what it really helped us to do is like, uh, once it arrives into the event store... So, so we have a kind of a mechanism called CDC or in NoSQL world this is called as a changefeed, with which like whenever there is an change or update happens over a table, so you will be getting notified, and

43. 9:49

    those changes could be propagated into different read models. So because in, in, in a real scenario, like you won't be able to rely entirely upon the event store for the query operation, for the read operation, what you would do is like in turn like you will be having different read models which are optimized for the read operations.

44. 10:09

    So what we really had is like we had multiple read models, so one for timelines, one for customer information, and one for the fraud indi-- uh, indicators, that is PU.

45. 10:20

    So these are the different, uh, read model layer, layers that we had within our transaction context. And essentially like you can't, um, you can't, you can't say other teams to follow the same patterns because event sourcing is not the one that, uh, other teams are also following.

46. 10:36

    So what we did is like we also had this asynchronous way of communication by, uh, emitting those events into the message broker and those, uh, those events in turn will be processed by a worker process and then it will be reaching out to our projection layer.

47. 10:51

    So the idea is like we need to gather all the datas from all these different contexts and to have or build a semantic layer, or you could call it as a materialized view, which you could further use w-within your agentic AI flow.

48. 11:10

    So this is how the high level flow looks like. Like we have all these different contexts, and you could either use a CDC mechanism or a message broker to propagate those events into your projection layer.

49. 11:22

    And essentially like you will be having this intermediate layer like which is an worker process. You need to massage your message events and then process it and then store them in the materialized view.

50. 11:34

    So this, this is the entire high level picture of how you could gather those datas and form a materialized view.

51. 11:41

    Now coming to the agents. So, so we all know like agents are comprised of these three main components, the language models, the tools, and the memories, right? So this language model, it, it is not necessary to be a la-large language model.

52. 11:55

    It could be S-SLM, it could be an open source model. And the tools are by which like you could interact with the external APAs or the methods, methods that you define in-inside your, uh, application layer.

53. 12:07

    And you also need to have a memory layer. For this particular use case, like we are using an inch, um, short memory because, uh, you, you can't really, um, rely on the long-term memory because you need to, um, adhere to the, uh, SLA that you, uh, provided to the customers because For, for the transaction, uh, to be

54. 12:27

    processed, like, it should be sub five hundred milliseconds. And y- y- we are currently using in-memory for this. And essentially, like, what happens is, like, whenever there is an query comes into our language model, so it has this reasoning capability, the thinking capability with which, like, it tries to decompose the task into multiple chunks.

55. 12:46

    And each of these tasks in turn will, uh, go into the language model, do cert- certain processing with the help of the tools, and it will see, like, if the end goal is reached.

56. 12:57

    And if it is not reached, like, it will try to go on in this loop. So this is what essentially happens inside the, uh, agentic framework.

57. 13:06

    But you should really know, like, when to stop this loop, uh, because, uh, th- this difference, uh, this, this, this might be varying, this might be differing for different use cases.

58. 13:17

    For, for our use case, like, we do have a metrics w- beyond which, like, if we go, like, we, we could break out of this loop, and this could be varying for different use cases.

59. 13:26

    So you should be really careful on avoiding this infinite loop. And now coming back to our, uh, existing architecture. So what we did is, like, earlier, like, we learned about the transaction aggregate.

60. 13:40

    So now we are emitting some events that gets translated into an integration event, which gets passed on to the message broker, and then it, it is handled under the orchestration layer.

61. 13:52

    So under this orchestration layer, like, we have different sub-orchestrator. So we segregated the tier one. Earlier, like, we just had the tier one. Now we have this tier one layer, which is handling the, um, rule-based engine or the ML-based engine.

62. 14:07

    And we also have now the tier two layer, which is gonna be agentic AI processing. So for this agentic AI processing, like, we used fan-out pattern. So we kind of used multiple agents within this layer, and we are trying to use a fan-out pattern.

63. 14:22

    Like, uh, once the, uh, event has been reached out to the a, uh, tier two layer, we will be fanning out this, uh, event to two different agents. One is the risk analyzer agent, the other one is the behavior analyzer agent.

64. 14:38

    And once these agents, like, come to an conclusion based on the different tools that it has, it will finally send the response to the verdict, and this verdict could be a metric, it could be just an if condition inside your application layer, or it could be an- another agent.

65. 14:56

    Because what we've seen is, like, if we are using just the metrics, it is again going back to the same criteria, like, where we had this rule-based, um, mechanism.

66. 15:06

    Uh, so there are many false positive cases that we are, uh, that we faced. So, so we, we in turn, like, we are trying to use a third agent in this verdict layer which analyzes both the agents' responses and come to a conclusion, which is then gonna be emitted as an event back to this message broker.

67. 15:25

    So this is how our saga flow continues, and it gets into the payment context, and then the payments are approved, and then it is going back to the transaction context.

68. 15:35

    This entire loop is being done within our orchestration layer.

69. 15:41

    And this, this is what I was talking about two agents that we had in this tier two. So one is the risk analysis agent. Like, what are the tools that we have in this, um, agent is that, like, we do have this, uh, kind of, uh, storing the fraud histories inside the semantic layer that we earlier see,

70. 15:59

    and we are currently having a tool for that. Like, um, it will just get those details from these projections layer. And we also have a device trust layer. Like, all the device information will get stored into this semantic layer, and it will just get those chunks alone.

71. 16:16

    And we also had, um, business rules trying to migrate some of the rules that we had in the rule-based engine over these tools, and these are really specific to our business use case, so we are trying to move those into these tools.

72. 16:30

    And we also have a be- behavior analysis tool. So, so there, like, we are analyzing the transaction patterns with two different plugins that we have. So based on these two different agents, the responses, like, we get to a final consensus, and then that will be published as an event, which will be captured by the message broker.

73. 16:50

    And this will be listened or, uh, this will be subscribed by all the different contexts which are interested in those events.

74. 16:58

    Now, as, as, as we seen earlier, like, how does these g- um, agents gather context? So

75. 17:06

    from the projection layer. So these orchestration layer is gonna consume these i- um, datas from the projection layers, and how it is gonna do is by the help of tools that it already has.

76. 17:18

    So the transaction context, it is gonna denormalize some of the counts, averages, average amounts that we had. The recent transactions all gets into this, um, semantic layer. And the device context is gonna send all the details about the device trust score, location histories, and all other, um, related information alo- um, about the locations, IP addresses to this

77. 17:42

    semantic layer. And the account context is gonna send the statuses of the accounts, the KYC status, account age, um, whether the customer has been, uh, with us for the past few years.

78. 17:54

    Uh, so based on that, like, uh, it will try to validate the, uh, leg- um, uh, the, the customer based on those informations. And there is also the payment context from this, like, we get to know about the recent, uh, payments that we, um, gathered over these contacts into the semantic layer.

79. 18:14

    And, and as I, as I said earlier, so these tools get, uh, access to these projection layer, and which in turn will be used by these agents to come to an conclusion.

80. 18:26

    And this is the high level, um, a-architecture that we had. So it could be any event source that you could use, and basically, like you could have a relational database or NoSQL database.

81. 18:38

    In turn, like you, you need to just create the semantic layer for you to, uh, provide enough context to these AI agents. And we do have this agentic layer in the orchestration, so we have this verdict tool, and then the short-term memory.

82. 18:52

    And based on that, like whatever the result that we get, gets on passed to the saga orchestration layer.

83. 19:00

    So we-- I, I, I have prepared a POC, uh, so based on the synthetic data. So I will just show you like how this works, uh, in, in, in...

84. 19:11

    Let me run this one. So these are the different contexts that we have seen earlier, so the transaction context, accounts, and device context. So from these, like we will get into the projections layer, and we will be having a CDC mechanism with which like you will be able to progress-- propagate those events back

85. 19:36

    to your semantic layer, which will get in turn used by these AI agents.

86. 19:44

    So here you could see, so there are multiple simulation events that we are pr, um, simul-simulating into the-- our systems. And you could see bunch of events are currently populating.

87. 19:56

    So these, um, events are propagating through different layers, the transaction layer, accounts, and then payments context.

88. 20:05

    And finally, like we also have this, uh, AI agent layer, like which, which is gonna try with the tier one processing, uh, the ML processing.

89. 20:24

    Let me see. I think it is just the database which is in the serverless mode. Um, I think it is coming up slowly.

90. 20:40

    So but, but yeah. The idea is like you will be able to see like two different agents coming up with two different, uh, conclusions, and you'll be having a third agent which will take the final decision.

91. 20:51

    Uh, that will be in turn emitted as an event back to your message broker. And this saga flow will be continuing as for, uh, like how you design your business architecture.

92. 21:04

    I'm going back to my presentation. Yeah, that's it. Uh, if you have any questions, like you can reach out to me, uh, offline, or you can reach out to me on LinkedIn.

93. 21:16

    Thank you. [clapping] [upbeat music]