Machine transcription with reviewed terminology and names. Some recognition errors may remain. Timestamps refer to the original livestream.
4:50:25Hello, I'm Karthic, CTO at Perpetual.
4:50:28In this presentation, I'll talk about building workflows that are both reliable and intelligent for finance, wealth and insurance.
4:50:35In regulated domains, workflows must be reliable enough to meet regulatory standards while still being intelligent enough to handle the real world.
4:50:43In fields such as finance and wealth management, this presents both a technical and a compliance issue.
4:50:48Reliability and intelligence are not opposites.
4:50:51It is up to you where you draw the boundary between them.
4:50:54And that is the point of this talk, deciding where that boundary lies.
4:50:57For us, it's not merely a thought experiment, but a question that surfaces serious consequences.
4:51:03Perpetual is a wealth management roll-up.
4:51:05We acquire running traditional wealth management firms.
4:51:08And we maximize their throughput and revenue with the help of AI.
4:51:13Wealth management is a regulated industry.
4:51:15And that is where we build with AI in production.
4:51:18Everything in this talk is grounded in building for those constraints.
4:51:23Workflows in finance and wealth management come with four facts,
4:51:26and each of them pushes the architecture in the same direction.
4:51:30First, they're complex and they need runtime strategies as the process unfolds.
4:51:34You can't script every path in advance.
4:51:37Second, the data is sensitive, so you need a path to move these workflows entirely onto privately hosted models.
4:51:43Third, in a highly regulated industry, a policy violation isn't a bug report.
4:51:48It has severe consequences.
4:51:50So, fourth, execution has to be highly reliable and deterministic.
4:51:55Inject intelligence only where runtime judgment is genuinely required
4:51:59and minimize reliance on LLMs everywhere else.
4:52:04AI is spreading through regulated industries,
4:52:07but there's still no consensus on an architecture
4:52:09that gives you high reliability with the least use of LLM intelligence.
4:52:13Meanwhile, most of what developers hear from the big labs is how to burn more tokens.
4:52:18But in a regulated industry, token maxing is the wrong goal.
4:52:22These tasks are close to main.
4:52:24You can lay them out in advance and build them into a massive state machine.
4:52:28And that machine needs only a sprinkle of intelligence to navigate it.
4:52:31Most steps just follow the next edge.
4:52:34Only at a few judgment points does the model pick a branch.
4:52:38So the intelligent way to build is to use intelligence minimally and on purpose.
4:52:45Let's make this concrete with a real wealth management workflow.
4:52:49A wealth advisor turns to their assistant and says one sentence.
4:52:53Fill out the IRS W-9 for Mr. Harper to a person that's a simple instruction.
4:52:58Watch what it actually triggers when a human being executes it.
4:53:02The task has complexities that go beyond what meets the eye.
4:53:06First, the assistant finds form W-9 from the firm's records or from the IRS website.
4:53:11Then they look for Mr Harper in the system of record and find
4:53:14three Harper households. So they call the advisor. Which Harper
4:53:18do you mean? Only once that ambiguity is resolved do they move
4:53:21on to the form. A lookup, an ambiguity and a human in the loop
4:53:25all in the first minute of a one line task.
4:53:34Filling a form sounds like data entry. It isn't. Every form is a
4:53:38small program. A W9 first decides who the taxpayer is, the person or the owner if it's a disregarded
4:53:46entity. A foreign person stops right here because they need a W8 instead. Then how they're taxed?
4:53:53Exactly one box on line 3A and an LLC adds its letter. That decides which name goes on line 1
4:54:00and which on line 2. Then whose number? An SSN or an EIN. Never both. Taken from a verified document.
4:54:08never from memory. Exemption codes apply only to some entities and never to an S-corporation.
4:54:14And finally, who signs under penalties of perjury? Each answer changes the next question
4:54:19and the people involved only ever see three kinds of gates, a question, a document to collect, or
4:54:25a signature. Now they start filling the form and realize the system of record is missing three of
4:54:35the values it needs. They check the documents on file and the system of record again still
4:54:41missing so they start an email thread with the client asking for exactly those three
4:54:45values. The task now depends on someone outside the firm on their timeline.
4:54:55now the workflow waits. The request is with the client and all three values are still
4:55:00missing. If the client doesn't reply within two days, the assistant follows up and brings
4:55:05it to the advisor at the daily stand up. If the client replies with only two of the three values,
4:55:09they follow up again for the missing one. When all three arrive, they complete the form and write
4:55:14the new information back to the system of record, including the trail of how they obtained those
4:55:19values. In this industry, how you know a value matters as much as the value itself.
4:55:33Once the form is filled, it goes out for e-signature.
4:55:36If the client doesn't sign within a day or two, the assistant nudges them again.
4:55:41Once it's signed, they file it for the client, send the acknowledgement and update the advisor
4:55:46at the next day's stand-up.
4:55:47One sentence from the advisor became a process that runs for days, crosses several systems,
4:55:52weights on two people and loops back on itself more than once.
4:56:10Look at what's inside that one sentence. Ambiguity to resolve
4:56:16data together. Approvals, weights measured in days, follow-up loops and an
4:56:22evidence trail at the end.
4:56:29Workflow automation tools exist, but they rarely handle this real-world complexity and
4:56:34ambiguity.
4:56:35A fixed flowchart breaks at the first witch harbour and the lack of a native system of
4:56:40record with an integration and an agent that improvises its way through is exactly what
4:56:45a compliance team can't sign off on.
4:56:47One breaks on ambiguity, the other can't be signed off.
4:56:50neither handles what's inside that sentence. So what do we do about it? The fashionable framing
4:56:56is more agency and better models. Given agent the whole task and wait for the next model to
4:57:03make it reliable. The reflexive answer, especially inside regulated firms, is the opposite. Less
4:57:09agency, more rules, lock it down and hard code every path. Both are the wrong axis.
4:57:16In regulated verticals, reliability comes from where you place the boundary between the two,
4:57:22between the parts that must be deterministic and the parts that genuinely need judgment.
4:57:27Here's the principle.
4:57:29Determinism governs control flow, evidence, and side effects.
4:57:33What runs next, what we know, and how we know it, and anything that changes the world,
4:57:38like a record written, an email sent, or a form filed.
4:57:47Intelligence provides interpretation and judgment. Reading an ambiguous instruction, deciding which Harper, understanding a client's reply. The model answers a typed question and the deterministic system decides what to do with the answer and the interplay is recursive. A judgment can open a new smaller plan that's compiled, gated and deterministic again. You'll see that in a moment with follow ups. The boundary between the two is the design.
4:58:15Here's how that works in the workflow engine we've built.
4:58:26The request goes to a planner. That is a model call and its
4:58:31only job is translation. It emits a typed semantic graph of
4:58:35intents, entities, facts, artifacts, events, and
4:58:39constraints, each grounded in the words of the request with any
4:58:42any ambiguities it found. Every node points back to the words
4:58:46it came from. That graph goes to a compiler that never calls a
4:58:50model. In 11 deterministic stages, it turns the graph into
4:58:53an execution tag or it fails closed. There are only three
4:58:58verdicts. Valid needs clarification or refused.
4:59:04Which Harper comes back as a question, not a guess. The plan is
4:59:08is compiled, immutable and pinned by its hash.
4:59:21The model translates. The plan is compiled. The compiler checks
4:59:26the plan against the grammar. The vocabulary of what the firm
4:59:30is allowed to do. 42 typed action contracts across forms,
4:59:34documents, email, e-signature, calendar and human steps.
4:59:39Each action the planner asks for has to match a contract. Here
4:59:44form fill finds its forms contract and locks in. If the
4:59:48planner asks for an action that isn't in the grammar or wires
4:59:51one up wrong, the compiler refuses.
4:59:54There are about 40 ways to fail closed from a cycle in the graph
4:59:58to an unsupported action
5:00:00And the same contract digest is pinned in the engine, the backend and the UI. So all three agree on what an action means. That is what a closed domain buys you. The model can only ask for things the firm already knows how to do safely. Here is a compiled plan for a W9 chase. Send the client the form, wait for it to come back, check it and file it.
5:00:21Before anything runs, a person approves the plan itself.
5:00:26Find the form, prepare it for the client to fill, render the PDF and draft the email.
5:00:31Drafting is a model call, so it is violet.
5:00:38Sending is an action gate. Nothing leaves the firm until a person approves that exact email.
5:00:44Then the plan waits for a reply and a judge steps in to read it.
5:00:49The returned form is verified. If it matches, it's archived behind another gate. If it doesn't,
5:00:56the plan follows up. Now look at the dashed steps. Rendering the PDF, attaching it, judging the reply.
5:01:05The planner never asked for them. The grammar added them because the contracts require them.
5:01:11In the story, waiting, approvals, and follow-ups were most of the work.
5:01:15Here there are nodes in the plan, not prompts. A reply wait defaults to 72 hours and an explicit
5:01:23deadline can be anything from a minute to 30 days. When it passes, a rule decides what happens,
5:01:30not a model. One approval covers the plan and every external action like sending an email or
5:01:36a signature request has its own gate. The approval is tied to a hash of the exact payload and it's
5:01:42checked again right before sending. When a client replies with only part of what we asked,
5:01:48the reply judge marks it partial. That judgment is provisional. The step stays blocked until a
5:01:54person confirms it. A person confirms it and the step unblocks. Then the follow-up runs as a new
5:02:00child plan. The approved parent is never changed and there are at most five rounds.
5:02:11Once a plan is approved, a state machine runs it. It picks every step and records every result.
5:02:18When a step needs judgment, it makes one fixed schema model call. Compose an email, judge or
5:02:24reply, spot missing values, extract fields, verify a document. The answer comes back typed,
5:02:31and the state machine decides what to do with it. Side effects get a life cycle of their own.
5:02:36Prepare, dispatching, succeeded, or when the world doesn't answer cleanly, outcome unknown,
5:02:42and manual review. There are no automatic retries. A chat message can approve a step
5:02:48or answer a question. It can't change an already approved plan. The model steers. It never drives.
5:03:01Now look at what is left for the model to do. Outside the planner, the judgments are
5:03:05small and typed. Is this reply complete? Does this document match? Which field is which?
5:03:11They don't need frontier intelligence. A fast small model, the flash tier, can handle
5:03:17them. In this illustrative run of 20 decisions, 19 go to the flash tier and one goes to the
5:03:23frontier. That is why we designed for about 95% of model decisions in a workflow never
5:03:30never needing a frontier model and the flash tier can run on
5:03:33hardware you control. Only the genuinely hard cases go to a
5:03:37frontier model.
5:03:40the 5% is not a permanent tax because every judgment is
5:03:45typed and logged and the provisional ones are confirmed
5:03:48or rejected by a person. Each frontier decision produces
5:03:51eval grade distribution real training data by construction.
5:03:55That data can post train the flash tier. Logging and evals
5:04:01exist today. The training loop is next. So the frontier share
5:04:05shrinks release over release until even the hard cases run
5:04:09locally.
5:04:16Remember the requirement we started with because data is
5:04:20sensitive. You need a path to move workflows entirely to
5:04:24privately hosted LLMs. So can this design really get there? Take
5:04:28one type call from the Harper Burbinine plan. Every model call
5:04:33is small, typed and checked against a schema. So pointing it
5:04:37at a privately hosted model is a configuration change, not a
5:04:41rewrite. One line changes and the same answer comes back. Every
5:04:45other call has that same small type shape. So the same change
5:04:49applies to almost all of them.
5:04:51That is how about 95% local inference becomes viable as an architectural consequence, not a policy promise.
5:04:59That is the path that sensitive data demands and compliance risk goes down as an engineering roadmap, release by release.
5:05:07So if you remember one thing, don't ask whether your system should have more agency or less.
5:05:14Ask where the boundary is.
5:05:15Let determinism govern control flow, evidence and side effects.
5:05:20Let intelligence handle interpretation and judgment.
5:05:24At typed points, you can inspect and use every judgment you log to need less of it next time.
5:05:30Ask where the boundary is. Thank you.
5:05:32I'm Karthic, CTO at Perpetual.
