Insurance Claims Are AI's Messiest Data Problem — Denys Linkov, Wisedocs

All right. Thank you for joining, uh, the last workshop of the day. So I'm Denys. I'll be sharing a little bit about, uh, insurance claims data and how powerful it can be for enterprise-wide AI programs. So let's get started. So who here works in insurance? Few people. Who here wants to work in insurance? Okay, interesting. Uh, a few people. All right. Um, so insurance is a very interesting, uh, sort of sector of our economy where people sort of

find their way working there, but it powers such an important part of our ecosystem. So we're gonna talk a little bit about, uh, insurance as a whole, and we're ultimately gonna focus this talk on how do we improve our lives in general. So we'll focus on medical claims. We'll improve the claimant and patient outcomes, uh, as a focus of this talk, and I'll keep bringing that back. I think oftentimes in a lot of fintech, uh, sort of finance insurance discussions, sometimes things can get a little bit too business-focused. So I always want

everybody to, to focus on this aspect. So a little bit about myself. Uh, I'm Denys. I lead the AI program at Wisedocs. Uh, we help companies with their complex medical claim processing. I've been with the role-- uh, in, in the role for two years now, and we've seen a huge amount of transformation since 2024. So you might have seen some of the news coverage in terms of medical coverage and insurance in general. Um, the whole GLP-1, uh, transformation in our economy and our healthcare environment has changed how insurance is pricing

things as well. Uh, we also see that special insurance and broadly insurance coverage has continued to evolve with increasing costs, uh, and other factors that change the, the approach of insurance. So the talk is gonna focus on three main chapters. The first one is on the technology transformation that we have. The second is I'll go a little bit into the insurance space, uh, in terms of some of the, call it nitty-gritty, but sort of a, a brief introduction there. And then also I'll talk about how AI is going to help make insurance and the patient experience

even better. Let's talk about technology transformation. So we all probably in our work have started some form of digital transformation within our companies, whether we work in smaller startups or within larger enterprises. Now, the challenge with that is that we never finished this transformation. AI hit us in 2023 and 2024, but most companies have not finished their cloud migrations, have not finished leveraging big data to its full potential, have not deployed a clean data lake, right? Who here has a data lake they're proud

of? All right. Very, very few people. Is it even possible to have a data lake you're proud of, right? Uh, and on top of that, we're now getting all this hype about AI, and we need to really make sure that we have the other prerequisites down before we can actually deploy real applications.

So in insurance and many other industries, uh, paper was the big, uh, way of processing things, right? You'd get a stack of papers of going through a medical claim. You'd organize it. You'd have these giant tables full of them. You'd use cards to put in certain sections. And this is very difficult to actually get any, anything outside of the claim itself because how do you do cross-claim analysis when everything is paper, right? You could put as many folders as you want, but it's very difficult. And we still have not exited the paper era. There are still a number of

practices that, that use paper, uh, around the world, whether it's in the doctor's office, in, in legal departments, or within insurance companies themselves. So we are still not over paper, but we're getting close.

In terms of digitizing, uh, we are still also not over the fax machine. The, the fax market is continuing to grow. Uh, it's over seven billion dollars globally, which is always surprising. Uh, I referenced this, uh, in my talk at AI Engineer SF last year, and it has not changed. Fax is still being used. Now, what we're doing now is we're jury-rigging f-faxes into our digital systems where we can convert them in a fairly clean format. Uh, but it still does not solve the problem of how we actually exchange information in the healthcare system.

Now, if we think about how quickly technology has been adopted, uh, EHRs, uh, electronic health records, uh, were only at s-- eight percent adoption in 2008. And only with the, the COVID pandemic did we actually hit the eighties and nineties, uh, in EHR adoption. And again, many hospitals still run on paper. So we're still not fully through this digitization, digital transformation era, and yet everybody wants to do AI.

If we think about the types of forms people, um, receive, uh, when, when doing claims processing, uh, these are some three sample forms from three large states, uh, for workers' compensation. So workers' compensation is if you get hurt at work, you file a claim saying, "Hey, I hurt my arm falling down," or, "I was driving a forklift, and then something went wrong." Uh, and none of these forms look the same, right? So within fifty different states in the US, you have fifty different types of forms, generally. They all ask for different information. They're not standardized.

Uh, they're fairly standardized in that every form that comes in looks the same, so you can have some determinism there, but even then the information is different. So even as a country in the US, let alone any other country in the world, we don't capture information the same way, and these versions sometimes change as well.

If we talk about what people care about in the insurance space is a loss ratio, right? So how many premiums are you collecting, and how much money are you paying out? So I'll talk a little bit more about, uh, the-- these numbers here and what they mean. But in general, this is what drives the bottom line of an organization, right? When you report a quarterly earnings or an annual earnings, you're going to be emphasizing what is your loss ratio, how well can you predict risk, how well can you underwrite, how, how well are you managing fraud, how well are you managing your expenses And so

companies want to keep costs down and make more money, right? That's just capitalism in general. Um, and if we think about some of the statistics here, we, we generally h- uh, hover around the fifty percent, um, loss ratio here in terms of the amount of money we have to pay out for insurable claims, uh, compared to the premiums we get.

Now I'm, I'm introducing all of these sort of pieces of context, and I've only said AI once so far in my talk. And it's really important, again, to think about the problem that we're trying to solve and then only choose AI if it's the right solution.

Now for AI, uh, to get proper AI solutions, we need to go through the right amount of transformation, right? We need to have the right goals in mind. We need to have a team that can actually articulate what we want to do with AI. We can't just have people get on stage and say, "AI, AI, AI," right? We need the right people skills to actually accomplish this transformation, and we need the right frameworks in place, whether it's governance, security, infrastructure management, and data. Now as a question here, um, for those working at an enterprise context, who here could get a

database or a VM within a week if they requested it? Okay. Who could not? Okay, that's progress. Who here could deploy an application to production in a week?

A new application. Okay. Who could not? Okay, so, so we, we still need some progress. Who can choose a new AI vendor in a week? Ah, there we go. No hands. Right. So it still takes time for us to go through all these different processes. Uh, they're certainly better than they were ten, twenty years ago in the digital transformation component, but a lot of companies still get stuck in governance and security, and a lot of those have to do with, with the factors here.

Who here has been asked for some kind of query like this from an executive? "Give me some dataset that is perfect data." Right? A few people, right? We, we, we've seen this before. "Hey, can you pull that report for me quickly?" Um, it's much easier now with AI, but you need to assume that that data exists, right? So if we come up with a more complex query here, right? Uh, let's say that we, we have a medical claim and for a given patient and their condition, uh, in this case, uh, an ankle injury, we want to make sure that

we can assign them to a doctor who has top fifty percentile, uh, patient outcomes, right, within some recent timeframe.

Now, this query is fairly complex, but if you could actually just write this query on your dataset and get that result, right, I'd be amazed, right? Data is very, very challenging in an enterprise environment. So for us to get these kind of outcomes and get these kinds of insights, we have to structure our data systems in a, in a, in a very well-defined way.

So let's talk a little bit about m- about medical claims. So in general, I've grouped, uh, insurance based on the, the NAIC, uh, classifications. We have property and casualty, life, accident, and health, and sort of broader health, uh, insurance. I'll actually be talking about property and casualty since there the claims are typically the, the most complex. If you think about P&C, um, I mainly worked on the casualty side, so, uh, auto, uh, insurance, uh, various types of liability, general liability, product

liability, workers' compensation. There's also the whole property side, uh, which is its own, uh, set of complexities. But generally, this is a one trillion dollar market, right, uh, based on, on premiums, and reinsurance is its own can of worms. So if we think about these blue boxes, many of these handle complex medical claims. Somebody gets hurt because a product is defective, somebody gets into a car accident, uh, somebody gets hurt at work.

Now, adjudicating a claim and getting a response back to a patient or a claimant takes a long time. In the best-case scenario, it takes a few weeks. In the worst-case scenario, it can drag on for a few years. Depending how complex the, the scenario is, depending how complex the regulatory environment is, depending how litigated the environment is. And even once you get the, the medical services delivered, the claim can keep dragging on and on. So this is not a simple process, and I want all of us to imagine how much more effective the, the

healthcare system could be if we could shrink this by a factor of ten, right? You get hurt at work, you s- you submit the application, uh, you go through that process, and everything is digital and everything is smooth.

So why does it take so long? There's a lot of handoff points with getting even the medical side, not accounting for, for legal side, the actual insurance side as well, right? So when you, when you get hurt, uh, in one of these scenarios and have to be insured for a complex claim, you have to get the, the diagnosis, you have to go to various specialists. Sometimes specialists disagree, right? Then you have to make sure that you have the right coverage, uh, whether it's primary care or one of these, uh, additional forms of liability. You have utilization reviews where doctors have to agree with

one another that a procedure is necessary. Uh, we have the treatments, we have billing. You try to figure out what your bill means, right, with the explanation of benefits and coverage. You get the payments, and you have continued treatments. Now, there's so many additional bubbles here that can be added and forks in the road, um, depending o- on the steps, but in general, it, it is not a simple process.

And along the way, lots of things can go wrong. Uh, I just listed around fifteen that, that come to mind immediately, but this has to do with the HR situations in ins- insurance companies with claim adjusters having high turnover, higher retirement rates. Has to do with technology of people reprocessing the same data over and over again. Has to do with the types of, uh, situations we end up with where claimants are, are stressed and sometimes make mistakes, uh, and are in, in the position where, uh, they're not acting rationally, right? 'Cause if

you're-- you've gone through a traumatic car accident, you're not gonna be the most rational person. So let's talk about where technology can help us.

So let's go through, uh, a worker's compensation claim. Has anybody seen a workers' comp claim before? A few people. Okay. So typically, when somebody gets injured, they have to report, uh, it within the workplace and then make a filing of that as well. Uh, those three forms I showed, the Texas, New York, and California ones, those are typical intake forms. And you go through this process of actually visiting various people, whether it's in the emergency room, whether you have additional treatments and so forth. And you don't

always have your claim reviewed until you get claims later on. Now, as you go through that, there's various types of claims. Here, I generated some synthetic ones as an example, uh, but they're, they're fairly close to what you have in, in production. So you go through, you go through all these types of treatments, you get different types of procedures. You might be in-network, you might be out-of-network, you might be traveling for these, and sometimes you have conflicting information as well.

And as we go through each of these processes, all the documents look different. Every hospital has its own way of s-- of, uh, structuring a bill, right? Every doctor has a different way to write a progress note. It, it's not consistent in any way, shape, or form. So this is why the data is unstructured, and many times it's not in an electronic feed. It's a PDF, it's an email, it's a fax. So you have all these technology complexities that layer on top of all the business complexity that lies underneath.

Then you have something like this, right? Somebody does an assessment, and there's three different results, right? For, let's say, uh, the range of motion that somebody has. How are you going to resolve this, uh, in order to adjudicate the claim?

Now, we haven't even layered the, the other portion on top of this. While these medical processes are going through, there's also the risk of litigation. So within insurance, uh, typically insurance is on the, the defendant's side of a litigated claim, and you have the plaintiff's side. Now, the plaintiff's side is incentivized to handle as much volume as possible, uh, and to get as big of a settlement as possible. So you actually get into a situation where you have two different incentive structures. So on the plaintiff's side, go as quickly as possible, make the

most-- uh, the biggest claims possible, put up giant billboards everywhere. We, we saw those when we were flying in probably, right? Uh, all over the place. Uh, and the plaintiff's side has actually adopted AI a lot more quickly compared to the insurance or defendant's side because you get a percentage of how fast you move and how big your claim is. Now, the range of the actual percentage of compensation for, for lawyers, depending on your jurisdiction, can range from nine to fifteen percent if you're in California for workers' comp, uh, fees. In other

jurisdictions, you get into that one-third territory or even forty percent. So now on top of healthcare, on top of insurance, on top of technology challenges, on top of administration, you now have litigation, and insurance companies set a l- uh, set a lot of money for, for litigated claims. Now, the challenge here is that if you have not gone through a proper digital transformation, you are especially vulnerable to, to bad actors, right? So you can be sent repeated claims, you can be sent fraudulent claims,

AI is making that even easier to do right now. So if you h- don't have the ability to actually handle those and process things efficiently, you get more and more fraud in the system, right? It's similar to if your website is easy to DDoS, right? People are going to keep DDoSing you.

And w-we forget about what matters most as we start solving all these sub-problems, right? So these are the things that matter most in terms of the quality of somebody's care, reducing costs, making sure it's a quick process, and it's a good experience.

So let's go through and, and solve some of these problems. I'll talk about three different phases and how they can help. So number one, let's talk about clean data, how we can accomplish that. Number two, let's talk about within-document analysis, and then let's talk about cross-claim factors.

Okay. Clean data. So has anybody here worked with claims data before or any kind of insurance data? A few folks. Okay. How many duplicate records sort of have, have you processed or have you seen?

M-many. Yeah, millions, right? Uh, depending on the jurisdiction that you work with, they can be as much as fifty percent or even higher, right? So you have so many duplicates coming in, and sometimes this is intentional, right? Sometimes to get a claims process, you actually have to keep pinging people, right? Following up with my claim, following up with my claim, right? Uh, so because people are slow, you get into this sort of cascading failure effect that we see in technology. Commingled records with other patients, right? You have Bob Joe, and you have Bob Joe Jr.

Right? Those medical records get mixed in a-- at a, at a fairly often occurrence. It violates HIPAA. There's a whole bunch of regulatory risk for that, but the system and the digital transformation is not set up for that. And decisions can actually be made when other people's medical records are, are mixed in, right? So this is especially sensitive when there's a high, uh, complexity claim where things can be missed. Somebody's record gets mixed in by a doctor upstream or administrator, and suddenly the whole claim falls apart. Then

we have the, the more expected items of poorly scanned or undigitized data, right? You have some, some handwriting that you can't understand, let alone doctor's handwriting, right? Some things come in on paper. So we, we keep getting these poorly, uh, clean data examples, and they happen for many reasons, right? So some of this happens because, uh, you have a lot of copy-pasting that happens in the medical system, right? You go through, uh, doctors or nurses as, as they're updating the medical records, right? They'll

copy-paste between patients. They'll go through, and they'll copy-paste between visits. Uh, you might have seen that personally, but it happens a lot, uh, in the space. Uh, EMRs are actually not set up consistently. So has anybody here seen an EMR before? Their own medical record, right? Anybody here moved across providers or networks? Yeah. Do they look the same? No, they don't look the same. Um, I, I'm from Toronto, Canada, and many of our hospitals do fresh implementations every time

they set up an EMR. And this often happens both due to the initial processes that exist within a hospital. It happens across a network as well. There's very interesting dynamics that happen on why it's set up a certain way. And if you want to talk about data cleanliness, people do some very strange stuff in EMRs with nested fields.

So even though we do have electronic health records and medical systems, they're all different. They're structured weirdly, so you do have to do some, some transformations on it as well. And oftentimes it's, it's difficult to pull data out, so people just choose to, to send faxes or PDFs at the end of the day. So solving clean data is not an easy problem. Uh, AI can help us. Digital transformation can help us, but upstream there's a lot of factors that go into it.

So, uh, this is an example of some illustrative data, uh, of duplicate records. Uh, real data looks quite similar as well, with a high number of duplicates across certain networks. And this is actually very frustrating if you are on the receiving end of these duplicate records, right? So we have a number of customers who are like, "Can this organization stop sending us so many duplicates?" Right? And unless you have good systems to detect this, you, you just keep getting the same bad quality data. So it's really important to have those technical implementations and detection systems to

actually make sure that you can safeguard your, your own data q-- uh, data, um, cleanliness. Now, once you have this data, there, there's mechanisms of, of actually fixing things, right? For regulatory or, uh, interpersonal reasons, right? But you still have this messy data that exists as a baseline. And once you can detect things like duplications, uh, again, seems like a solved problem, uh, you no longer incentivize things like sending claims multiple times with, with different

dates or the same claim, uh, or just other folks have to clean up their data as well.

The other part that beyond just the cleanliness of the data is that these claims are, are really large. Uh, when I first started working and saw within our database that there was a PDF that was eight gigabytes in size, that was very surprising. Um, so it-- these claims have anything and everything within them. They have, uh, images, right? Uh, if you think about auto accidents, right? There, there are images, uh, within the site of injury collisions and such. There's X-ray reports, uh, ultrasounds,

everything in between. And sometimes this just crashes a lot of PDF viewers. Uh, if you think about some of the old computers that you might have, trying to open up an eight giga, uh, byte PDF will be, will be quite challenging. Uh, and even on the technology systems, right? We talk about context windows all the time. How are you going to fit a context, uh, into a context window an eight gigabyte PDF, right? It's, it's not as easy as saying, "Hey, Claude, take a look, uh, at my ten-page PDF and, and give me a, a response," or, "Hey, ChatGPT or Gemini," right? Not that easy. Um, and on top of that,

actually going through and working through a case requires a significant amount of time. So these are the technical challenges, uh, or the business challenges that result in very complex technical challenges.

Now, if we work through and we're adding more and more complexity into here, a lot of these claims also have irregularities. So you might have some progress notes, for example, of a earlier patient report that says, "Hey, here's this X-ray that was booked for this date. You received the claimant package, and that X-ray is not there." Right? And it's very easy to miss that without actually having this highlighted in some sort of way. Um, and without that, again, the whole trajectory of a case can change whether there was adequate care given, uh, whether the person actually went to the,

the, the care provided, uh, and how do you actually cross-reference this with the billing side of things. So this is a very, very common scenario where upstream, again, because the data is messy, you actually have to ask repeated times, "Hey, can you send me this record? Oh, we missed faxing this. Oh, the fax glitched. Oh, we changed our back-end system." Right? So with-- as an organization, you have to have these lines of defense in, in order to make sure that these get flagged. And as a claims adjuster, a claims adjuster might be processing two hundred claims at a time. So

they're not gonna be able to keep everything within their mind saying like, "Oh yeah, I need to follow up on, on this X-ray or not." Right? They can take notes, they can move forward, but things are gonna get lost. So this is again where technology can be a great lift, uh, to these kinds of problems.

Then you get conflicting diagnoses. This happens fairly often as well. So who here has sought a second opinion for a medical condition? A few folks, right? Um, sometimes we, we seek them out, um, to get another perspective. Sometimes you get completely different answers, right? And the more complex a claim is, the more likely there is to be some ki-- some, some kind of conflict. So in this case, uh, what often happens is when you adjudicate a medical claim and you see some conflicting, uh,

diagnosis or you want to double-check how much you should set aside to actually pay out for the claim, you send it to an IME or an independent, independent medical evaluation. Then you have another doctor who performs their, their review of the file. And then you have to manage those doctors and the provider networks and the IMEs themselves. So now we're stacking again, more people into the ecosystem, more complexity.

Now, AI can be quite useful here. Um, and it would, it would be, I guess, poor taste to give a talk in twenty twenty-six without talking about evals. Um, but we did use AI to, to try to solve these issues, uh, in some ways, the conflicting diagnosis, the missing records, and a number of other ones. So we built this benchmark called MLCR, uh, with Artificial Analysis to, to measure the effectiveness of AI in actually answering these questions. So I'll flip over to our blog post, uh, that we talked about, and I'll scroll

down to what this benchmark looks like. So we released, uh, six different tiers of questions related to medical claims. Some of these are very simple, like when did this injury happen and what was the injury, right? Then they get a little bit more complex. How many surgeries get performed? Uh, how much capacity does the-- this patient have? And eventually, we get into more and more complex questions that are even harder to, to resolve. And oftentimes, the way that you ask a question as a

claim adjuster, a doctor, an administrator, you don't have full context of what's happening. So this is where we see hallucinations show up, where people assume something is in a medical record, but it's not.

So let's go a little bit more into the results. So, uh, this is available on the Artificial Analysis website, um, if you want to take a look. Uh, so we released an open source, uh, dataset for the first three tiers of questions that we have, the easy ones, and we have a private set that we host with Artificial Analysis for, um, to stop benchmarking to, to a certain extent. So it's very interesting to see how, how models perform. Uh, we actually measure this on three different, uh, axes. So let me flip back to that slide. We measure this on

conciseness. So how concise is a response? Uh, and we measure this against a human, uh, expert-validated answer. We then go through accuracy. Is that-- is the answer correct? Uh, and the final one is completeness. Are all the relevant, uh, informations-- uh, pieces of information answered, uh, in the question?

So we go back here. Um, the frontier has continued to evolve very, very quickly, and this is where having evals is really important. Uh, when we released the eval, uh, the highest accuracy that, that we saw on the levels of reasoning that, that we had was twenty percent. A few months later, we're in the seventy-five percent with, with the very high reasoning levels, right? So models move very quickly, um, but this is great because it means that we can start productizing a number of the more complex questions, and we have more certainty of the confidence that somebody has to review

this, uh, in a system. So within our system, there's always a human in the loop, uh, so that's really important. You also start getting the, the cost curves here on the Pareto frontiers on the cost to actually ask this question and get a suitable answer, and this has been improving through the past few months as well. And you get some of these trade-offs as well. So one of the interesting things when we were running these benchmarks on, uh, document analysis was that the GPT models were, were excellent at accuracy, but they omitted some key pieces of information.

Uh, Anthropic's models were, were quite good, um, and they're mu-much more complete on the amount of information. Um, they, they do have slightly lower accuracy here as well. Can I ask a question? Sure. Accuracy, how do you measure it? Compared to the human factor? Yes, compared to, to a human answer, uh, and then we do the triple judge, uh, verification. So yeah. So we have accuracy here. You can see how the curve looks like, and we have completeness, uh, which is, uh, is still not solved across most

model providers. And yes, to, to the earlier question, uh, this is based on a human verified answer and on a calibrated set of judges. Uh, within our blog post, we describe the, the methods we have here, uh, and we released a follow-up as well on, on how we did this calibration. So even though we're using AI here to structure our data and extract important insights, we wanna make sure that each step, uh, of that process is well calibrated because if upstream, uh, your data is incorrect and you start using

it downstream, right, you haven't gained anything. So again, really important as you're going through the claims, uh, uh, process to, to calibrate your judges to build good evals. Uh, I think that's a recurring theme at, at most talks. And you wanna make sure you have those experts who can actually verify this information, so you work across teams to, to get there.

Okay, so that's within a claim. Uh, we know that w-with some amount of certainty we can extract some types of insights. We can flag conflicting diagnosis, we can deduplicate, we can search for missing records. Let's go to something more interesting, which is cross-claims. Right? So if you are a mid-size, uh, insurance company, you might be collecting one billion dollars in premiums within a, a portfolio of lines of businesses. And you might be paying out, let's say, twenty thousand, uh, claims, uh, at a

time per year, uh, or even more, depending on what line of business you deal with. Now, you want more insight of where you actually-- like what's affecting your loss ratio. What kind of claims are people making? In which geographies? What type of injuries? And what type of doctors are being involved?

So this is where I think it gets quite interesting as well, is actually being able to... Once you have that data extracted from the claim, you can start doing interesting visualizations, regressions, and work as well. Right? So two of the questions that we, we've been asked and we've, uh, worked on answering is sort of how much are people traveling within a network, right? Uh, so if a patient is, let's say, in San Francisco or New York, right, are they getting treatment in their local area? And is that the right person to, to be giving the treatment? We also have these networks of who's actually working together. So this is

some redacted data, uh, a subset from, from California, and you can actually track to say which lawyers are working together, which doctors are working together, uh, which chiropractors, physiotherapists, and what is the actual, uh, working relationship look like and the story within the claim itself. So this is that sort of higher tier insight that you can get within, uh, extracting data from your claim.

You then get into some interesting medical outcomes, right? So which conditions co-occur together. So if somebody has a back injury as an example, right? You might have a, a series of d-different diagnoses, uh, that could be mapped to ICD codes that co-exist together. Uh, and you might see, okay, why does this exist from a certain, uh, doctor's, uh, write-up versus another's? And in this case, again, you, you only get these capabilities if you can solve the data extraction problem within one claim and then roll it up into your broader,

uh, insurance portfolio.

Now we get to the money portion, right? So pulling a small subsegment here of two treatments, uh, of physiotherapy. Different doctors charge different amounts, right? We've probably seen that. Um, and sometimes that's fine. If you're a better doctor, you should be able to charge more, right? If you can get somebody to recover as quickly as possible in five treatments instead of ten, you should be able to charge more, right? So, um, but it's really important to understand costs and, and focus on minimizing them, 'cause as, as soon as there is cost inflation through different parts of the medical system, it

keeps compounding, right? Premiums increase, uh, the fees that we are charged increases, uh, and we get less for our money. So within this analysis, uh, you can see across claims, is there a consistent factor? What, what is the amount of coverage that you should be providing? How, how should you be underwriting these costs for claims as well? And you can estimate how things are gonna change over time, right? So year over year as an example here, uh, there was a ten percent increase for this type of treatment for a given doctor, right? So is that gonna happen across,

uh, the jurisdiction? Is that gonna happen across the, the industry in general?

So then you can start doing some counterfactual analysis or, or changes in a, in analysis to say, if we change our network, if we start directing folks into a certain direction, if we change how we, how we bill, right, what are the potential savings that exist? And what kind of procedures should people be having based on the standard of care that exists? Um, there's, there's recently an article about how... Has anybody see, seen at the overbilling article, uh, about one billion dollars of additional billings that have come in? Anybody seen that? Um, so

those are examples where again, if you cannot process data effectively, people are going to find ways to, to, to go through and, and submit additional records, uh, or do some, some shadier behavior as well. So all this to say is that once you have good data, you can start making these decisions.

And then once you have this decision, uh, this claims information, you can start writing it into a broader system, right? So you can start using this in, in other lines of business of saying, if you know these are the trends in auto injury or workers' compensation, are we seeing the same thing in general liability? Uh, if you know that your underwriting model is missing cer-certain trends, uh, or you don't have as much explainability there, can we make the model more explainable with what we're seeing? Um, how do we provide this sort of magical data table that we

wish that, that existed, uh, within our data lake? So again, the more data you have, the more complete of a picture you, you can make, and the better you can un-underwrite and price the product at the end of the day for the consumer. And this is information you need to actually power the, the effective AI applications, whether they're on, on the underwriting side, uh, on the fraud side, uh, on the claims side as well.

So if you think about like a blue sky world, right? We, we want to reduce something more broadly. So if we wanna reduce costs for everybody, right? Taking into account that patient experience. If we wanna reduce the time that a claim is open, if we wanna improve how quickly people get back to work and the, the treatment they get, we need to make sure we have the right data and the right digital transformation to do that. So this is an illustrated example, uh, in the work that, uh, we've done within Wisedocs as well. We've seen significant savings as

well. Um, so going through this digital transformation process and having the right data is really, really important. So I'll wrap up the workshop here. I did want to leave quite a bit of time for questions 'cause it is pretty comprehensive topic. Uh, so thank you for listening and happy to take some questions.

Can you share the blog post link as well? Yes. I'll, I'll share the blog post. Yes, question over here. Yeah. So, so the question was about how do we actually structure the data to build, um, all, all this information. So, uh, it goes through a number of different levels. Um, I'll share some talks that I've given on, on the topic as well. The baseline is we, we use OCR and images to, to extract information from the underlying records, sort of standard pre-processing. Uh, pre-processing PDFs is not an easy task as well, if anybody has worked with them. Uh, those eight, eight gigaby-byte

ones have some, some weird one in a million examples in them. So once you have the, the data, the text and images, we have a series of models on top. Um, NLP models, extraction models. We use VLLMs, uh, to structure it. Sometimes like a, a more standard entity extraction model works well. We have various classifiers that we build on top. Um, we then have a summary layer that sits on top of that to summarize the documents, uh, that is then human verified to make sure that the information is correct. And finally, we run those additional layers of analysis on top of

that. So we have this abstraction pattern that sort of concentrates the, the density of information as you go up. Um, but it's verified at various points in time with various models and, and human intervention as well. So that means once you get to the top and the highest value items, that's when AI can be effective. And those are sort of those MLCR, uh, type questions that can have the right answer. So within MLCR, this actually uses clean data to ask these questions. Real world data is not clean, so you have to have various steps ahead of that to, to clean up the data to get

there. Yeah. So typically our, our customers... So bigger customers have multiple lines of business internally, and they have different, uh, rules, I would say, across sharing, across different teams. Uh, other customers just work in one line of business. So they might just work in auto, they might just work in disability, they might just work in workers' comp. Um, but we take the trends, uh, across the board to try to figure out and flag those. So there's the question that within sort of an office, right? How do you surface some of those insights, uh, or,

or sort of geography? Then you have the, the layer on top, which might be, um, the broader company. But com- insurance companies also have a collection of companies within them, sometimes, uh, separate businesses. Uh, and then how do you actually feed that into a centralized data lake, uh, as well? So again, every company is a little bit different, but we, we go as fast as our partners want us to. We live in claims. So, so we focus on claims. So claims are, are our bread and butters, spec-specifically complex claims. That's where our whole AI

pipeline, uh, is focused on. Um, but as we sort of evolve and go and solve the sort of easier challenges, I would say, with adjudicating a claim, processing a claim, extracting that information, you start seeing these more sort of insightful aspects that help, uh, organizations make portfolio-level decisions. Yeah, yeah. So that's a good question. So the question is, how do we onboard somebody when there, there's a lot of these challenges? So typically, we work with-- The, the most important thing is a willingness to change, right? So we, we've all probably

worked with teams or organizations who want to change versus those who don't. Um, very, very different experience. So assuming it's an organization that wants to change, uh, there'll be a few sample files typically that we'll work through s- uh, in various formats. Typically, people will send us either things they're immediately working on or have or basically are trying to back test. Uh, and then we go through a review process to see what did you extract, uh, what did our system extract? How does it align? Does it match the information you want to see? Do you have different types of, uh,

standards? So if we go back, for example, uh, to the different types of documents here. Let's just go back to the intake form. There might be various dates on this form. Do you want the, the date of injury, or do you want the, the report date when it was submitted, right? So these are things of, of alignment of business objectives. Go through, process a few standard cases. If things work well, then we start sort of a, a broader pilot, and then gets into the fun integration aspect of integrating with, with upstream providers, apart from just, like, hand-uploaded

documents, API integrations, like dealing with all, all the messiness. So as long as there's a, a clear business outcome and transformation goal, then it's, it's a fairly straightforward process. Um, and again, as, as we go through that journey, again, sometimes it's like we don't have the capabilities out of the box. Sometimes we do. Sometimes there's multiple integrations on internal systems as well. So. Yeah. So let's start with the first part of the question, which is handling regulations and how do we work with partners. So we-- It depends on how, how quickly people are, are moving.

Uh, it also depends how quickly regulations are moving. So the, the way we have set up the system right now is that it always requires a human in the loop to make a decision. Uh, so we surface these insights as an example for things like conflicting diagnosis. We'll see, hey, on page two, on page twenty, there, there's a difference of diagnosis between these two doctors. You should take a look, right? So as long as there's a human in the loop, I'd say it, it removes like seventy percent of the, the regulatory concerns that people have. Uh, regulation is also still

catching up. So, uh, there's a lot of things to read about, like the NAIC reports that I mentioned. They-- that organization publishes a number of different reports. There's the guidelines that exist for that. We talk to the internal governments and regulatory teams as well. Uh, and if they want to see some pieces of information, like we will share with them, like this is how the AI pipeline works, right? This is the type of data that's being used. Um, but at the end of the day, as long as you don't build a black box and things are auditable, uh, and there's a human in the loop, that take cares-- uh,

takes care of many things. The regulatory side, there's like a whole other, like, business deci-decision aspect, like how to approach nuclear verdicts, uh, for... Uh, nuclear verdicts are, um, very large, uh, awards or, um, claims that, that are ten million dollars or thermonuclear ones which are a hundred million dollars, which really reallocate, uh, businesses' reserves. So for those ones, typically people come with, like, an approach on how they want to handle them. And within the platform, you can say, "I'm gonna handle my most complex claims

first," or, "I'm gonna handle the one that was submitted first." Um, and those are all sort of business decisions people can make. Uh, the second part of the question on, like, people's internal, like organizations' internal goals. Um, I mean, we want to work with partners that share the same sort of mindset about improving patient care, uh, reducing backlogs, uh, for claims, providing the best decisions. A lot of times our customers are not even thinking too much about, like the, the cost of healthcare itself is everything around it,

right? So, uh, if you're a large insurance company, you actually spend hundreds of millions of dollars on legal fees, right? So that is something that is not even tied to the patient experience, that there's a large amount of optimization you can do there and to speed things up, right? Um, there's regulatory concerns that in certain jurisdictions like California, things are very litigated. Um, you want to figure out how, how to deal with those. Um, there's a whole bunch of middle parties that are involved as well, uh, that are also inefficient. So depending on the customer, everybody has

a different goal. Um, but in our conversations, it's, it's always been like, we have this very obvious problem to solve with technology. Let's solve that first. Um, and again, the insights start surfacing afterwards of being like, "Oh, I didn't know that about my business. Let's solve that afterwards." So. Yeah. So the question is about decision APIs in Jev. Um, as part of the-- I don't know if anybody's seen the paper, um, copies of the AI News that are-- that the conference is handing out. There's actually an article that I'm writing for that. Um, so we,

we've been using classification models, uh, for a long time. I've generally had a strong opinion on that. It hasn't come out today. It's coming in a later edition. Um, I think it's very useful, um, but LLMs are not the right solution for everything, right? So when we talked about this, right? This segment of AI can be split up as well, right? Classification, extractive, um, versus sort of the more complex decision-based ones. Uh, so for us, cost is really important, so that's why we trained our own models as well. Accuracy is really important and

hallucinations as well. So I think Jev was not as surprised that it came out. It was more so a surprise of how long it took because I think the LLM, like token max reasoning for every problem, uh, phenomenon lasted for a very long time until we started thinking about costs a lot more this year. So, uh, once, uh, Jev is sort of HIPAA ready, we can test it out. Um, but we've sort of built our own models with the same paradigm in mind. So. Yeah. So I'll answer part two. Um, so as an organization, we have not

moved the broader industry, um, to the extent we want, right? So to say that we can say we've reduced the, the claim, uh, time by five days on average for somebody across every patient, we'd love to say that. We, we're not in that position yet. For specific organizations, like we had some case studies on the website. I quickly put, put some up as well. There are some significant ones within lines of business, uh, for cost reduction and speed. Um, so you can check those out, and we can chat afterwards as well if you're looking for more specific sort of

context on that too. So. Yeah. So, so the, the first notice, uh, of injury or loss, um, or depend-depending on the, on the subdomain. Yes. Um, so depending on the jurisdiction, there's also turnaround time requirements when handling these. So for example, in Texas, you only have a few hours of processing a claim to get an answer. In other states, it might be seven days, twenty-one days that you need to get a response. Now, it gets more complicated because if you're a worker who's been working in a factory, let's say for thirty

years, you're sixty years old, you're gonna have large medical record file, right? So there's a lot of information here and in the initial report. But to do a full claim analysis, you have to go back, and you have to try to pull those. Maybe your name changed because you got married, maybe you moved, uh, cities, providers, right? So you have to reconstruct sort of that, that full, uh, history as well. Uh, and I think unfortunately, in a lot of litigated cases, there's information that's left out, uh, as well on purpose to--

due to the fact that a lot of organizations have not digitally transformed and can't process cases fast enough. So, uh, if this was all that's needed, this would be, uh, much faster. But unfortunately or more realistically, there's a much broader backstory as well, so.