AI Copilots for Tech Architecture: The Highest-ROI Use Case You’re Not Building — Boris Bogatin and Toufic Boubez, Catio

Hi, I'm Boris Bogatin, CEO and co-founder of Cat.io.

Hi, I'm Toufic Boubez. I'm CTO and co-founder, also at Cat.io.

Today, we're here to talk about AI copilots for tech architecture, the highest ROI capability you're not yet using, a topic that's been near and dear to our heart. Over the last few years, I would say, coding copilots have become truly table stakes.

You know, a- and it's interesting because, you know, you take it back three, four years ago, I know Toufic and I talk about this a lot, to his days-

Yeah

... back when he was the VP at Splunk, a lot of the, you know, hotshot developers would always talk about how, you know, coding copilots would never be able to kind of supplement them.

That's right.

Right, Touf?

Yeah.

Yeah.

Would never work, yeah.

Never work, right? Yeah, because how could you? And now coding copilots are helping us tremendously multiply productivity, output, and, you know, if we look at the whole cycle, as you can see on the slide here, you know, the full life cycle of software development has been so well, you know, uh, situated and, and, and served with tooling,

from software project management to execution to operations, Splunk and Datadog. Today, the software life cycle, software development life cycle is filled with tooling. Coding copilots are multiplying the productivity, and we're excited about that, which we should be.

But when you step back, you step back and you ask the question, you know, is there something missing and something yet not addressed? Because isn't the highest leverage copilot the one that we're really not using yet, the architecture copilot?

Why architecture? At the end of the day, architecture is where ROI is won or lost. If you're going into the wrong direction with a lot of coding output, are you not gonna get to poor code, poor results, and a lot of redo and tech debt versus moving truly into the right architectural direction?

To us, architecture decisions is what drives things like nine-figure spends, um, through business objectives and, and, and how tech fuels them instead of slowing them down, how you can stay ahead and best in class versus drown in tech debt and always playing catch up.

That's really at the heart of, you know, why we've come together here, um, around this topic and, uh, we're seeing this across the board with a number of stakeholders that we'll talk about today.

Today's reality, a lot of the orgs manage this with spreadsheets, tribal knowledge, gut instinct. It's always been done by very smart folks, CTOs and architects, and increasingly delegated in shift left fashion to developers, and it's fantastic to see the, the whole organic process.

We love it. But we, we, we've always thought that there's gotta be a better way, and especially in the day of AI, there's gotta be a better way. So today we wanna walk through the three critical challenges that are keeping leaders up at night, that we hear day in and day out, and how they're being solved and

what that future looks like. In closed-door CTO dinners and, you know, our work with enterprises and growth stage companies alike, we keep hearing the same pain points. Toufic, I know you've been in the weeds on this.

What are the top three things you keep hearing from architectural leaders-

Yeah, no-

... that are keeping them up at night?

Oh, great. So, so based on a lot of conversations we've had, and actually on my own experiences as an architect and as a CTO, long-term CTO in, in many companies, there's at least three big challenges that we typically encounter.

Um, the first one is visibility. So as your tech estate grows, you start to fly blind across your landscape. Excuse the mixed metaphor here. I like these metaphors. But you know, you start flying blind a- across that landscape and, and it's really hard to kind of gauge where you are or, or, or to make real plans.

So that lack of visibility is one of the biggest issues. The other one is, uh, having ROI tied and data-backed path forward. You know, knowing where to focus, what to prioritize, and how to defend your decisions in a way that can be backed up by data is really, it's, it's always been a challenge.

I mean, I, I sit on boards or with other executives, you know, at startups and at big companies, and the question's always, well, you know, I ask for things or the, I ask for stuff and it's hard to always to, to have a good answer that is data-backed, right?

So, so how do you do that?

Absolutely.

And especially that is, uh, tied to ROI, because at the end of the day, you know, how do we spend, how do we manage our spend? The third one, though, is some form of autonomous guidance.

Now that a lot of organizations are shifting left and, and delegating more and more decision-making to the, to the, and empowering the developers, which is a great thing, figuring out how to guide them and equip them with expertise, um, you know, at scale is the third big, big issue that we're, that we're constantly facing these days.

So, um, and the main reason for these issues is, uh, you know, there's the, there's no dependable live holistic map of our services or dependencies and drift, how things change over time.

Really there's no baseline from us to go from. So as a consequence, you get slow, defensive decisions. You got redundant spend, you know, that you can't justify. You know, you got risk that's not properly managed.

You know, you're planning, you know, you mentioned, Boris, about, you know, tribal knowledge and so on.

Yeah.

You're planning basically by opinion instead of planning by data. So what we really need is some kinda live visibility that captured all, all that messiness in our system, all that knowledge, and the shifting dependency, in that sense like a share- like, you know, the, the developers have a shared code book, a shared current reality for us for,

for our working systems. You know, because without it, really you're making sometimes multimillion-dollar bets without knowing what you already own. And, and, you know, we've seen that actually in, uh, in, in some of the prospects, some of the people we're talking to behind closed doors.

So-

Absolutely.

Yeah, yeah. So to continue the analogy, I know I'm mixing metaphors using analogies here, but, you know, to continue the analogy, if you wanna chart a fruitful path forward, uh, you know, what you really need is an accurate up-to-date map.

So when you're charting path forward, you need a map. You need some kind of living architecture map that updates itself as your system evolves. So that's kind of like, uh, one of the major, major things that we're looking for.

Absolutely. Thanks, Toufic. No, completely. So you have visibility.

Yeah.

But now what? How do you prioritize? I know there's a lot of scarce resources. Business wants to achieve some, you know, very important objectives, rapid growth.

Yeah.

And everyone thinks their project is the key to success-

Of course

... but without the good proof.

That's right.

How do you reconcile that?

Yeah. My project is, is the critical thing. You have to do it.

Of course, obviously.

So, so what you're asking me is how can I get expert ranked actions that are tied to business impact? 'Cause at the end of day, that's what, that's what matter, like cost, performance, risk, time to value, all these things that matter to the business, right?

So-

Right

... it's not just what should I do next or sh- what should we next and/or whose project is in favor, right? It's what should we do next given our constraints, our existing investment, and our strategic goals.

That's the real question. That's really what you should be focusing on, right?

Completely. I mean, I think this is what we're hearing where the challenge really lies.

Yeah.

And it's always, what I always love is getting into those dinners that we are doing and podcasts and the whole architecture deconstructed movement, and asking those questions... Oh, that nice T-shirt. [laughs]

Asking those questions, asking those questions openly. You always, you know, kind of, uh, I'm always surprised by the, you know, kind of the, the, the, the honesty and, and the intimacy of the responses that are really almost demerit from what you expect.

You're expecting certain things like, "I want perfection here," or something else, and people are like, "I'm just trying to make sure that, you know, business understands what we're doing is important and allocating budget to us, and we're able to drive-

Right

... the business forward, really care-

That's right

... and not, like, kinda poke in a bunch of random directions." So anyway, so I, I totally get this one, and how do we tell what is the right architecture and how do we prioritize the work?

What are the metrics? What insights do we use to know this-

Yeah

... kind of to achieve that kind of impact, right?

Yeah, absolutely. So, you know, you have to have a system of recommendations. And these recommendations really to fulfill what you're talking about must be explainable and traceable. In essence, why is this recommendation valid?

Where is it coming from? What is the expected impact of it? And then, you know, what are the measurable outcomes against some, our key objectives, right? So what this results is, is a roadmap where every initiative is clearly scored for impact with the ROI justified and kind of the business objectives and best practices are all taken into

account. That's really what it comes down to.

Completely. And if I may just, just jump in for a second.

Of course.

To me, it seems, speaking about this, seems like an almost complete no-brainer. Why would you ever wanna start coding and developing software until you have this answer? Because if you answer this, then everything from there, that's true productivity.

Get more lines of code out, that's great because now you know you're coding-

Right

... in the right direction versus the wrong direction, right?

Absolutely. It's the old, you know, ready, fire, aim joke, you know? [laughs]

Totally.

You don't wanna do that. You don't wanna do that, right? So, uh, so it's, it's the same, it's the same thing here, right? So this is even-

Yes

... more critical these days though, to your point, Boris, because the shift left promise which empowers developers to make more decisions has a flip side, a little bit of a darker side, which is that architecture expertise and standards are not scaling.

They didn't scale with that empowerment, right? So developers are-

That's right

... making architectural choices whether you like it or not, and then the architectural guilds or the enterprise architecture team, whatever, they review, they just don't scale effectively to that.

So the question is: How do you guide them without being a bottleneck, right? That's, that's the key question there in, in enterprises, right?

You know, and we, we hear it all the time, right?

Yeah, absolutely.

We hear, we hear teams say, you know, "Yes, it's difficult, you know. We have all the presentations, we have all the strategies. We get together every two weeks," and, you know, we hear crickets.

We're, we're, we're talking to everyone, and everyone is kind of trying to absorb. But ultimately we get it because they're trying to build features and ship to business needs and ship fast, and their features have nothing to do with our standards.

They're trying to fit their specific, you know, uh, capabilities and how do they kind of architecturally map that to the baseline that we want.

That's right.

What's needed, right? What's needed are tailor-fit designs that are suited for the developers, copilot that can give them an, a kind of conversational guidance, ongoing guidance.

That's right.

But all of this, I mean, I know it sounds magical, but all of this with policy and guidance built in, so it's all policy and guidance aware, right? And it's embedded in developer workflow.

That seems like the right answer.

Yeah.

We'll talk about whether that's achievable, but that feels like the right answer, right?

Yeah.

And, you know, the governance paradox is all about, like, autonomy without alignment creates chaos, and gates without autonomy kills productivity, and we know that that's true.

Exactly.

And so how do you reconcile, right?

Exactly.

We wanna get... Yeah, we wanna get developers to get that expert guidance, generate designs that are compliant, and stay aligned to strategy so they're not waiting and they have built-in alignment-

That's right

... uh, built in.

Exactly.

Right?

Yeah. Absolutely.

Well, let's, let's shift now to a little bit about how do we solve this, right? So we talked about these three challenges, really important. Let's address how we really-

Yeah

... kinda can think about the most effectively. What are those three pillars that make a true architecture copilot possible, and what it takes to kind of accomplish them? Go ahead, Toufic.

Yeah. Absolutely. So Boris, as you know, you and I, Boris and I have been thinking about this for quite some time and, and we've developed this kind of, these three pillars that are really, really important that together hold up this whole foundation, this whole business, uh, ar- or architecture, right?

So the first one is what we call stacks. You know, it's your live visibility layer. Remember I talked about the map earlier, having an updated, up-to-date map if you wanna chart the course.

So in essence, being able to ingest data across clouds, across Kubernetes services, across logging platforms, you know, building model dependencies drift and change over time, and then maintaining this kinda living architectural in form of a digital twin.

So you get all that data from everywhere, and then you fit it into this, build together this digital twin of your deployment, your architecture, and a true system model that reflects the reality, not what's in your wiki or not.

It's what you have as opposed to what you think you have, right? That's really the first pillar, having that, that map, that live visibility map.

That makes sense.

Yeah.

At the end of the day, if you don't understand what's this all about, what are you trying to drive to, where do you-- where is the puck going, right?

Um, y- you won't really be able to get there. And f- and in that context, you have to be able to curate those business objectives, those requirements, the standards and strategy, and-

Yeah

... and be able to kind of couple that together-

Absolutely

... into a context that the AI can leverage in order to make very informed and tailor-fit recommendations with expertise, you know, very custom fit to the specific, you know, business objectives and workspace objectives, specific team objectives they're trying to serve, right?

Does that... Is that kind of, uh-

Absolutely.

Yeah.

Absolutely. So now, you know, this, this is where-- I mean, we mentioned AI a couple of times, but this is kind of essential. I mean, one of the major goals is to provide the, these kind of data-backed, you know, best practices, uh, ROI-based recommendations, right?

And es- especially when it comes to architecture, you know, not to, not to kind of minimize the amount of work that it takes to do coding copilot, but architecture is yet an, a, an, a higher level, a higher degree, higher order of magnitude in terms of, uh, complexity.

So, so this is a really hard problem, and it's, it's a, um, you know, the, uh, the, the, the typical problem that you use what, what's called, you know, uh, distributed problem-solving because it's not a one-shot deal.

It is a problem that where everything is interconnected, right? So you have to break out all the dependencies and then attack them and then, uh, and, and then work together to actually come up to some kind of recommendation that is global in context, right?

So this is a typical distributed problem-solving, I think, and this is where, you know, this is perfect. So a type of solution for multi-agent systems, right? So we've, you know, if you look at, uh, how multi-agent systems work, if you build agents that actually focus on various parts of the problem and then they collaborate towards a solution,

that's really kind of one of the best ways to solve this kind of complex problem, right? Now, a multi-agent system right now today rely on large language models, LLMs, right?

Uh, and, and LLMs have read practically every, every best practice, every architecture book, and so on. So they have a lot of intrinsic knowledge that you can leverage. But eventually, if you think about the evolution of how AI could go in the architectural space, we can start thinking about maybe large architectural models as opposed to large language

models. And then beyond that, some kind of true simulation of your environment, you know, some kind of system behavior modeling so that you can actually try different scenarios and maybe simulate different things, so you can look at the impact before making an actual decision.

So that's kind of where we see the evolution of this architectural AI going. I mean, we're not there yet, but, but that's actually the, the path forward for us as an AI community for architecture.

And Toufic, you know what I love about the notion of multi-agent systems is that ultimately, you know, in our exploration, you know, uh, when we try to think about what's the right way, what's the best way, you know, it's, it's amazing to, to be able to step back and say, "Well, listen, all this stuff that we're doing

as human teams isn't wrong."

Yeah.

It's, you know, we've perfected this art with very, you know, uh, you know, high aptitude and, and, and, and care. And so the process of design reviews is an important process, and it's a very effective process except that it doesn't leverage the right amounts of data, and we want it to kind of be able to leverage computational

intensity that's maybe higher. And that's what we're trying to do with multi-agent systems, isn't it? Just replicate human processes-

Absolutely

... effectively with AI, right?

Yeah. And that's just, yeah, taking that and, and, and expanding it at scale using these agents that can function like twenty-four/seven, you know, at scale, right? Yeah.

Absolutely. Absolutely. No, that's amazing. And look, the outcome is ROI-ranked explainable recommendations that truly understand your tech stack and objectives and act as that trusted advisor across your tech estate, proving clear trade-offs across cost, performance, risk, and time, and help prioritize the totality of the roadmap.

And what I think, what I'm really excited about, Toufic, in this context, is what we hear from customers.

Yeah.

What we hear from customers when they think about architecture copilots and they say that, you know, what, what, what's really gonna move the needle in such a dramatic way is when you go from, you know, even the best practices that are good and are really important to highlight, but they're a little bit more straightforward, like migrating from

GP2 to GP3.

Yeah.

To when you go and you really understand the intricacies of the overall architecture, and then the data pipeline can be streamlined for next efficiencies on reusability across a variety of applications or other architecture patterns that truly move cost and performance needles forward.

Yeah.

That's when you get so much bang for the buck.

And yeah. And it's tied to an ROI, and it's, uh, tied to impact, and there's a clear traceability, as we said before. So that's-- I mean, you take that to your board or to your executive meetings, whatever, and it's there.

There's, there's no controversy around it, right? That's perfect. Yeah. Um, you know, so that's good. Now, there's a third pillar. Remember, there's three pillars, Boris. We don't want the thing to topple down, you know? [laughs] [laughs]

The third pillar is having some kind of conversational architectural agent. This is where the world is moving to, this conversational mode of interacting with any system that you have.

So interacting with your architectural through a conversational agent is, is critical for us as an AI community to move forward. So it allows us to embed, you know, tailor fit designs, guidance, and expert QA, Q&A into the, into the workflow, right?

So this achieves two goals, you know. Allows developers and architects and, you know, anybody for that matter, uh, as a matter of fact, you know, to answer questions about the a- architecture, to ask questions, and then be able to get answers about their architecture.

And the second thing, you know, um, a- and it gives you, the developers and architects expert advice on optimizing and, and the refactoring of the architecture, so that having that knowledge in a conversational agent is really, really critical.

It also helps developers by g- you know, the next step would be by generating designs for their features, giving a set of requirements like a PRD, and knowing all the governance and controls and guidance that, say, the architecture team or the chief architect or, or whoever has put together.

They're built in into that agent, so whatever designs are given ... actually follow this guidance intrinsically, right? That's really, really critical, right?

Absolutely. And Toufic, you know, you said it earlier in the challenge category. I wanna tie that back here. And we talked a little-- a lot about the solutions impacting leadership and impacting ability to steer the ship, right?

The, the overall tech estate.

100%.

But the reality is, is that like we talked about, it's shift left, it's developers that are really steering that tech estate ultimately, and this is that point, right? How do you translate that top-level guidance, that visibility and strategic roadmapping to embed that across day-to-day workflows that developers are facing, and this is exactly it.

You know, I think the other thing that's really powerful here is that, you know, we wanna be able to see the architecture review process change, right?

Yeah.

You wanna change from having these architecture guild style, like once every two weeks kind of reviews that are very merit-worthy, but very hard to execute, to where that architecture review process is actually proactively baked in.

Like the beautiful thing about AI is that it allows us to get alignment by design, right? If AI is able to bake in that architecture guidance into every single piece of AI advice that it's giving to developers, isn't that the amazing answer, which is tailor fit for developers, the guidance already baked in, and we have that opportunity.

Yeah.

We can set the AI context, we can set the AI training and narrative based on the leadership's imperatives, but yet again, tailor fit to the specific context that the developers need answered for them, right?

Yeah. And this is how you scale your architecture guild or your enterprise architecture team, right? This is how they scale. They scale through the guidance they give to that AI, right?

Perfect. Yeah.

That's it.

Yeah.

And then we can change the paradigm, right? We can change the review role from being, you know, kind of trying to figure out if standards are being met, to knowing the standards are be-met b-by, by design and-

By default. By design. Yeah.

Yeah. And instead now, you know, we talk a lot about like, is AI gonna take our jobs, right? Instead to actually being able to do more. Now we're talking about productivity.

Now we're talking about strategic, uh, you know, multipliers because now instead of doing those mundane things of the past, AI is solving that, we can focus on strategy. How do we solve hard problems with our development teams?

How do we actually move the needle forward in a way we never had time before? Because we were always mired down into how do we just make it like fit the designs that, that we, the standards that we need, right?

Yeah. Exactly. Yeah.

So Touf, why don't you tell us a little more about how you bring this all together in this context?

So here's how it works. I mean, in our minds at least, end-to-end, right? The first step is to ingest and understand these messy systems, right? So you're getting data from everywhere.

Your systems are messy. Every system is messy. I mean, if you say your system is not messy, I don't think it's true. So you take that data and you normalize it to a live model, this digital twin that we talk about.

So now you have it normalized in a, in a, in a way that you can look at, you can introspect, you can, you can navigate, and so on. So, uh, and so, so having that.

So now that you have that, the second step is to kind of align yourself and, and have some kind of align and advise strategy. So you have your goals, you have your requirements, you have your context as a company, right?

You know, my ideal in this industry. My, uh, I'm in a, in a hypergrowth phase or what have you. So all these things together come in together, and then what you need is a s-- is a ranked recommendation set with some projected impact on cost, performance, ROI, whatever metric that you want that's really important for you as

a company, as your context, right? So that's the second thing. The third thing is, you know, having some kind of guideline, as we were just talking about, intrinsic, you know, intrinsic governance into these guidelines, these, these designs.

So generate designs, answer what if in real time, and enforce standards in the workflow. You don't want your developers or your architects to go to another tool or do something else and then come back.

It's part, becomes part of the workflow, right? And then, you know,

you know, how do you manage things? You, you can't, you can't manage what you don't measure, right? So eventually the last step is be able to track these decisions, verify outcomes, and then continuously improve on it.

So these are kind of the, the four steps that we see as getting to this changing the paradigm of how architecture is done.

Absolutely. And Touf, you know, it's funny. Uh, I, I, I know your, your great way with jokes, but you know, ready, fire, aim, right? I mean, in the context of ready, fire, aim, you know, isn't the right answer then ultimately if this is the way to aim, then doesn't this ultimately, you know, seamlessly get interconnected to our

coding copilots so then you can fire? You can aim with, with an architecture copilot, and then right away, right from there you fire with the coding copilots-

That's a great-

And now you've hit productivity, right?

Absolutely. That's a great concept. I can see a world where, you know, the agents, the architecture agents are talking to the coding agents, right?

100%.

And you're just there to guide them, make sure they're okay, they're doing the right thing, to correc-correct course and so on, and give them the directives, right? Yeah, that's coming.

Absolutely. Absolutely. So you know, at the end of the day, you know, the, what I think we see is a hub for architecture and tech decision-making being a really essential part of the software development cycle for these, for these kind of, you know, kind of aim imperatives, right?

It's a hub that transforms how companies plan, build, evolve their tech estate, and then execute software on the back of it, not just writing more lines of code for the sake of it, right?

It unlocks org-wide clarity and faster decision cycles, ability to strategically roadmap so that your roadmaps are truly tied to highest impacts on your business objectives.

Cool.

Fully equipping the tech org to execute with expertise. True shift left enablement and, and outcomes that don't just, you know, scale but reduce quality, that scale and dramatically improve productivity across the board with guidance baked in, and reframes copilots really from productivity tools to, to yet a new dimension.

You know, productivity is nice, but yet a new dimension. Strategic levers for the business. We all know that tech driven is the paradigm for how we're moving industry forward.

Well, this is a true new, new frontier- For how we can move, um, things forward even further competitively, the from competitive advantage perspective, stay best in class with architecture copilot setting, uh, setting us up for to have two strategic levers in our tech stacks.

Absolutely.

So the companies that get this right, I do believe that will be the ones that stay modern, agile, and ahead, and others that don't are gonna be buried in legacy and debt, just like we're seeing with coding copilots.

Companies that are not embracing it fast enough, finding themselves o-on the outside, uh, looking in.

We are, we are as an example, right? We're fully on with the coding copilots, and it's helping us a lot. We've written, you know, Boris and I have written, and the team have written some LinkedIn articles and blog posts about that, how effective it's been for us.

Absolutely, yeah.

Amazing. Well, and so just to wrap this up, you know, to, uh, quickly-

Okay

... where, where should, where should leaders start?

Yeah. Well, do everything at the same time. Or actually, no. [laughs] You start small and, like, scale little by little deliberately. So for example, pick a portfolio area and s- get visibility in that portfolio area, like build, you know, get, get that, you know, digital twin built on that particular area, generate recommendations in that particular, uh, start small,

tied to business outcomes, to specific business outcomes in that area, and then start piloting some autonomous guidance with one team, you know? You don't wanna do this throughout the whole company all of the time.

Right.

You do it step by step, right? And then scale little by little to the full hub once you've gotten ROI and you've proven that this tool, this new tool, because there's gonna be maybe some resistance at first or some skepticism of course.

I mean, architects, CTOs, developers, all skeptics by nature, right? So prove out the ROI first before you start scaling to the, to the full hub. That's kind of the, you know, start small and scale up to it.

The bottom line, architecture copilots are where ROI is gonna be won or lost, and the question isn't whether you'll adopt one, but whether you'll be early or late.

Yes.

And if this resonates and you wanna see what an architecture copilot, copilot would look like on your stack, reach out and we'll walk you through how to best pursue this from our lens and, uh, be able to impart how you can do it on your own or working with us at Cat.io.

You can visit catio.tech to connect with us or reach us out at, go to [REDACTED:email_address] and ask how your team can adopt an architecture copilot for your org. We, we'd love to be a part of your journey.

Absolutely.

Thanks everyone for joining us today, uh, for this session. It was hopefully informative for you and we're, uh, we're delighted that you've given us a chance to, to, to tell you more about this, and we look forward to working with you shortly.

Sure.