AI Engineer World's Fair 2026
Research to Reality: Bringing frontier ML research to production
Read the talk
Bringing Frontier ML Research into Production
Higharc’s approach connects research documentation, a modular Python monorepo, and deliberate pull-request decomposition so specialists can turn working ML prototypes into production services.
From a talk by Vaidas Razgaitis
Before you start: Familiarity with Python APIs, microservices, and pull-request review will help you follow the architecture and workflow.
From a sketched floor plan to production software
A hand-sketched floor plan has to become structured data before software can work with it. At Higharc, computer vision parses those sketches into an internal data model. Vaidas Razgaitis, a senior research engineer on the company’s Labs team, describes a research remit that extends from spatial reasoning to agents that guide users through product experiences. Custom transformers and diffusion models for image generation round out a toolkit shaped by the multidisciplinary demands of home building.
Turning those capabilities into production features requires people with different expertise to work on the same system. Platform, infrastructure, and backend engineers know how to build robust software, but may not know the research methods behind computer vision or training language models. ML researchers can combine recent papers into novel features, but may not have been responsible for production APIs. A working prototype therefore leaves a substantial handoff problem: how can each group contribute without having to acquire the other group’s entire specialty?
The handoff is a systems and process problem. Higharc organizes it around three connected decisions: make the research understandable to incoming engineers and product managers; give it a clear destination in the codebase; and design how to decompose the prototype into production-quality contributions. The document explains what exists, the repository establishes where it belongs, and the decomposition plan connects the two.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the prototype legible
As engineering teams grow, written designs let people align before they build. Razgaitis draws on The Pragmatic Engineer’s discussion of that practice and applies it to research through a required research prototype taxonomy document, or RPT. It is a technical design document with additions that make ML work understandable to people joining the project. Higharc writes these documents in Notion; the medium matters less than making the design explicit.
Start with the domain context. In home building, a prototype might use a parti diagram, a graph representing circulation through a home, embeddings, or a latent-space representation. Those representations cannot remain unexplained vocabulary. Razgaitis’s test is to imagine a software engineer newly hired from JPMorgan: what architectural language and data representations would that person need explained before contributing? Once that context is established, state the business goal—why solving this problem matters and what value the ML tool creates.
The rest of the document exposes the engineering boundaries:
- Type safety: Define the contract between the core-product repository and the ML repository, including how types are shared and kept in sync.
- Persistence: Identify databases and other storage requirements, and record how far the prototype has progressed. Higharc prefers researchers not to spend excessive time here; unfinished persistence work can be a useful first contribution for an incoming software engineer.
- Architecture: Describe the system’s anatomy. Is it one workflow, a chain of workflows, or a system that makes external LLM calls?
- Decomposition: Explain how the prototype will be broken apart and merged into the production codebase.
Together with domain context and business value, these boundaries let collaborators see both what the research has established and where engineering work remains.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give research an independent service boundary
The documented prototype needs a destination that can accommodate its continued evolution. Higharc maintains an all-Python ML monorepo separate from its core-product repository. Inside it, research initiatives become isolated, decoupled microservices. Data-driven entity prediction, for example, uses a custom transformer that can evolve independently of another research initiative. Razgaitis describes an approximately one-to-one researcher-to-microservice ratio in this setup.
Web-application clients send requests to a gateway, which guards those requests and routes them to the appropriate microservice. The services share a Docker bridge network. The gateway is the described client entry point; sharing a bridge does not itself enforce gateway-only communication between containers. This arrangement separates the product-facing route into the ML system from the individual services that researchers develop.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Repeat the service structure, not the research
Inside each microservice, Higharc uses a simple layered architecture: API, business logic, and data. Clearly documented specifications also help coding agents navigate the repository and assist researchers. The services layer holds the core business logic, which may call an external foundation model or use Higharc’s own model weights pulled into the build through CI/CD. Controllers wrap that logic, and API routers expose the controllers through standalone FastAPI applications.
A small Python example makes that boundary concrete. Here, an entity-prediction implementation is supplied to the application rather than embedded in its HTTP handler. The request and response models define the API shape, the controller translates between that shape and the predictor, and the predictor remains responsible for the model-specific work:
python
from typing import Protocol
from fastapi import APIRouter, FastAPI
from pydantic import BaseModel
class PredictionRequest(BaseModel):
entity_ids: list[str]
class PredictionResponse(BaseModel):
predicted_entity_ids: list[str]
class EntityPredictor(Protocol):
def predict(self, entity_ids: list[str]) -> list[str]: ...
class PredictionController:
def __init__(self, predictor: EntityPredictor) -> None:
self.predictor = predictor
def predict(self, request: PredictionRequest) -> PredictionResponse:
result = self.predictor.predict(request.entity_ids)
return PredictionResponse(predicted_entity_ids=result)
def create_app(predictor: EntityPredictor) -> FastAPI:
controller = PredictionController(predictor)
router = APIRouter()
@router.post("/predict", response_model=PredictionResponse)
def predict(request: PredictionRequest) -> PredictionResponse:
return controller.predict(request)
app = FastAPI()
app.include_router(router)
return app
This illustrates the layering with a minimal entity-ID contract; Higharc’s actual prediction schema is not specified in the talk. The same separation allows API work and model work to proceed in distinct parts of a service.
Clients still reach these standalone applications through the gateway. Each service’s root directory contains its metadata, build instructions, Dockerfile, project dependencies, and Poetry or uv lockfiles as appropriate. Razgaitis maps the structure of three production microservices to compare their common architectural backbone. A consistent shape makes it easier to see whether a new service is developing along the same engineering conventions.
Shared tooling supports those services across the monorepo. GitHub Actions handle build and deployment automation, alongside automated tests, linting, formatting, and type checks. Researchers also use Jupyter notebooks on Modal for GPU compute, additional ML studies, and an internal CLI. These tools support the work of packaging and serving microservices; the RPT remains the handoff document, and the monorepo remains the destination.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Design the path from prototype to reviewed code
A mapped-out prototype and a suitable repository still leave a design decision: how should the prototype enter that repository? Higharc treats decomposition as an explicit design problem. For a platform-wide agent feature, the team studied which axes to split the project along and what dependency graph those pieces would form. The task is to find coherent changes that can be understood and reviewed separately, while preserving the dependencies between them.
Higharc uses Graphite stacked diffs to turn proven monolithic prototypes into smaller pull requests. The practical sequence is:
- Choose the slices and dependencies. Use the decomposition design to establish which changes build on which others.
- Represent the work as a stack of PRs. A later change can remain in development while an earlier one is under review.
- Route each slice to the right specialist. Ask subject-matter experts across the organization to review the tightly scoped changes that need their expertise.
This supports asynchronous review: the author can continue working farther up the stack while a domain specialist reviews another PR.
The decomposition plan draws directly on the taxonomy document. Once the layers, architecture, persistence needs, and type contracts are visible, they can inform the boundaries between PRs. Research legibility becomes reviewability: the initial description of the system helps determine how engineering specialists can safely change and assess it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Locate the bottleneck before changing the process
The three focus areas also provide a diagnostic sequence. Begin with research legibility: when product staff, software engineers, and AI engineers join an initiative, can they tell where to concentrate their effort? Can they identify concrete tasks that move the prototype toward production? If not, revisit how the research is explained and handed off.
Next, inspect the receiving codebase. New code should have an obvious home, with existing templates, frameworks, and patterns that contributors can follow. Repeatedly fighting old abstractions is a different signal: the team may have outgrown the repository’s architecture. In that case, clearer documentation alone will not remove the constraints imposed by the destination.
Finally, examine decomposition and delivery. Can the team consistently estimate timelines and delivery dates for moving research into the repository? Is it clear which subject-matter experts should review and help productionize the work? Difficulty answering these questions may point upstream—to research coordination, the handoff, or the codebase that must host the result. The place where delivery stalls is not necessarily the place where the underlying problem began.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
A practical guide to organizing FastAPI applications across Python modules and composing API routers.
Explains bridge-network connectivity, container name resolution, port publishing, and user-defined networks.
A 2024 guide demonstrating how to launch a Jupyter notebook with GPU compute on Modal.
Explains how dependent branches and smaller pull requests support incremental development and focused code review.
Further reading
Gergely Orosz explains how written engineering proposals, explicit reviewers, and evolving templates help teams coordinate as they grow.
Read the complete timestamped transcript
- 0:01
Hey, I'm Vaidas, and I'm a senior research engineer at Higharc on our labs team. So at Higharc, our labs team is basically our research and development arm, where we have machine learning researchers who kind of explore the frontier of AI ML, uh, and figure out ways to apply it to home building. [upbeat music]
- 0:27
Now, because we're in home building and we're in spatial reasoning, uh, we pretty much end up needing to use a lot of what's out there in AI. So computer vision to scan hand-sketched floor plans and parse them into our internal data model, um, reasoning agents to kinda carry the user through these agentic experiences.
- 0:47
We have custom transformers. Uh, we use diffusion models for image gen, uh, and basically a lot of kind of what's out there in AI ML, uh, we end up needing to use because of the kind of multidisciplinary nature of our product.
- 1:06
So hopefully that video gives you some idea of the research areas that we focus on that I've mapped out here, and that kinda leads us to, uh, the problem, which is in this, in this challenge of getting frontier research into production, uh, we need to start working with software engineers, let's say platform engineers, infrastructure engineers, back-end engineers,
- 1:30
who are very familiar with building robust and production-grade code, but are likely not familiar with the methodology and research in computer vision, in training your own LLMs, uh, and even some top secret, uh, topics that I can't reveal here. [chuckles]
- 1:49
Now, you kinda have the flip side problem with our, uh, ML researchers, who are very up to date with the latest papers and can, uh, pull together these concepts in novel and creative ways to develop new features.
- 2:02
Uh, but they have not really worked, uh, as software engineers typically, uh, where they've been responsible for production-grade, uh, APIs. So that's kinda what I wanna get into, is this, uh, this handoff, this, this baton pass of how do we facilitate that, and how do we do it productively?
- 2:22
And we look at this basically as a systems and process problem, and I want to kinda zero in on three main focus areas that, uh, you can use to, uh, improve, um, this, uh, the velocity of teams that, that are bringing research into production.
- 2:42
So the first thing we'll look at is research legibility. So let's say you have a, uh, ML researcher who's produced a, a prototype concept. How can they map that out, um, and make it digestible and understandable for these different software engineers and product managers who are gonna be jumping into the project?
- 3:02
Uh, the second is how to structure your code base. So we use a monorepo. Um, how do you arrange your code and modularize it so that it's ready to receive these new prototype concepts, um, and turn them around and stand them up quickly?
- 3:18
And then the third thing we'll look at is basically, you know, making the jump from a mapped out, uh, proven out research prototype that's gonna be going into this, uh, repo.
- 3:29
How do you decompose that prototype and stand it up on best practices in software engineering?
- 3:37
So taking a look at the first step, um, there's a really good article that, uh, that I wanna link, uh, a blog from the, The Pragmatic Engineer, where he talks about, uh, software engineering teams, uh, as they grow and start to scale, how important it is to write out technical design documents, uh, requests for comment, uh, whatever
- 3:58
you wanna call it, uh, specifications before building software, uh, as a way to align teams. So we have a very analogous document, um, that we require from, uh, all research prototypes that we call the research, uh, prototype taxonomy document.
- 4:16
So it's really just a technical design document from software engineering, uh, with some specific twists, uh, that, um, make it more, uh, digestible given its, its nature in machine learning.
- 4:30
So the first thing in that document, uh, we use Notion for this, but obviously any written document will work, is that we start with the kinda domain context and, like, what are the domain-specific, uh, you know...
- 4:41
We're in the architectural domain in home building, so what are these kind of novel, uh, ways to represent data? Maybe it's a Part T diagram. Maybe it's a graph to, to represent the kind of circulation graph through a home.
- 4:55
Maybe it's, um, embedding models or latent space representations. Uh, I like to say kind of picture a software engineer who just-- we just hired from JPMorgan. What are the kinda, uh, specific lingos and, and data representations that they might need to know before jumping into this project?
- 5:14
The second is kinda mapping out the business goal, right? What's the-- why does solving this problem matter, and what's the value, uh, in this ML tool? And then the four, uh, re-remaining parts of this document, uh, are, are pretty kinda conventional software engineering principles you might see in a TDD.
- 5:32
So, uh, the, the type safety, so we'll see later what our, um, machine learning repo looks like. But what is the type contract between our core product repository and this, uh, machine learning repo?
- 5:47
How are those types shared, and how do they stay in sync? Um, then kinda mapping out the, the persistence layer. Uh, is there a database? Uh, this is an area where, uh, we think it's probably best, uh, not to have our researchers spend too much time Uh, in the persistence layer, and just map out how far they
- 6:07
got, and this is a great first entry point once we start bringing in software engineering, uh, help on the project. Uh, then kind of mapping out the, the overall system architecture.
- 6:18
Um, is this a workflow? Is this a chaining of workflows? Are there external LLM calls? Like, what is the anatomy? What is the kinda taxonomy of this, uh, research prototype?
- 6:30
And lastly, how are we gonna merge this? How are we gonna decompose it? Um, and we'll get into that more, uh, later as well.
- 6:39
So going back to these kind of three, uh, levers we can pull, the first is kinda that research project, uh, taxonomy document. Uh, the second, uh, I wanna get into are, uh, how we structured our code, um, and how we, um, serve our existing, uh, features.
- 6:57
So we basically have a, a separate, um, repository from our core product repo, and this is, uh, all Python-based. Uh, right? It's all AI, ML stuff. Um, and it's basically a monorepo of, uh, cleanly isolated and fully decoupled microservices.
- 7:18
And so let's say data-driven entity prediction is our, uh, custom transformer model. So that is able to opt to, um, kinda grow and be iterated on and be fully decoupled from a different research initiative.
- 7:34
Uh, and it's pretty much a one-to-one, uh, researcher to microservice, uh, ratio.
- 7:41
So we find that works really well. We have a kinda gateway that, that guards requests, um, and it's all in one Docker bridge network, so the core, um, consumers are clients in our, in our web application make kinda API calls to this gateway, which then routes them to the appropriate microservice.
- 7:59
Now, uh, looking a bit closer at these, uh, individual microservices, uh, we tend to build them in a pretty simple layered architecture, right? There's an API layer, the business logic, and the data layer.
- 8:12
And then we tend to have, like, some really, um,
- 8:16
really cleanly documented specs, uh, so that agents can navigate these repositories and help accelerate our ML researchers as much as possible. So yeah, kinda taking a, a another way to look at this layered architecture, we have kinda the core business logic at the services layer.
- 8:33
Uh, that might, you know, make external, uh, LLM calls to, to foundation models, or we might need to pull in our own, uh, machine learning model weights in CI/CD.
- 8:45
Then we wrap that business logic with controllers, um, then we put API routers around those and expose them in FastAPI applications. And each of those microservices is a standalone application.
- 8:57
And like I said on the last slide, it's not really the client that makes a direct call to this microservice. It kinda goes to that, uh, gateway first, which routes it to the appropriate microservice.
- 9:08
And then within each of those microservices, um, in the root rep- in the root directory, we have essentially metadata, build instructions, a Docker file on how to, how to build this application, uh, the, the kind of project dependencies, and then Poetry or, or uv, uh, lockfiles, um, as needed.
- 9:27
And then so I like to kinda map out this, um, anatomical arrangement of our kinda three microservices that we have into production. And we can kinda see these, these trends and, and consistent skeletal backbones of these projects, uh, and it's very easy to map them out and make sure that they're growing, um, along best practices in software
- 9:48
engineering. So that's basically it for our, for our monorepo. We have these kind of microservices, and then, uh, sharing basically, like, underneath all of that in the repository, we have some GitHub actions to, to build and deploy our automated d- test suites and kinda linting, formatting, and type checks, all the stuff you'd, you'd kind of expect.
- 10:09
Uh, we have some Jupyter notebooks that run on, on modal for GPU compute, um, as well as some additional ML studies. But really the way we look at it is we have this kinda tooling layer and even this kinda fun CLI, but these all just support our ML engineers in bundling up these microservices that we, uh, serve
- 10:30
in production. So that's kind of the, the RPT is the, is the handoff document between an ML researcher and additional software engineering, uh, talent. The monorepo is, is where that, um, research project is ultimately going.
- 10:48
And then the only remaining step is, well, how do we get there, right? How do we jump from one to two? And the third lever I wanna talk about is the kinda decomposition, uh, and PR review plan.
- 11:01
Uh, and we look at that really as a design problem, where we need to kind of, um, figure out how to slice and dice a, a large research, uh, monolithic prototype.
- 11:12
So here are just some images of, of, um, an, uh, platform-wide agent feature where, you know, we really studied what axes to slice, uh, and dice these projects up on, what that dependency graph would look like.
- 11:26
And then we use Graphite, um, for kinda stacked diffs to then, uh, decompose these large monolithic prototypes that have been proven out, um, and then get the right eyes on, on review to make sure that these are ready for production.
- 11:40
Uh, we really like Graphite because it allows for asynchronous review, right? I could be working on a PR all the way up here while a domain specialist is still reviewing a different PR.
- 11:51
And once we've kinda decomposed these PRs, uh, thoughtfully, we can start tapping, uh, subject matter experts throughout the organization on the specific slices or smaller tightly, uh, decomposed PRs, uh, that they need to look at.
- 12:10
And so again, there's, there's a lot of overlap between this initial research project, uh, taxonomy document, where once we've mapped out these layers, these arc- the architecture, what kind of persistence there is in the types, that tends to inform, uh, your decomposition strategy on how to bring it into the, into the monorepo.
- 12:31
So to wrap things up, I want to revisit these kind of three focus areas that you can use to evaluate how well your team is bringing research into production.
- 12:43
And I wanna talk about kind of some, some diagnostic frameworks to understand if you need to spend a bit more time and attention in any of these three. So in the first step with your research legibility, as your team starts to staff research initiatives with product people, with software engineers, with AI engineers, is it obvious where they
- 13:07
should concentrate their efforts? And is it clear, uh, what kind of tasks they need to pluck off to bring this prototype into production? If there's ambiguity there, you may need to spend some time revisiting this process.
- 13:23
In step two, your code repository. As you start getting ready to bring prototypes into your production grade code base, is it clear where these buckets lie to put in new code?
- 13:37
Are there templates and existing frameworks and patterns that you can mimic? Or is it possible that maybe you've started to outgrow that code base and that system architecture, and every time you bring in new research concepts, you're fighting these old abstractions and having headaches from limitations of your repository.
- 13:59
And then lastly, in the decomposition phase, are you able to consistently estimate the timelines and delivery dates for moving research concepts into your repository? Is it clear which subject matter experts you should tap for review and for productionizing this research?
- 14:21
Uh, and if you're having issues here, it probably points to upstream issues, either in, uh, how this research is being coordinated and handing off, or perhaps the code base that's hosting it.
- 14:35
Uh, anyway, that's about all I've got. I hope this was helpful. I love, uh, thinking about ways how our team can speed up how quickly we can bring research concepts into production.
- 14:47
And if you made it this whole way, you're probably interested in that stuff too, and I would love to compare notes, uh, and trade ideas anytime. Thanks again.