AI Engineer
← All organizations

AI inference infrastructure and model optimization

OctoAI

OctoAI built infrastructure for developers to run, customize and scale AI applications. Its hosted inference APIs supported language models, while OctoStack brought model serving, customization and asset management into customer-controlled cloud and on-premises environments. This let businesses deploy models alongside their own data. The company identified Apate.ai, Otherside AI, Latitude Games and Capitol AI as platform users; Apate.ai used OctoStack to run multilingual conversational models for combating telephone scams.

Founded in 2019 as OctoML, the company brought together Luis Ceze, Tianqi Chen, Thierry Moreau, Jared Roesch and Jason Knight. Co-founder and CEO Ceze now serves as NVIDIA’s VP of AI Systems Software. Its technical foundation was Apache TVM, a compiler project that predates the company. TVM combines graph and operator optimizations with learned cost models to search for efficient code across different hardware targets. OctoML commercialized that work to simplify model optimization and deployment across CPUs, GPUs and specialized accelerators.

OctoML closed a $15 million Series A in 2020, led by Amplify Partners with participation from seed investor Madrona. NVIDIA subsequently completed its acquisition of OctoAI. OctoAI’s commercial service wind-down took effect on October 31, 2024; its independent services are shut down, and Weaviate has deprecated and disabled the associated integrations.

octo.ai ↗

1 talk

Newest first

2 speakers at AIE

Affiliations reflect their AIE appearances, not necessarily current employment.

Thierry MoreauNVIDIA engineer and co-founder of OctoAIPedro Torruella

Messages from the stage

From fine-tuning to deployment assets

The workshop demonstrates OpenPipe fine-tuning and model-weight export, then walks through deploying checkpoints and LoRA assets with the OctoAI CLI. A LoRA upload issue surfaces a practical obstacle in the demonstration.

Affiliations reflect each recorded session, not necessarily current employment. The workshop's reported cost comparison is specific to the demonstration and does not establish general savings or performance.

Company sources · checked 2026-08-28