Benjamin Clavié

Benjamin Clavié works in research and development at Mixedbread, where he builds multimodal retrieval systems. He created RAGatouille, a toolkit that makes ColBERT-style retrieval easier for application developers to use, and initiated ModernBERT, which he co-led with Benjamin Warner and Antoine Chaffin. His work combines better retrieval models with the software and efficiency improvements needed to put them into applications.

From applied machine learning to open research

Clavié is a French machine-learning practitioner based in Tokyo, specializing in natural-language processing and information retrieval. He earned a research master’s degree in artificial intelligence at the University of Edinburgh and spent roughly five years in applied data science, often as a consultant.

His earlier legal-technology work at Jus Mundi and LexisNexis included contributions to French legal language modeling with transformers and their deployment in 2018. He also explored support-vector machines as strong baselines for simpler legal tasks and ULMFiT-style models. That experience encompassed both adopting new techniques and testing how much established methods could accomplish.

At Bright Network, he worked on detecting skills in job advertisements and matching them to the European ESCO skills database. His team combined language-model-generated synthetic training data for first-stage retrievers with language-model reranking, reporting improvements over previous approaches. Karel d’Oosterlinck subsequently refined the approach using DSPy. The work brought together two distinct parts of search: finding plausible candidates and using a stronger model to order them.

RAGatouille extended Clavié’s retrieval work to a broader developer audience. Its simple interface made ColBERT usable without requiring developers to navigate the research implementation first. Clavié credits Omar Khattab with creating ColBERT; his contribution was making that research easier to build upon.

In February 2024, he announced plans to join Answer.AI on March 4. RAGatouille’s release had prompted conversations about continuing consulting, accepting venture funding, or joining a research organization. He wanted room to explore ideas, allow some to fail, and develop an ecosystem—priorities he felt could conflict with an investor’s pressure to find product-market fit quickly. Conversations with Jeremy Howard, following an introduction from Khattab, made Answer.AI feel like a close match for that approach.

His open research expanded into Japanese retrieval, compact English retrievers, and encoder models. At Mixedbread, he now divides his work between internal models and contributions to open research.

Models, efficiency, and tools for experimentation

Clavié’s research and software projects address complementary obstacles to useful search: model quality, storage costs, training choices, and the difficulty of trying unfamiliar methods.

• Reducing multi-vector storage. Multi-vector retrieval systems store many vectors, creating a substantial storage burden. Clavié helped introduce token pooling, a clustering-based method that reduces the number of stored vectors without retraining. His account reports reductions of 50–66% without a retrieval-quality loss in the work described, and notes that other teams have applied the method to multimodal retrieval.

• Improving retriever training. JaColBERT and JaColBERTv2 developed Japanese late-interaction retrieval. JaColBERTv2.5 extended that effort into a study of how to train better retrievers, producing a roughly 130-million-parameter Japanese model and informing a compact, 33-million-parameter English ColBERT model. Clavié describes the latter as competitive with models exceeding 500 million parameters. His work on ColBERT’s projection layers also investigates the learning properties of MaxSim, seeking improvements through a better understanding of how the model works.

• Revisiting encoder models. ModernBERT, which Clavié started and co-led with Warner and Chaffin, updated the BERT-like encoder approach through modern architecture, longer context, and updated training data. He presents it as evidence that encoders still have room to improve. His exploratory study It’s all in the [MASK] examines a related possibility: encoders trained on modern data mixtures can exhibit some instruction-following behavior without autoregressive generation, even at small parameter counts.

• Making retrieval approachable. RAGatouille provides a simple interface for ColBERT-style retrieval. Byaldi lets developers try multimodal retrieval with ColPali, including retrieving PDF content from text queries for use by a language model. The rerankers library gives commonly used ranking methods a shared interface, reducing the work of installing separate implementations and changing APIs whenever a developer wants to compare approaches. Reranking uses a stronger model to reorder candidates produced by an initial retriever.

• Simplifying clustering. With Warner, Clavié built fastkmeans, a focused K-Means library using PyTorch with optional Triton acceleration and CPU compatibility. It addresses installation and dependency difficulties as well as speed, making clustering easier to use across environments.

Machine learning as infrastructure

Clavié’s 2024 essay on joining Answer.AI explains the reasoning behind his developer tools: application developers should be able to use useful machine-learning techniques without first acquiring every specialist’s background. He compares this to Redis. A developer can benefit from fast key-value storage before studying memory allocation or distributed systems; an approachable first experience can then lead to deeper understanding and customization.

RAGatouille applies that idea to retrieval. ColBERT was already powerful and familiar to search researchers, but developers building retrieval-augmented generation applications faced both an implementation hurdle and a knowledge hurdle: understanding why they should try it. Clavié sees toolkits as a way to connect research flexibility with an interface people can learn and use.

He develops that position further in the Mixedbread research vision co-authored with Aamir Shakir. They argue that search gives people and artificial agents access to information that is impractical to memorize, such as changing catalogues or newly issued reports. Their research priorities include understanding retrieval mechanisms and making improvements usable at realistic serving costs. Latency, hardware requirements, compression, and indexing therefore belong alongside model quality in deciding what is worth pursuing.

Their proposed balance lets research feed Mixedbread’s product and product experience inform research, while sharing selected findings, tools, and models publicly. Clavié also supports the wider retrieval community as the main organizer of the Late Interaction Workshop series. He describes himself as affiliated with Tokyo’s National Institute of Informatics under Makoto P. Kato and the Kasys lab, while distinguishing that affiliation from being an academic. These activities extend his model and software work into a community where retrieval methods can be studied, tested, and built upon.