# Cohere for VPs of AI

Vivek Muppalla, Cohere

AI Engineer World's Fair 2024

[Talk page](https://ai.engineer/talks/u3NofYYstaY-cohere-vps-ai) · [Video](https://www.youtube.com/watch?v=u3NofYYstaY)

## Cohere for VPs of AI

Enterprise AI depends on more than model quality: retrieval, customization, deployment control and operating cost determine whether a useful prototype can become a production system.

Basic familiarity with language models and document search will help; retrieval-augmented generation and reranking are explained as they appear.

### What does an enterprise need from a model?

[0:29](https://www.youtube.com/watch?v=u3NofYYstaY&t=29s)

How do you put an AI model to work on business data while keeping control of where it runs? That is the starting problem for Vivek Muppalla’s introduction to Cohere: trustworthy enterprise models, developed with partners, that can deploy across cloud providers rather than requiring customers to commit to one infrastructure ecosystem.

The research background helps explain the product emphasis. Muppalla points to cofounder Aidan Gomez’s coauthorship of [Attention Is All You Need](https://arxiv.org/abs/1706.03762), Phil’s background at DeepMind and Oxford, Nils’s expertise in BERT, and Patrick’s coauthorship of [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401). Generation and retrieval are both central to the resulting lineup.

The generative family has two main choices: **Command R**, presented as the inexpensive enterprise workhorse, and **Command R+**, a larger model for more demanding reasoning, tool use and retrieval-augmented generation, or **RAG**. The retrieval family supplies embeddings and **Cohere Rerank**, which adds a relevance-ranking step to a retrieval pipeline.

| Component | Role in the application |
|---|---|
| Command R | General enterprise generation at lower cost |
| Command R+ | More complex reasoning, tool use and RAG |
| Embeddings | Retrieve candidates by semantic similarity |
| Rerank | Reorder retrieved candidates for the query |

This separates two decisions that are easy to conflate: which model should generate the answer, and which information should reach that model.

![Three-column slide groups Command R/R+ under generative models and Embed 3 and Rerank 3 under advanced retrieval models, with use cases and language support below.](https://ai.engineer/api/talk-essay-media/575e713634471e4386a468e29b47c473a26fb0018e1ace1e72724f7fd3ee210f)

Cohere’s model lineup: Command R/R+, Embed 3 and Rerank 3.

### Evaluate the workload, then design for its constraints

[2:28](https://www.youtube.com/watch?v=u3NofYYstaY&t=148s)

Cohere’s development principles begin with **enterprise-specific evaluation**. Academic benchmarks provide one view of model performance; the stated release goal is an evaluation suite built with partners around health, HR, finance and other customer workloads. Repeated evaluation across model versions is meant to reveal whether a release improves the tasks customers actually need.

The next principle is operating efficiency. The objective is not simply to produce the largest model, but to make the model useful, affordable and straightforward to run. Customization follows from the same practical concern: a broadly capable model may still lack the behavior or domain knowledge a particular enterprise needs. Muppalla describes domain adaptation by retraining a base model on enterprise data, including full retraining, and contrasts that with the then-available self-service option he describes as retraining the last few layers.

Data provenance and privacy form the fourth principle. Muppalla says Cohere applies enterprise standards to its training data, offers indemnification for IP claims, and does not use customer data to train its models. That assurance concerns Cohere’s own model training, distinct from customer-requested adaptation. It is also a historical assurance: the later [Enterprise Data Commitments](https://cohere.com/enterprise-data-commitments) distinguish SaaS training opt-out controls from private and third-party cloud deployments where prompts and generations are not sent to Cohere.

The fifth principle is deployment flexibility: major cloud providers, a customer’s own VPC, or on-premises infrastructure. Together, the five principles turn model selection into a system decision—task performance, operating efficiency, customization, data handling and the location of compute all have to fit the application.

![Five orange rows list model performance, efficiency and scalability, customization, data provenance and privacy, and deployment flexibility beside a green panel describing corresponding approaches.](https://ai.engineer/api/talk-essay-media/89ac92d2c17474024193e3e2852d1387804e329a96012e23d4531fee8ed77e77)

Five enterprise needs paired with Cohere’s approach.

### RAG citations and multilingual operating cost

[4:58](https://www.youtube.com/watch?v=u3NofYYstaY&t=298s)

The capability priorities follow the enterprise workloads: multilingual support, RAG and tool use for agentic applications. Muppalla names HotpotQA, Bamboogle and Berkeley Function Calling as evaluations where Cohere performs well. These are qualitative performance claims in the presentation, not a basis here for a numerical comparison between models.

A more concrete integration feature is **built-in RAG citations**. Cohere’s models and APIs can return citations with an answer, reducing the work required to connect generated statements to supplied sources. That does not remove the application’s responsibility to retrieve documents and construct the request; it moves citation generation into the model/API path instead of requiring developers to build that behavior themselves.

For multilingual applications, Muppalla cites strong FLORES performance and attributes lower operating costs partly to the tokenizer. Tokenization matters economically because the amount of text represented by a token affects the token volume processed for a workload. His practical objective is to let a customer use the same model across geographies while controlling total cost of ownership.

### Finding the right attention paper

[6:30](https://www.youtube.com/watch?v=u3NofYYstaY&t=390s)

The retrieval demonstration makes the pipeline problem tangible. After describing embedding quality on noisy data and low cost as product strengths, Muppalla introduces a search over arXiv papers: when was the attention paper associated with Aidan Gomez published? The query sounds simple, but the records combine titles, dates, author lists and full paper text. That mixture resembles enterprise data much more closely than a collection of uniformly formatted passages.

An embedding represents that mixed content in a vector space. A query asking for a particular author’s paper and its publication date may not align cleanly with the representation of the whole record. A semantically related result is not necessarily the precise document needed to answer the question.

**Reranking separates candidate discovery from final relevance ordering.** Cohere’s reranker is a cross encoder: it evaluates a query together with each candidate document and reorders the already retrieved set before the generator receives its context. The procedure is:

1. Retrieve candidate documents using the existing search system.
2. Score the candidates against the user’s query with the reranker.
3. Select the most relevant candidates for the generative model’s context.

The important boundary is the retrieved set. Reranking can improve its ordering; it cannot promote a document that the initial search never returned.

The demo compares three columns: lexical search using OpenSearch, embedding search, and Cohere Rerank. All can return results for the transformer-paper query, but the relevant document appears at different positions. In the displayed rerank column, *Attention Is All You Need* appears first. That ordering matters because a RAG application may pass only a small part of its retrieved set to the generator.

![Cohere-Rerank demo with a query above three result columns labeled Lexical Search (OpenSearch), Embeddings Search and Cohere Rerank. Attention Is All You Need appears first in the rerank column.](https://ai.engineer/api/talk-essay-media/5473029f3bf0a345e03563fecb8bf9d86c195db17c4a0a2d2c00c863ddf1d2ee)

The reranking demo compares lexical search, embedding search and Cohere Rerank results.

Reranking complements chunking rather than replacing it. Chunking determines the units available for retrieval; reranking helps choose among the resulting candidates. A small Python function makes the handoff explicit: given the arXiv candidate records and scored candidate indices, preserve the records while selecting them in relevance order.

```python
from collections.abc import Mapping, Sequence
from typing import Any


def select_context(
    candidates: Sequence[Mapping[str, Any]],
    scored_indices: Sequence[tuple[int, float]],
    limit: int,
) -> list[dict[str, Any]]:
    if limit < 0:
        raise ValueError("limit must be nonnegative")

    ranked = sorted(scored_indices, key=lambda item: item[1], reverse=True)
    return [dict(candidates[index]) for index, _ in ranked[:limit]]
```

Here, `scored_indices` contains the reranker’s candidate indices and relevance scores. Keeping each selected record intact preserves its title, authors, date and text for the answer-generation step. The function performs selection; the cross encoder supplies the relevance judgment.

Better selection also changes the cost calculation. If a small set of relevant documents is sufficient, the generator needs fewer input tokens. Muppalla emphasizes input-token expense as a major cost driver, but whether it dominates—and whether reranking lowers the total bill—depends on the workload and the reranker’s own cost. The mechanism is narrower and more useful than a universal savings claim: **send less irrelevant context to the generator**.

### Putting the pipeline inside the enterprise

[9:32](https://www.youtube.com/watch?v=u3NofYYstaY&t=572s)

Deployment options extend from Cohere’s managed SaaS API to cloud AI services including SageMaker, Bedrock and OCI, alongside private cloud and on-premises deployments. Muppalla also describes compliance with standards customers request, without naming particular standards in this discussion. The architectural choice is where the model runs relative to the organization’s data, compute and security requirements.

Developer integration is the other half of that deployment story. Muppalla names LangChain and LlamaIndex integrations, then describes an open-source toolkit with connectors for ingesting data. The intended benefit is a shorter path from enterprise data sources to a working application while retaining control over the system being built.

### Partners, private fine-tuning and customer choice

[11:08](https://www.youtube.com/watch?v=u3NofYYstaY&t=668s)

The first audience question pairs two concerns: how do partnerships with Accenture and McKinsey work, and why do customers choose Cohere over alternatives? The first answer concerns the last mile between a general model and a deployed business application. Enterprises need customization, and established partners bring existing customer relationships and knowledge of the relevant business domains. Muppalla describes their role as co-developing products and helping close those implementation gaps.

For customer choice, his emphasis is **control over data and compute**. Private deployment is useful not only for inference but also for fine-tuning inside the customer’s own cloud environment, so enterprise data can remain within that ecosystem. He gives HR, healthcare and models adapted to proprietary codebases as examples of application categories where that control matters. These are descriptions of customer needs, rather than named case studies or quantified competitive wins.

### When should classification use a specialized model?

[13:27](https://www.youtube.com/watch?v=u3NofYYstaY&t=807s)

The next question asks whether the Classify endpoint is still the recommended way to build a text classifier. Muppalla answers through development stage and production scale rather than treating an endpoint as a universal recommendation. An off-the-shelf generative model offers a fast starting point; moving into production creates a reason to reassess whether a general model is adequate and economical.

| Situation | Decision to examine |
|---|---|
| Early prototype | Use a general model to get started quickly |
| Production workload | Compare quality, operating cost and maintenance |
| Very high-volume classification | Consider a purpose-built model |

Muppalla uses a hypothetical workload of tens of thousands of TPS to motivate a purpose-built classifier, not to report measured throughput. A heavier general model may serve several needs, while a specialized model can be a better fit for a narrow, repeated task. Both remain options; the decision depends on the workload and the cost of maintaining the solution.

A follow-up asks whether training such a classifier still produces a transformer model, just a more specialized one. Muppalla confirms that it does. Specialization here changes the model’s task and operating role; it does not imply abandoning the transformer architecture.

### A larger context window, with retrieval still doing useful work

[15:10](https://www.youtube.com/watch?v=u3NofYYstaY&t=910s)

The final question returns to a concrete limitation: an audience member had evaluated Cohere in the previous year and found the input context limits small. Muppalla says the then-current generation offers 128K-token input context windows, with further expansion under investigation.

His assessment is that this capacity should suffice for most applications at that point. Capacity, however, is not a guarantee of answer quality across the entire window. The earlier retrieval demonstration remains relevant: having room for more documents does not make every document useful, and selecting the right context still affects what the model can answer and what the application costs to run.

## Resources

- [Attention Is All You Need](https://arxiv.org/abs/1706.03762) — The original Transformer paper, coauthored by Aidan N. Gomez and first submitted to arXiv on June 12, 2017.
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401) — Patrick Lewis and colleagues describe generation that combines a pretrained model with retrieved Wikipedia passages.
- [Command R's historical release note](https://docs.cohere.com/changelog/command-r-retrieval-augmented-generation-at-production-scale) — The 2024 release note describes Command R's RAG and tool-use positioning and 128k context capacity.
- [Cohere Toolkit](https://github.com/cohere-ai/cohere-toolkit) — An archived RAG application repository with configurable retrieval, model providers and data integrations. Archived May 14, 2026.
- [Build a RAG pipeline with Chat, Embed and Rerank](https://docs.cohere.com/docs/rag-complete-example) — A current Python walkthrough of retrieval, reranking, document-grounded generation and returned citations.
- [Cohere model and API deprecations](https://docs.cohere.com/docs/deprecations) — Lifecycle notices covering historical Command models, Classify, connectors and fine-tuning capabilities.
- [Enterprise Data Commitments](https://cohere.com/enterprise-data-commitments) — Cohere's deployment-specific data handling and SaaS training controls, updated December 5, 2025.

## Slides

### Presenting sponsor: Microsoft

[0:05](https://www.youtube.com/watch?v=u3NofYYstaY&t=5s) [Microsoft logo beneath the heading Presenting Sponsor.](https://assets.ai.engineer/sites/aiecode2025/sha256/dc7d5b1bbf1b6fbe0fc5a21c87936d440d1f4cdfb2a62bdf8ec07c44a1e30faa.webp)

PRESENTING SPONSOR

Microsoft

### Innovation and platinum sponsors

AWS is the innovation sponsor; MongoDB, Google Cloud and Neo4j are platinum sponsors.

[0:10](https://www.youtube.com/watch?v=u3NofYYstaY&t=10s) [AWS is the innovation sponsor; MongoDB, Google Cloud and Neo4j are platinum sponsors.](https://assets.ai.engineer/sites/aiecode2025/sha256/0fb22a09e9f20fd3ec8d6559e31e6a8481a7f7015b31652ad3245e98a2e6a284.webp)

INNOVATION SPONSOR
aws

PLATINUM SPONSORS
MongoDB®
Google Cloud
neo4j

### Cohere at a glance

Cohere’s overview lists a 2019 founding, a team of more than 300 and four offices. The presenter describes partnerships across clouds and flexibility in where Cohere can deploy.

[0:31](https://www.youtube.com/watch?v=u3NofYYstaY&t=31s) [Cohere overview showing its founding year, team size, offices, investors and strategic partners.](https://assets.ai.engineer/sites/aiecode2025/sha256/0ddee049a85a3dd5edbe282ea708d15f47fd6de800ab77db9bd5a60b93ee57b9.webp)

Cohere at a glance

cohere
Founded: 2019
Team: +300
Offices: 4
Toronto, London, San Francisco, New York

SELECT INVESTORS
NVIDIA
ORACLE
salesforce
RADICAL VENTURES
Schroders capital
Index Ventures
MIRAE ASSET
DTCP
THOMVEST
SAP
inovia
SentinelOne
TIGER GLOBAL
S32

STRATEGIC PARTNERS
ORACLE
salesforce
Google Cloud
McKinsey & Company
Microsoft Azure
aws
accenture

2

Interface text:
AIE
Microsoft
smol ai

[0:49](https://www.youtube.com/watch?v=u3NofYYstaY&t=49s) [Cohere overview showing founding year, team size, offices, investors and strategic partners.](https://assets.ai.engineer/sites/aiecode2025/sha256/8281e60dd5e2c69afcfc83bbe4041dde64b01f90aeb60300d8234df9dac98c71.webp)

Cohere at a glance

cohere
Founded: 2019
Team: +300
Offices: 4
Toronto, London, San Francisco, New York

SELECT INVESTORS
NVIDIA
ORACLE
salesforce
RADICAL VENTURES
Schroders capital
Index Ventures
MIRAE ASSET
DTCP
THOMVEST
SAP
inovia
SentinelOne
TIGER GLOBAL
S32

STRATEGIC PARTNERS
ORACLE
salesforce
Google Cloud
McKinsey & Company
Microsoft Azure
aws
accenture

2

Interface text:
AIE

### Cohere ML team

Cohere’s team includes Transformer co-author Aidan Gomez, sBERT builder Nils Reimers, and RAG paper co-author Patrick Lewis.

[1:04](https://www.youtube.com/watch?v=u3NofYYstaY&t=64s) [Cohere founders and ML team members with portraits, roles and prior experience.](https://assets.ai.engineer/sites/aiecode2025/sha256/453c881662e2139d36c39e6f23070670c352572620d337fc9b273df9074eb75a.webp)

Unique combination of ML innovators and seasoned enterprise operators

WORLD-CLASS ML TEAM

Nick Frosst
Co-founder
Google Brain, Geoff Hinton’s Lab

Aidan Gomez
Co-founder & CEO
Google Brain, Co-authored Transformer

Ivan Zhang
Co-founder
ML Ops, Co-founder For.ai

Sara Hooker
VP Cohere4AI
Google Brain, Founder Delta Analytics

Phil Blunsom
SVP Science
Led NLP at Deepmind, Prof. at Oxford

Nils Reimers
Dir. ML
HuggingFace, built sBERT

Patrick Lewis
RAG Lead
Meta, Co-authored RAG paper

Interface text:
AIE
aws

### Cohere offers a comprehensive suite of proprietary GenAI models

Command generates text, Embed turns text into semantic vectors, and Rerank improves search precision. Embed and Rerank support over 100 languages.

[1:35](https://www.youtube.com/watch?v=u3NofYYstaY&t=95s) [Comparison of Command R/R+, Embed 3 and Rerank 3, including use cases and language support.](https://assets.ai.engineer/sites/aiecode2025/sha256/2b7aa9fa05a140e18711e62069bf2364eb83bdece3f281b407e3388bd414f50a.webp)

Cohere offers a comprehensive suite of proprietary GenAI models

Generative Models
Advanced Retrieval Models

MODELS

Command R/R+
Command is a generative language model optimized for long-context tasks, such as retrieval-augmented generation.

Embed 3
Embed is a representation model that translates text into numerical vectors that capture the semantic meaning of text.

Rerank 3
Rerank is a search precision model that optimizes existing search and retrieval applications without having to replace any infrastructure.

USE CASES

Command R/R+
• Text generation
• Text summarization
• Knowledge assistants, Chat, Q&A
• Tool Use
• Advanced Retrieval (RAG)

Embed 3
• Semantic search
• Advanced Retrieval (RAG)
• Classification
• Clustering

Rerank 3
• Optimal search performance accuracy
• Prioritize highest quality responses
• Existing application integration
• Easy Integration w/ SDK/APIs

MULTILINGUAL

Ten Primary Languages
Languages: English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic, and Chinese

Global Languages Support
Embed and Rerank support and are trained on over 100+ languages

4

Interface text:
AIE

[2:22](https://www.youtube.com/watch?v=u3NofYYstaY&t=142s) [Comparison of Command R/R+, Embed 3 and Rerank 3, including use cases and language support.](https://assets.ai.engineer/sites/aiecode2025/sha256/382b64612376019f181dd05df827bb74ed59b012442ee5be629719757b0c6353.webp)

Cohere offers a comprehensive suite of proprietary GenAI models

Generative Models
Advanced Retrieval Models

MODELS

Command R/R+
Command is a generative language model optimized for long-context tasks, such as retrieval-augmented generation.

Embed 3
Embed is a representation model that translates text into numerical vectors that capture the semantic meaning of text.

Rerank 3
Rerank is a search precision model that optimizes existing search and retrieval applications without having to replace any infrastructure.

USE CASES

Command R/R+
• Text generation
• Text summarization
• Knowledge assistants, Chat, Q&A
• Tool Use
• Advanced Retrieval (RAG)

Embed 3
• Semantic search
• Advanced Retrieval (RAG)
• Classification
• Clustering

Rerank 3
• Optimal search performance accuracy
• Prioritize highest quality responses
• Existing application integration
• Easy Integration w/ SDK/APIs

MULTILINGUAL

Ten Primary Languages
Languages: English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic, and Chinese

Global Languages Support
Embed and Rerank support and are trained on over 100+ languages

4

Interface text:
AIE
Microsoft
smol ai

### Cohere's approach to enterprise needs

Cohere pairs enterprise-specific model evaluation and efficient inference with fine-tuning, data controls and deployment options ranging from managed SaaS to private containers.

[2:29](https://www.youtube.com/watch?v=u3NofYYstaY&t=149s) [Five enterprise needs—performance, efficiency, customization, data privacy and deployment flexibility—paired with Cohere's stated approaches.](https://assets.ai.engineer/sites/aiecode2025/sha256/b4eb09e7d38c904efcdee8a4269d747c1c6b681dbcd753274722ac262473cecd.webp)

Cohere addresses the unique and strict needs of the enterprise

Enterprise need

1. Model performance on enterprise use cases
2. Efficiency / scalability
3. Customization (fine-tuning)
4. Data provenance and privacy
5. Deployment flexibility

Our differentiated approach

1. Full-stack LLM technology, with SOTA models to drive accurate retrieval-augmented generation (RAG) for true enterprise search
2. Complete model inference framework that optimizes models post-training, producing performant models at lower compute & cost
3. Sophisticated, enterprise-grade fine-tuning tools, with superior accuracy at industry-leading inference costs
4. Full control on IP & data used for training, with data never used for base model training without authorization, and indemnification for IP claims
5. Most comprehensive set of deployment options, from Cohere-managed SaaS, to 3rd-party Managed Cloud, and containerized private deployments

5

Interface text:
AIE
Microsoft
smol ai

### Command R+ enterprise performance comparisons

Command R+ falls between GPT-4 Turbo and Mistral Large on multilingual and RAG, and leads both on tool use. The slide claims lower cost but shows no price comparison.

[4:59](https://www.youtube.com/watch?v=u3NofYYstaY&t=299s) [Bar charts compare Command R+, Mistral Large and GPT-4 Turbo on multilingual, RAG and tool-use benchmarks.](https://assets.ai.engineer/sites/aiecode2025/sha256/ae969a10e22291c4bb06d7ff8fceb281f22b3f990283c330cb316289315495e9.webp)

Generative Models: Command R+ provides competitive performance across enterprise use cases at significantly lower cost

Strong Generation performance vs. competitors across Enterprise Use Cases.. ..at significantly lower cost

Multilingual
Y-axis: Translation Quality (BLEU)
Y-axis ticks: 20, 25, 30, 35, 40
Command R+: 35.9
Mistral Large: 31.4
GPT-4 Turbo: 36.6

RAG
Y-axis: Average Accuracy
Y-axis ticks: 0, 20, 40, 60, 80
Command R+: 71.5
Mistral Large: 60.7
GPT-4 Turbo: 76.7

Tool Use
Y-axis: Average Success Rate
Y-axis ticks: 0, 20, 40, 60, 80
Command R+: 74.5
Mistral Large: 63.1
GPT-4 Turbo: 73.7

About LLM Performance: Performance comparison of models available across three key capabilities: Multilingual, RAG, and Tool Use; [illegible] of per-token price for model. Source: Artificial Analysis, medium prompts

6

Interface text:
AIE
Microsoft
smol ai

### Document summary with citations

A document summary pairs underlined claims with numbered source passages. The speaker says RAG citations come out of the box with Cohere’s models and APIs.

[5:32](https://www.youtube.com/watch?v=u3NofYYstaY&t=332s) [A chat interface summarizes an attached presentation and displays cited source passages alongside the response.](https://assets.ai.engineer/sites/aiecode2025/sha256/29c114f2cdf49aa5f84ab86df5c9618a9939ff09ace0715e11cd292375f28460.webp)

Attached Files
RAG External Keynote Q1 2024 - Building Connected Language Models.pdf
Building Connected Language Models Tales from the Retrieval and Tool Augmented Generation Team Presented by Sebastian Hofstätter About Cohere Cohere ... (743 Words)

summarize this

Reading the file: RAG External Keynote Q1 2024 - Building Connected Language Models.pdf

Building Connected Language Models ⁰
Presented by Sebastian Hofstätter ⁰, this is an overview of Cohere’s work on connected language models and their applications. ⁰

Cohere Introduction
• Cohere is a generative and search AI startup ⁰, founded in 2019. ⁰
• The team has a global presence ⁰, with offices in multiple locations (Toronto, SF, NYC, London) and remote workers. ⁰
• They train their own LLMs from scratch ⁰, with a focus on connected language models and search. ⁰
• The models are available across all major cloud providers and private deployments. ⁰

Connected Language Models
• The aim is to build a single LM ¹ that supports highly modular building blocks. ¹
• The model should be capable of composing multiple tools together ¹ and be able to ingest tool results to provide natural language answers. ¹
• An example of a use case is an improved private data search. ³,⁶

Verifiable Output
• The issue of language models being untrustworthy ² is addressed by only generating claims that are verifiable with fine-grained citations. ²

Source passages:
0 RAG External Keynote Q1 2024 - Building Connected Languag...
Globally distributed team with offices in Toronto, SF, NYC, London, +Remote ... Strong focus on connected language models & search

1 RAG External Keynote Q1 2024 - Building Connected Languag...
We are building a single LM which supports highly modular building blocks ... Capable of composing multiple tools together in a chain Strong focus on ingesting tool results back into natural language answers

2 RAG External Keynote Q1 2024 - Building Connected Languag...
Ultimately, it is the user’s responsibility to check, but we should make LLMs that make it easy to check. ... By focusing on verifiability, we introduce a forcing function towards more responsible, useful and factually-accurate models

3 RAG External Keynote Q1 2024 - Building Connected Languag...
General purpose chat connected to web search & APIs ... Improved private data search

6 RAG External Keynote Q1 2024 - Building Connected Languag...

Interface text:
Let’s go! Ask me anything, or give me a task to do.
New Chat
Screenshot
Flag Response
File Explorer · File Read · File Search · Directly Answer
AIE

### Multilingual model evaluation

Command R+ leads L2-to-English at 39.6, while GPT-4 Turbo leads English-to-L2 at 38.2. Cohere also claims substantially cheaper, more efficient tokenization, but shows no cost figures.

[6:08](https://www.youtube.com/watch?v=u3NofYYstaY&t=368s) [A FLORES chart compares translation evaluation scores for Command models and several competing models.](https://assets.ai.engineer/sites/aiecode2025/sha256/6bc03d7a21037134d39d051b3c6ef4034f1a03e7d8841266cab3c802efcfc6ad.webp)

Cohere’s multilingual models have both industry-leading performance and tokenization costs

Best OOTB multilingual performance vs peers on key set of business languages

...with substantially cheaper and more efficient tokenization

FLoRES Multilingual Evaluation
Legend: FLoRES (en -> L2); FLoRES (L2->en)
Y-axis ticks: 20, 25, 30, 35, 40
Mistral Large: 32.3, 34.2
Command R: 33.1, 35.7
Claude 3 Haiku: 34.4, 34.9
GPT-3.5 Turbo: 37.2, 34.5
Claude 3 Sonnet: 35.4, 37.1
GPT-4 Turbo: 38.2, 37.4
Command R+: 37.7, 39.6

8

Interface text:
AIE
Microsoft
smol ai

### Embedding performance and storage costs

Cohere’s charts show Embed v3 near OpenAI on English retrieval and ahead on noisy retrieval; its int8 and binary variants cluster at lower annual vector DB costs in the separate comparison.

[6:40](https://www.youtube.com/watch?v=u3NofYYstaY&t=400s) [A bar chart compares retrieval performance, and a scatterplot compares embedding performance with annual vector database costs.](https://assets.ai.engineer/sites/aiecode2025/sha256/ac75d97441fa5d99dd41251a988a702b248dbad239d727d18d4453479c906bc8.webp)

Embedding Models: Embed 3 offers comparable search performance at significantly lower vector DB storage costs

Class-leading performance across standardized benchmarks and real world noisy datasets...

..at significantly cheaper storage costs vs. competition

Embeddings Performance
Legend: English Retrieval (Open BEIR); Noisy Retrieval (trec-covid)
Cohere Embed v3: 55.86; 81.90
OpenAI text-embedding-3: 56.52; 79.56
Google PALM gecko-001: 44.31; 61.40
AWS Titan Text Embeddings: 47.52; 54.70
X-axis: Accuracy (Higher is better)
X-axis ticks: 30, 40, 50, 60, 70, 80

Embeddings Performance vs Associated Storage Costs
Cohere v3 - int8
Cohere v3 - binary
Cohere v3 - light - int8
OpenAI 3-large - float32
OpenAI 3-small - float32
OpenAI ada v2 - float32
Y-axis: Miracl (Higher is better)
Y-axis ticks: 30, 40, 50, 60, 70
X-axis: Annual Vector DB costs ($)
X-axis ticks: $0, $100,000, $200,000

10

Interface text:
AIE

[6:50](https://www.youtube.com/watch?v=u3NofYYstaY&t=410s) [Charts compare embedding retrieval performance and annual vector database storage costs.](https://assets.ai.engineer/sites/aiecode2025/sha256/6da6548ab9355c1419e4f72b7bd0c5a18d0fbbca82e9178347dd13346db00b59.webp)

Embedding Models: Embed 3 offers comparable search performance at significantly lower vector DB storage costs

Class-leading performance across standardized benchmarks and real world noisy datasets...

..at significantly cheaper storage costs vs. competition

Embeddings Performance
Legend: English Retrieval (Open BEIR); Noisy Retrieval (trec-covid)
Cohere Embed v3: 55.86; 81.90
OpenAI text-embedding-3: 56.52; 79.56
Google PALM gecko-001: 44.31; 61.40
AWS Titan Text Embeddings: 47.52; 54.70
X-axis: Accuracy (Higher is better)
X-axis ticks: 30, 40, 50, 60, 70, 80

Embeddings Performance vs Associated Storage Costs
Cohere v3 - int8
Cohere v3 - binary
Cohere v3 - light - int8
OpenAI 3-large - float32
OpenAI 3-small - float32
OpenAI ada v2 - float32
Y-axis: Miracl (Higher is better)
Y-axis ticks: 30, 40, 50, 60, 70
X-axis: Annual Vector DB costs ($)
X-axis ticks: $0, $100,000, $200,000

10

Interface text:
AIE
Microsoft
smol ai

### Semantic search results

A demo asks when Aidan Gomez’s attention paper was published, but the displayed search results include a withdrawn paper and two unrelated discussion papers.

[7:13](https://www.youtube.com/watch?v=u3NofYYstaY&t=433s) [A query about an attention paper returns three unrelated arXiv results.](https://assets.ai.engineer/sites/aiecode2025/sha256/8c91dc643261098c1991b01540be3cb9a914b1abeb2aa203ad26a222d5ae5a28.webp)

When was the attention paper by Aidan Gomez published?

SEARCH RESULTS
Semantic Search

1. WITHDRAWN
arXiv
Authors: Nil Date: Mon, 21 Mar 2005 20:12:45 GMT
This paper has been withdrawn.

2. COMMENT ON ARTICLE BY FERREIRA AND GAMERMAN
arXiv
Authors: Michael Chipeta, Peter J. Diggle Date: Wed, 16 Sep 2015 06:14:38 GMT
Comment on Article by Ferreira and Gamerman [arXiv:1509.03410].

3. INTRODUCING THE DISCUSSION PAPER BY SZ\'{E}KELY AND RIZZO
arXiv
Authors: Michael A. Newton Date: Mon, 18 Oct 2010 12:17:11 GMT
Introducing the discussion paper by Sz\'{e}kely and Rizzo

Interface text:
AIE
Microsoft
smol ai

### Why doesn’t Embeddings Work?

An arXiv search for the attention paper returns unrelated papers. The speaker says dates, authors and titles create a complex vector space where queries may not map neatly.

[7:38](https://www.youtube.com/watch?v=u3NofYYstaY&t=458s) [A semantic search example sits beside labels for years, times, authors and paper titles.](https://assets.ai.engineer/sites/aiecode2025/sha256/16383460738db6f61ce4a939df5488e99047d598b09d3c7630fe4de3c04e7ee1.webp)

Why doesn’t Embeddings Work?

When was the attention paper by Aidan Gomez published?
SEARCH RESULTS
Semantic Search

1. WITHDRAWN
Authors: Nil Date: Mon, 21 Mar 2005 20:12:45 GMT
This paper has been withdrawn.

2. COMMENT ON ARTICLE BY FERREIRA AND GAMERMAN
Authors: Michael Chipeta, Peter J. Diggle Date: Wed, 16 Sep 2015 06:14:38 GMT
Comment on Article by Ferreira and Gamerman [arXiv:1509.03410].

3. INTRODUCING THE DISCUSSION PAPER BY SZ\'{E}KELY AND RIZZO
Authors: Michael A. Newton Date: Mon, 18 Oct 2010 12:17:11 GMT
Introducing the discussion paper by Sz\'{e}kely and Rizzo

2017
2016
2015
Withdrawn
Tue
Mon
20:12:45
06:14:38
12:17:11
Jacob Devlin
Michael Chipeta
Aidan Gomez
Attention is all You Need
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Welcome to NLP, now.
cohere.com

Interface text:
AIE
arXiv
Microsoft
smol ai

### Rerank 3 performance and RAG costs

Rerank 3 improves scores over BM25 across five search tasks. The speaker says narrowing the context sent to the language model can cut input-token costs; the slide shows cost reductions of 77% and 55% for two Command models.

[9:11](https://www.youtube.com/watch?v=u3NofYYstaY&t=551s) [Two bar charts compare BM25 with Rerank 3 across search tasks and show illustrative RAG costs with and without reranking.](https://assets.ai.engineer/sites/aiecode2025/sha256/3cfac7f7769d050dbce8c70b49f9c3532380fb92c698c1c8101bdf6fa82068e8.webp)

Rerank Models: Rerank 3 offers best-in-class uplift on search performance for RAG at leading efficiency

Rerank 3 improves search performance across tasks/modalities... ...at significantly more scalable efficiency versus competition

Rerank 3 performance uplift vs BM25 across tasks
Y-axis: Accuracy (nDCG@10)
Y-axis ticks: 0, 20, 40, 60, 80
Right Y-axis: Accuracy (Recall@5)
Right Y-axis ticks: 0, 20, 40, 60, 80
BEIR Performance (15 datasets): BM25 43.7; BM25 + Rerank 3 52.2
Multilingual evaluation (MIRACL): BM25 36.5; BM25 + Rerank 3 62.8
Long context (7 datasets): BM25 54.6; BM25 + Rerank 3 69
Code evaluations (6 datasets): BM25 34; BM25 + Rerank 3 51.7
Semi-structured data (JSON): BM25 47.5; BM25 + Rerank 3 60.3
Legend: BM25; BM25 + Rerank 3

RAG TCO With and Without Rerank
Y-axis ticks: $0, $50,000, $100,000, $150,000
GPT-4 Turbo: $132,500
Mistral Large: $106,000
Claude Sonnet: $41,250
Command R+: $41,250
Command R+ w/ Rerank: $9,500
Command R: $6,625
Command R w/ Rerank: $3,000
Command R+ to Command R+ w/ Rerank: -77%
Command R to Command R w/ Rerank: -55%
Legend: Total Standalone Cost; Differential

14

Interface text:
AIE
Microsoft
smol ai

### Enterprise deployment options

Cohere offers a managed SaaS API and access through cloud AI services, alongside private deployments in a customer’s cloud or on-premises environment using existing compute.

[9:41](https://www.youtube.com/watch?v=u3NofYYstaY&t=581s) [Four Cohere deployment options: SaaS API, cloud AI services, private cloud, and on-premises deployment.](https://assets.ai.engineer/sites/aiecode2025/sha256/1e7207720771441147c98f0b64437ff276f76ee185c7aca2d6dc6cbcb522ab92.webp)

Deployment options focusing on enterprise clients’ requirements

1. SaaS API
Add Cohere’s LLM capabilities to any system via APIs offered by Cohere’s managed platform.
cohere

2. Cloud AI Services
Procure Cohere models directly through several of the leading managed cloud AI offerings (Sagemaker, OCI, etc).
Sagemaker
Azure AI
Bedrock
OCI Generative AI
NVIDIA*
Vertex AI*
*coming soon

3. Private Deployment-Cloud
Cohere’s models are deployed in the customer’s cloud environment (AWS, Azure, GCP, and OCI).
Azure
NVIDIA
aws
ORACLE

4. Private Deployment-On Premise
Cohere’s models are deployed in the customer’s environment and private network, using their existing compute.

15

Interface text:
AIE

### Security and privacy compliance

Cohere cites GDPR, CCPA and SOC 2 Type II alongside Bugcrowd penetration testing and its governing role in the OWASP LLM application security project.

[10:11](https://www.youtube.com/watch?v=u3NofYYstaY&t=611s) [Cohere describes its security practices and displays GDPR, CCPA, SOC 2, Bugcrowd, and OWASP logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/7afc97f4449fecd94f95f62e330e769c3451f5f85da5cfbbab1dd5c0a1b5d48f.webp)

Leading the way on security and privacy compliance standards

Cohere employs security best practices to meet prevailing industry standards and is a leading as well as governing member of the novel OWASP LLM application security project.

Visit the Cohere Trust Center to learn more about our security and privacy compliance best practices.

GDPR
CCPA
AICPA SOC
SOC 2 TYPE II CERTIFIED
bugcrowd
Penetration Testing
OWASP
Top 10 for LLM applications

16

Interface text:
AIE
Microsoft
smol ai

### Enterprise application connectors

Cohere lists connectors for services including Box, Jira, MongoDB and Airtable, plus 50+ more. The speaker says its open-source toolkit helps developers ingest data; the slide notes limits tied to public APIs and self-serve applications.

[10:22](https://www.youtube.com/watch?v=u3NofYYstaY&t=622s) [A grid of enterprise application and datastore connectors, including Dropbox, SharePoint, Gmail, Slack, and GitHub, with a claim of over 50 more.](https://assets.ai.engineer/sites/aiecode2025/sha256/13b9efd9bed66f90ad628624174eaa7fcd0037945faf473e085f157df42a9fdd.webp)

Cohere quickstart connectors support popular enterprise applications

Connect to datastores such as:

box
elastic
redis
Google Workspace
Jira Software
freshworks
mongoDB
Pinecone
smartsheet
freshdesk
miro
zendesk
Trello
MySQL
HubSpot
Confluence
Office 365
Figma
servicenow
Qdrant
ORACLE
asana
EGNYTE
Airtable
coda

& 50+ more...

Note: Limited by public APIs and self-serve applications

17

Interface text:
AIE

## Full transcript

[0:00](https://www.youtube.com/watch?v=u3NofYYstaY&t=0s) [on-hold music] Um, hey, folks. Uh, this is Vivek. Uh, super excited to chat with all of y'all.

[0:16](https://www.youtube.com/watch?v=u3NofYYstaY&t=16s) Uh, we'll give a, a quick talk about, uh, what Cohere is all about, and, uh, we'll make sure we have enough time to chat about, uh, your production challenges with these models or anything else you wanna chat about.

[0:29](https://www.youtube.com/watch?v=u3NofYYstaY&t=29s) Um, cool. Uh, quick intro. Um, so we are a leading data security-focused, uh, enterprise AI company. Uh, our focus is building trustworthy, uh, enterprise AI models, uh, with our partners for real-world business, uh, use cases.

[0:45](https://www.youtube.com/watch?v=u3NofYYstaY&t=45s) Um, and, uh, we work with a lot of, like, strategics a-across, uh, various clouds, uh, and are, uh, have a bit of a Switzerland play when it comes to where we can, uh, ship and deploy, and I'll talk to that, uh, a little later.

[0:59](https://www.youtube.com/watch?v=u3NofYYstaY&t=59s) Um, so we have a crack team of, uh, ML, uh, researchers and seasoned enterprise, uh, operators. Uh, Aidan, uh, who's our co-founder and CEO, was one of the, uh, authors on the, uh, seminal transformer paper.

[1:12](https://www.youtube.com/watch?v=u3NofYYstaY&t=72s) Uh, we have, uh, Phil, who's our chief scientist. Uh, he was an NLP lead at DeepMind and a professor at Oxford. Uh, Nils, uh, who leads a lot of our retrieval efforts, was a BERT expert.

[1:25](https://www.youtube.com/watch?v=u3NofYYstaY&t=85s) Uh, and then Patrick, uh, was, uh, the co-author on the RAG paper, which is a big, uh, focus for us at, uh, Cohere. So, um, fantastic team, uh, that's, uh, helping us build lots of amazing things.

[1:36](https://www.youtube.com/watch?v=u3NofYYstaY&t=96s) Um, so here's a quick overview of our product line. Um, so we have, uh, two main arcs, so the generative side and then the advanced retrieval models. Um, on the generative side, uh, we have, uh, two flagship models, which is Command R and R+.

[1:52](https://www.youtube.com/watch?v=u3NofYYstaY&t=112s) R is our, uh, workhorse, uh, super low-cost model, uh, that's great, great for most enterprise use cases. Uh, and then we have R+, which is, uh, your more powerful, larger model, uh, for more complex, like reasoning, tool use, RAG, uh, use cases.

[2:09](https://www.youtube.com/watch?v=u3NofYYstaY&t=129s) Um, and on the retrieval side, uh, uh, most people have used some form of an embedding model, um, and I'll get into that a little later in the talk.

[2:18](https://www.youtube.com/watch?v=u3NofYYstaY&t=138s) Uh, but, uh, the Reranker is something special, uh, that I haven't seen quite often in the market, uh, but we think, uh, it adds a lot of value, especially to your, uh, RAG pipelines.

[2:28](https://www.youtube.com/watch?v=u3NofYYstaY&t=148s) Um, so we'll get into that too. Um, so when we build at Cohere, uh, so we have, uh, five guiding, uh, ethos as to how, uh, we want to go about things.

[2:37](https://www.youtube.com/watch?v=u3NofYYstaY&t=157s) Um, the first is, uh, obviously everybody's testing their models in all sorts of academic benchmarks. Uh, but for us, what is really important is the performance on, uh, enterprise use cases.

[2:48](https://www.youtube.com/watch?v=u3NofYYstaY&t=168s) Um, so we've worked, uh, with a lot of our partners to ensure that we have an eval suite, uh, that is highly customized to enterprise, uh, use cases across, like let's say health, HR, finance, uh, and that's, uh, a bit of our goalpost as we ship, uh, e-each of these like model versions and, uh, we constantly benchmark,

[3:08](https://www.youtube.com/watch?v=u3NofYYstaY&t=188s) uh, on how we are performing at each of these industries, uh, and use cases that our customers care about. Um, and then when-- the next thing is ef-- all about efficiency and scalability, right?

[3:18](https://www.youtube.com/watch?v=u3NofYYstaY&t=198s) We're, uh, not particularly chasing the race for having the largest model out there, but what we really care about is the practical use of these models, right? How, how do these models get used, and how cheap is it, uh, uh, and how easy is it for you to run it as a customer?

[3:35](https://www.youtube.com/watch?v=u3NofYYstaY&t=215s) Um, the next big thing is obviously customization. Um, you know, as, as much as we'd like for all of these models to work out of the box, there's always a, a certain niche that, uh, customers want to customize this for.

[3:49](https://www.youtube.com/watch?v=u3NofYYstaY&t=229s) Uh, and we offer a variety of things, uh, some of which are pretty intrusive. Uh, we've helped our customers, uh, with taking our base model, uh, and retraining that with their enterprise-specific data, uh, for domain adaptation.

[4:03](https://www.youtube.com/watch?v=u3NofYYstaY&t=243s) We can do a full retraining of the model for you with your data. Uh, and then obviously the, uh, pretty typical last few layers, uh, retraining, which is self-serve on our platform.

[4:14](https://www.youtube.com/watch?v=u3NofYYstaY&t=254s) Um, data provenance and privacy is another, uh, big focus, uh, for us. Uh, we've, uh, worked quite a bit to ensure that all of the data that we've, uh, collected for building our models, uh, is, uh, meets up to the enterprise, uh, standards, uh, and we offer indemnification for any IP claims, uh, that you might run into

[4:33](https://www.youtube.com/watch?v=u3NofYYstaY&t=273s) as a customer. Uh, and then obviously we don't ever use any of your data to train our models. So, um, so that, that's a, uh, guarantee from us. Uh, and, uh, deployment flexibility.

[4:44](https://www.youtube.com/watch?v=u3NofYYstaY&t=284s) Uh, as I mentioned, we're available on pretty much every major cloud provider. Uh, and then we also allow you to deploy, uh, on-prem or in your own VPC, um, uh, wherever, uh, your compute and your data is, that's where we'll meet you at.

[4:58](https://www.youtube.com/watch?v=u3NofYYstaY&t=298s) Um, cool. Um, so just a quick look at, uh, uh, you know, your typical performance metrics. Uh, as I mentioned, uh, something that enterprises like repeatedly tell us is, uh, they care about like multilingual, they care about like RAG, um, and tool use for upcoming like agentic use cases.

[5:16](https://www.youtube.com/watch?v=u3NofYYstaY&t=316s) Uh, so a, a lot of our focus has been, uh, in these areas. Uh, these are some benchmarks from HotpotQA, Bamboogle, uh, Berkeley Function Calling, uh, that our models, uh, are quite, uh, good at.

[5:30](https://www.youtube.com/watch?v=u3NofYYstaY&t=330s) Uh, and, um, and another example of like how we try to innovate is, uh, we try to make sure that as we are building, uh, w- these stacks, we're incorporating all of the features that people care about out of the box, and they don't have to do extra work.

[5:46](https://www.youtube.com/watch?v=u3NofYYstaY&t=346s) Citations on RAG is a great example of this. Uh, for most people, you have to do a lot of work to actually build this functionality with like other APIs, uh, but this really comes out of the box with like Cohere's, uh, models and APIs.

[5:58](https://www.youtube.com/watch?v=u3NofYYstaY&t=358s) You don't have to do anything additional as a developer, uh, to build, uh, get citations and which is, uh, very important for any RAG-based like application. Um, uh, on the multilingual front, uh, we have one of the best performance when it comes to, uh, the FLORES multilingual, uh, evaluation.

[6:19](https://www.youtube.com/watch?v=u3NofYYstaY&t=379s) Uh, and we also have a bit of a secret sauce with our, uh, tokenizer, uh, which, uh, helps keep costs really low, right? Uh, and, uh, that's again, uh, TCO is again a very big thing for enterprises.

[6:30](https://www.youtube.com/watch?v=u3NofYYstaY&t=390s) Uh, and, uh, it allows our customers to take that same model and de- deploy across the globe on, uh, with their customers, uh, which is very important. Um, switching gears towards the embeddings models, uh, again, uh, given, uh, Nils and his team, uh, have been innovators in this space for a while now, uh, and, uh, we've, uh,

[6:53](https://www.youtube.com/watch?v=u3NofYYstaY&t=413s) done quite a bit of work to make our-- make sure our performance is great on, like, noisy data and at a super low cost, uh, uh, in this particular space.

[7:02](https://www.youtube.com/watch?v=u3NofYYstaY&t=422s) Um, so we're, we're actually pretty excited about what our embeddings models, uh, can do, and this is al- almost always, like, one of the top things that, uh, our customers are, uh, excited about.

[7:14](https://www.youtube.com/watch?v=u3NofYYstaY&t=434s) Um, but embeddings, uh, is a pretty complex space and not without its, uh, challenge. Uh, so we, we try to build this, like, fun demo where we took all of the arXiv papers, uh, and we asked it a question, uh, "When was the attention paper, um, built by, uh-- paper published by Aidan, uh, Gomez," who's our founder?

[7:33](https://www.youtube.com/watch?v=u3NofYYstaY&t=453s) And we tried this across, like, a bunch of, like, embeddings model. So some common patterns that we see is, um, arXiv is a great example of, like, where you have different kinds of, like, data, right?

[7:44](https://www.youtube.com/watch?v=u3NofYYstaY&t=464s) You have, like, the title, when was the paper published, the dates. Uh, you have the various authors. You have, like, the actual paper itself. And in many ways, this represents the kind of data you might see in enterprises, right?

[7:55](https://www.youtube.com/watch?v=u3NofYYstaY&t=475s) Um, so when you, uh, actually build the embeddings for this, um, you get a fairly complex, like, vector space, uh, and your search queries might not actually, like, map neatly to this, right?

[8:07](https://www.youtube.com/watch?v=u3NofYYstaY&t=487s) Uh, and this is sort of like where our reranker comes in. Uh, so what our reranker does is once you have, uh, all of these, uh, retrieved documents, uh, it hel- it's a cross encoder that helps you, uh, rerank the, uh, retrieved set and make sure that that's the one that you send into your context with the

[8:25](https://www.youtube.com/watch?v=u3NofYYstaY&t=505s) generative model. Um, and, uh, here's the reranker in, uh, action. Um, so what this demo is showing you is, uh, we search, uh, for the transformer paper by Aidan.

[8:38](https://www.youtube.com/watch?v=u3NofYYstaY&t=518s) So you have three different types of, like, search patterns over here: a lexical search, then an embeddings-based search, uh, and, uh, the Cohere Rerank-based search, right? Uh, and the, uh, various forms of these, like, retrievals obviously give you the responses, but they are stack ranked at different places in the retrieval set, which means the overall accuracy of

[8:58](https://www.youtube.com/watch?v=u3NofYYstaY&t=538s) your RAG system might be low. Uh, and rerank is what's helping you to make sure, uh, that isn't the case. Uh, this builds on top of, like, other things that people care about, like chunking strategies, uh, but making those more optimal.

[9:12](https://www.youtube.com/watch?v=u3NofYYstaY&t=552s) Uh, another impact of this reranker is, again, total cost of operation, because most of the expense for your models is coming in from the input tokens, right? Uh, and if you were able to, like, narrow in, uh, to the right, uh, uh, context and do that quickly, uh, you could pass in very minimal amount of context to

[9:32](https://www.youtube.com/watch?v=u3NofYYstaY&t=572s) your large language model, which drives on-- drives down your overall cost of, like, operation. Uh, and that, uh, is again, very important in the enterprise, uh, setting. Um, yeah, and when it comes to deployment options, like I said, we have our SaaS API, uh, that we can help, uh, you manage run your wo- uh, workloads.

[9:52](https://www.youtube.com/watch?v=u3NofYYstaY&t=592s) Uh, but then we're also on all of the major cloud AI services, Sa-Sage, uh, SageMaker, Bedrock, um, OCI, uh, and private deployment across all of these cloud providers and pri-- um, also on-premise deployments if, if that's, uh, something you care about.

[10:08](https://www.youtube.com/watch?v=u3NofYYstaY&t=608s) Um, security and privacy obviously is a pretty, uh, top of mind for us, so we make sure we're compliant with, uh, uh, the standards that our customers are often asking us, uh,

[10:20](https://www.youtube.com/watch?v=u3NofYYstaY&t=620s) for. Um, and then the last bit is just enabling, like, developers. Um, so we, uh, obviously have a, a pretty tight integration with things like La-LangChain, LlamaIndex. Uh, but we also have an open source, like, toolkit, uh, that comes out of the box, uh, with, uh, various forms of, like, connectors, um, and that lets you, uh, ingest,

[10:40](https://www.youtube.com/watch?v=u3NofYYstaY&t=640s) uh, data pretty easily into your systems and lets you have full control over the things you're, uh, building, um, and don't have to really, uh, look, uh, for a ton of different, like, options, uh, as you're developing your, uh, enterprise applications.

[10:55](https://www.youtube.com/watch?v=u3NofYYstaY&t=655s) So, um, that's it from me, and I'd love to take any questions or chat. And we also have Sandra here, uh, from Cohere, so she'll, she'll be happy to help.

[11:06](https://www.youtube.com/watch?v=u3NofYYstaY&t=666s) Thank, thank you very much.

[11:07](https://www.youtube.com/watch?v=u3NofYYstaY&t=667s) Yeah.

[11:08](https://www.youtube.com/watch?v=u3NofYYstaY&t=668s) May- maybe two questions. So on the first or second slide, you show your, uh, investors and the selected partners, right?

[11:13](https://www.youtube.com/watch?v=u3NofYYstaY&t=673s) Yeah.

[11:13](https://www.youtube.com/watch?v=u3NofYYstaY&t=673s) I saw Accenture, I saw McKinsey.

[11:15](https://www.youtube.com/watch?v=u3NofYYstaY&t=675s) Yeah.

[11:16](https://www.youtube.com/watch?v=u3NofYYstaY&t=676s) Can you explain a little bit, like, how that partnership, uh, work, right?

[11:20](https://www.youtube.com/watch?v=u3NofYYstaY&t=680s) Yeah.

[11:21](https://www.youtube.com/watch?v=u3NofYYstaY&t=681s) And then, and then the second question, like, you can skip if someone else, like-

[11:23](https://www.youtube.com/watch?v=u3NofYYstaY&t=683s) Yeah

[11:23](https://www.youtube.com/watch?v=u3NofYYstaY&t=683s) ... has another one, is can, can you, like, maybe without disclosing, uh, customers and all, give us a, like a few samples of where you- ... clients picked Cohere versus other solutions and, and kind of why?

[11:37](https://www.youtube.com/watch?v=u3NofYYstaY&t=697s) So explain where you win, right, on the, on the enterprise world.

[11:40](https://www.youtube.com/watch?v=u3NofYYstaY&t=700s) Yeah, absolutely. Uh, happy to chat about that. Um, so, uh, I think the typical challenge with all of these enter- um, enterprise, like, generative AI models is the last mile challenge, right?

[11:50](https://www.youtube.com/watch?v=u3NofYYstaY&t=710s) Like, uh, there's, uh, so many different, like, arcs of, like, customization that's needed, uh, with the enterprises. Uh, and, uh, there's a lot of, like, traditional, uh, like, players who've been around for a while and have great relationships with the enterprises, uh, have a deep understanding of, like, the various, like, uh, business domains.

[12:10](https://www.youtube.com/watch?v=u3NofYYstaY&t=730s) Uh, that's where, uh, the McKinsey and Accenture and all of these companies come in. Um, so they've been, uh, really, uh, helpful for us to co-develop, like, the product.

[12:20](https://www.youtube.com/watch?v=u3NofYYstaY&t=740s) Like, make sure that we're able to effectively, uh, bridge that, like, last mile gap with them. Um, and yeah, that... hopefully that, that helps. Uh, and then onto your second question, I would say, um, uh, in terms of, like, winnability, I think, like, the main, uh, aspects that has been, uh, resonating a lot with our customers is

[12:39](https://www.youtube.com/watch?v=u3NofYYstaY&t=759s) this control over the data and control over the compute, right? Uh, given we're available pretty much everywhere, uh, a lot of, like, the customers care about that private cloud deployment.

[12:49](https://www.youtube.com/watch?v=u3NofYYstaY&t=769s) The ability to fine-tune in that, uh, private cloud environment, uh, which is pretty big, uh, for a lot of people, and making sure that their enterprise data does not leave their own ecosystem.

[13:00](https://www.youtube.com/watch?v=u3NofYYstaY&t=780s) Um, and specific examples for that have been, uh, companies in, uh, let's say HR or healthcare or even, uh, folks who are trying to take their in-house, like, code and, uh, build a custom model that's, uh, working with, uh, their code base.

[13:16](https://www.youtube.com/watch?v=u3NofYYstaY&t=796s) Uh, those are the styles of applications, uh, that, uh, we've seen a lot of like, uh, impact and success with.

[13:21](https://www.youtube.com/watch?v=u3NofYYstaY&t=801s) Thank you.

[13:22](https://www.youtube.com/watch?v=u3NofYYstaY&t=802s) Yeah.

[13:27](https://www.youtube.com/watch?v=u3NofYYstaY&t=807s) So I'm curious, um, uh, for text classification, well, what is the kind latest best practice? I see on your website you have a classify endpoint, right, build a classifier.

[13:38](https://www.youtube.com/watch?v=u3NofYYstaY&t=818s) Uh, is that still the recommendation?

[13:40](https://www.youtube.com/watch?v=u3NofYYstaY&t=820s) Yeah. Uh, that's a, that's a great question. I, I... The, the way I like to think about, uh, these things is there's always the arc of like, uh, um...

[13:49](https://www.youtube.com/watch?v=u3NofYYstaY&t=829s) A- and I think, like, Jerry in his earlier talk did this, right? Which was what phase of, like, development you're in, uh, if you're trying to, like, prototype or you're trying to productionize, and what's your scale of, like, a production setting, uh, which, uh, is important to consider for these things.

[14:04](https://www.youtube.com/watch?v=u3NofYYstaY&t=844s) Uh, for, uh, if you're just trying to, like, get off the ground, like, quickly, I would say just using the generative model off the shelf is obviously always great.

[14:13](https://www.youtube.com/watch?v=u3NofYYstaY&t=853s) It, it gets you off the ground really quickly. Uh, when you're trying to, like, productionize something, that's when I would start thinking, "Hey, do I need a bespoke model?

[14:22](https://www.youtube.com/watch?v=u3NofYYstaY&t=862s) Uh, or, like, the, the general model is good enough," and what are sort of, like, the cost of operation, like, differences and, like, the cost of, like, maintenance dif- differences and also the scale, right?

[14:32](https://www.youtube.com/watch?v=u3NofYYstaY&t=872s) Like, I mean, if, if you're going to try to do something that's, uh, you know, tens of thousands, like TPS, uh, then having a, a purpose-built, like, model for that is the route I'd go, uh, versus, you know, uh, a more heavy general model, uh, which might serve other, other needs.

[14:48](https://www.youtube.com/watch?v=u3NofYYstaY&t=888s) Um, so yeah, both of those are good options depending on what you're trying to accomplish.

[14:53](https://www.youtube.com/watch?v=u3NofYYstaY&t=893s) Just a quick-

[14:53](https://www.youtube.com/watch?v=u3NofYYstaY&t=893s) Yeah. Yeah

[14:53](https://www.youtube.com/watch?v=u3NofYYstaY&t=893s) ... uh, quick follow-up. If we actually train a classifier with you, is that also a transformer model, just more specialized?

[15:00](https://www.youtube.com/watch?v=u3NofYYstaY&t=900s) Yeah, exactly.

[15:01](https://www.youtube.com/watch?v=u3NofYYstaY&t=901s) Okay.

[15:01](https://www.youtube.com/watch?v=u3NofYYstaY&t=901s) Yeah.

[15:02](https://www.youtube.com/watch?v=u3NofYYstaY&t=902s) Got it. Thank you.

[15:05](https://www.youtube.com/watch?v=u3NofYYstaY&t=905s) Yeah.

[15:06](https://www.youtube.com/watch?v=u3NofYYstaY&t=906s) No, I think we're done. Oh, no, we've got one more question. Here we go.

[15:10](https://www.youtube.com/watch?v=u3NofYYstaY&t=910s) Just a quick question. When we were looking at Cohere, you know, probably early last or middle of last year-

[15:18](https://www.youtube.com/watch?v=u3NofYYstaY&t=918s) Mm-hmm

[15:19](https://www.youtube.com/watch?v=u3NofYYstaY&t=919s) ... um, one of the challenges with the models that we found were the input context size limit-

[15:24](https://www.youtube.com/watch?v=u3NofYYstaY&t=924s) Yeah

[15:24](https://www.youtube.com/watch?v=u3NofYYstaY&t=924s) ... were quite small. H- how has that evolved a- as you guys have sort of created the next, you know, sets of models on your side?

[15:31](https://www.youtube.com/watch?v=u3NofYYstaY&t=931s) Yeah, that's a great question. So our latest generation models are, uh, fairly competitive, 128K, uh, context input, uh, windows. Uh, and we're constantly looking to, uh, figure out how to, like, up them.

[15:42](https://www.youtube.com/watch?v=u3NofYYstaY&t=942s) Um, so context window, I would say, should be not a problem for, like, most applications, uh, at the moment.

[15:50](https://www.youtube.com/watch?v=u3NofYYstaY&t=950s) Okay.

[15:50](https://www.youtube.com/watch?v=u3NofYYstaY&t=950s) Yeah.

[15:53](https://www.youtube.com/watch?v=u3NofYYstaY&t=953s) Cool. Well, we're actually slightly early, but yeah, I'd like to thank you for giving the talk.

[15:58](https://www.youtube.com/watch?v=u3NofYYstaY&t=958s) Yeah.

[15:58](https://www.youtube.com/watch?v=u3NofYYstaY&t=958s) Thank you, Vivek.

[15:58](https://www.youtube.com/watch?v=u3NofYYstaY&t=958s) Absolutely. Thank you so much. Yeah.

[16:00](https://www.youtube.com/watch?v=u3NofYYstaY&t=960s) Thank you.

[16:00](https://www.youtube.com/watch?v=u3NofYYstaY&t=960s) Yeah.

[16:01](https://www.youtube.com/watch?v=u3NofYYstaY&t=961s) Thank you. [audience cheering] [upbeat music]
