# Accelerate your AI journey with Azure AI model catalog

Sharmila Chokalingam, Microsoft · Shubhi

AI Engineer World's Fair 2024

[Talk page](https://ai.engineer/talks/gL9kfxt6uo0-accelerate-your-ai-journey-azure-ai-model) · [Video](https://www.youtube.com/watch?v=gL9kfxt6uo0)

## From model selection to production with Azure AI Model Catalog

Follow model discovery, deployment, grounded chat, function calling, and flow evaluation to see how a shared inference interface supports application-specific model choices.

Familiarity with API endpoints, chat messages, and the basic idea of retrieving documents to inform an LLM response will help you follow the demonstrations.

### Can AI solve the use case—and can it scale?

[1:29](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=89s)

Can AI solve this use case? Which model fits it best? And how can the application scale into production? A growing model catalog makes experimentation easier, but it also makes these three questions harder to answer through model reputation alone. The application needs a process for establishing feasibility, improving the result, and operating it reliably.

Start by **prototyping**: try several models, compare benchmarks, and shortlist those that can perform the task. Then **optimize** for the application’s cost, latency, and regional requirements, using prompt engineering, retrieval-augmented generation (RAG), or fine-tuning as appropriate. Finally, **operationalize** the application with capacity planning, cost management, monitoring, scalability, privacy, and content filtering. These stages form a loop: a promising prototype still needs application-specific optimization before it becomes a production workload.

![Three connected circular diagrams labeled Prototype, Optimize, and Operationalize ask whether AI can solve a use case, which model fits best, and whether workloads can scale.](https://ai.engineer/api/talk-essay-media/4c146b0da5c07afc68a2cb3d06eb0dd06de06c66725a3896178dcd7bc1a02af6)

Model choice across prototyping, optimization, and operationalization.

That process need not end with one model for everything. Sharmila asks whether customers use multiple models within a use case or across different use cases. Shubhi describes the catalog as a way to plug models into an application, compare them quickly, and replace one without rebuilding the surrounding code. **Model choice becomes useful when the cost of testing another model is low.**

### Choose capabilities, not just model size

[3:14](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=194s)

The catalog shown in the recording includes **GPT-4o**, **Mistral**, **Llama**, and **Cohere** models, alongside smaller models such as **Phi-3** and Mistral’s open models. Selection also extends beyond text generation to multimodal tasks, image generation, and embeddings. A model’s size is only one part of its suitability; its supported operations and the data it must handle matter too.

- **Tools and structured output:** GPT-4o’s function calling and JSON support serve agent workflows that must connect generated responses to application operations.
- **Language coverage:** Shubhi highlights Mistral for European languages and Cohere’s multilingual embeddings for multilingual retrieval. **Jais** supplies an Arabic-language option.
- **Open and proprietary choices:** Hundreds of Hugging Face models broaden the selection, with Meta, Databricks, Snowflake, and NVIDIA named as partners in bringing models onto the platform.

![Four columns list LLMs and SLMs; modalities, tasks and tools; regional and domain-specific models; and open and proprietary models, with provider logos below.](https://ai.engineer/api/talk-essay-media/5eaf71e91e575cfe1562050b72d8d33acd123226c4f3e22e8fc69194c05c7cef)

Azure AI model selection spans model sizes, modalities, regional needs, and open and proprietary models.

### Inspect a model, deploy it, and compare benchmarks

[4:25](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=265s)

The walkthrough begins in Azure AI Studio’s **Model catalog**. Shubhi reports more than 1,600 models in the catalog at recording time. Filters narrow that collection by deployment type, inference task, or model family, so discovery can begin with an application requirement rather than a long list of names.

Filtering for Cohere leads to **Cohere Command R**. Its model card describes customization, tool use, and RAG capabilities, and provides inference samples for integrations such as LangChain and LiteLLM. The card is therefore both a capability reference and a starting point for application code. Standardized APIs are intended to let those integrations work across the catalog’s supported models.

The deployment procedure then moves from model selection to a billable endpoint:

1. Review pricing and select **Deploy**.
2. Subscribe to the model’s Azure Marketplace offer, which supplies the billing arrangement for token usage.
3. Continue with an existing subscription when one is already in place.
4. Choose a deployment name, decide whether to enable content filtering, and submit the deployment.

Shubhi describes an expected setup time of under a minute for obtaining the endpoint URL and key.

While deployment runs, the walkthrough opens **Model benchmarks**. Accuracy, coherence, and groundedness provide common dimensions for narrowing the model shortlist. Returning to the deployment page, Shubhi reports that this deployment completed within a few seconds; its target URL and key are ready for use in application code. That observed deployment is the basis for the timing claim, rather than a service-level guarantee.

### Give the same question a better source

[7:12](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=432s)

Before moving into an IDE, the Chat Playground provides a way to test the deployed model interactively. Shubhi selects an existing [**Mistral Large**](https://azure.microsoft.com/en-us/blog/microsoft-and-mistral-ai-announce-new-partnership-to-accelerate-ai-innovation-and-introduce-mistral-large-first-on-azure/) deployment and asks how Microsoft promotes a culture of giving. Without custom data, the answer describes general initiatives, including employee matching programs and information available publicly.

Next, **Add your data** connects the chat to an existing index named Microsoft Give. The same question is submitted again, this time with the expectation that indexed documents will supply organization-specific context. The useful comparison holds the question and model constant while changing the information available to the answer.

While the response is pending, Sharmila asks whether Microsoft or the model providers use customer data to train models. Shubhi’s assurance is that inference prompts and completions are not shared with model providers and are not used to train the models. This concerns the handling of inference data; deliberately uploading data for a fine-tuning job is a separate operation.

The returned answer includes a concrete detail: the giving program provides a $50 new-hire credit. That detail was absent from the earlier response. The demonstration’s improvement is not simply a longer answer—it is an answer containing information from the connected index. The $50 amount is the playground’s reported output, not a separately established statement of current employee benefits.

### Separate infrastructure choice from the inference interface

[9:36](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=576s)

The deployment just demonstrated uses a serverless API. The catalog also offers managed compute, which changes both the billing unit and who must arrange capacity.

| Deployment option | Capacity responsibility | Billing | Recording-time selection |
| --- | --- | --- | --- |
| Managed compute | User obtains GPU capacity and manages quota | VMs per hour | Hundreds of open models |
| Serverless API | Ready-to-use hosted endpoint | Input and output tokens consumed | More than 30 flagship or premium models, according to Shubhi |

The serverless route is described through Azure OpenAI Service and Models as a Service. Managed compute makes GPU quota and capacity explicit application concerns; serverless APIs let the application begin with an endpoint and usage-based charges.

Above those deployment choices sits a shared inference contract. Azure works with model providers to standardize REST schemas and SDK structures, supporting integrations such as LangChain and provider-specific SDKs. The intended workflow is to change the endpoint, run evaluations, and compare the resulting application behavior.

**A shared schema provides interface portability, not identical model capabilities.** The later [Azure AI Model Inference REST API reference](https://learn.microsoft.com/en-us/rest/api/microsoft-foundry/modelinference/) makes that boundary explicit: models can reject unsupported parameters. Endpoint substitution therefore depends on compatible capabilities, including the tools or output formats the application requires. The recording uses the historical Azure AI Studio interface; the later reference’s API examples should not be treated as the exact version demonstrated here.

### Use a function for the shop’s actual bill

[11:08](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=668s)

A stationery-shop chatbot exposes a different grounding problem. The model may know what a stapler is, but it does not automatically know this shop’s prices or current discounts. Shubhi uses Mistral Large’s function-calling capability to connect the conversation to a **Get Bill Amount** function. The model identifies the operation and its arguments; the application’s function supplies the business-specific result.

The tool definition describes the callable operation to the model. Separately, the system message asks the assistant to help users work out what they must pay and to offer environmentally conscious advice about using the items. Those responsibilities remain distinct: the external function supplies the bill, while the system instructions shape the accompanying explanation.

For a request about 10 staplers, the model selects the bill function and supplies the item and quantity. In Python, the application-side handoff can keep that boundary explicit: validate the requested operation, call the shop’s pricing function, and package its result as a tool message for the next model turn.

```python
import json
from collections.abc import Callable


def run_bill_tool(
    tool_call: dict,
    get_bill_amount: Callable[..., dict],
) -> dict:
    function = tool_call["function"]
    if function["name"] != "get_bill_amount":
        raise ValueError("Unknown tool")

    arguments = json.loads(function["arguments"])
    item = arguments["item"]
    quantity = arguments["quantity"]
    if not isinstance(item, str) or not item.strip():
        raise ValueError("Item must be a nonempty string")
    if type(quantity) is not int or quantity <= 0:
        raise ValueError("Quantity must be a positive integer")

    bill = get_bill_amount(item=item, quantity=quantity)
    return {
        "role": "tool",
        "tool_call_id": tool_call["id"],
        "content": json.dumps(bill),
    }
```

Here, `get_bill_amount` belongs to the shop’s application. The model’s proposed arguments do not themselves establish a price; executing that function does.

The demo’s final answer prices 10 staplers at $45, including ongoing discounts. The notebook also shows environmental and safety reminders, reflecting the system message. This separates two sources of response content: a tool result for the amount owed and instructions for how the assistant should discuss the purchase.

![Jupyter notebook showing function-call arguments for 10 staplers, code passing the function response back to the model, and a final answer quoting $45.00 with reuse and stapling-safety advice.](https://ai.engineer/api/talk-essay-media/9ceda57445a4616b7b7e08354e8d499324721fce93de0292c7c6efa97f6eefd4)

The notebook returns a price for 10 staplers, followed by environmental and safety reminders.

The notebook returns to the broader endpoint-swapping workflow, while retaining provider-specific capabilities where needed. Its `safe_prompt` setting is set to `true`; this is a Mistral-specific option, not a universal safety switch for every model behind the common interface.

### Build retrieval as a flow, then vary the models

[13:42](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=822s)

The next demonstration moves from notebook code into **prompt flow**, building a shopping assistant with a retrieval pipeline. A basic version takes a user question, looks up relevant information in an index, and sends that context to an LLM for generation. Shubhi adds a query-transformation step before retrieval:

1. **Rephrase** the user’s succinct question into a more explicit search request.
2. **Lookup** relevant context in the index.
3. **Generate** an answer using the retrieved information.

Shubhi says that expanding short questions has produced better RAG results in their experience; no measured comparison is supplied.

With the compute session running, the example starts with: “Do you have any new hiking shoes?” The rephrase step expands the request to ask about hiking-shoe availability, materials, and features. The lookup output then exposes the context retrieved from the supplied index before that information is passed to the generation step. Inspecting these intermediate outputs makes it possible to distinguish a retrieval problem from an answer-generation problem.

The narration next points to a women’s jacket recommendation as the generated output. That recommendation does not match the earlier hiking-shoes request, so this sequence demonstrates the flow’s stages without establishing a successful answer to that particular question. The intermediate query, retrieved context, and final response need to be evaluated together.

To compare models without rebuilding the flow, the generate node supports **variants**. Its initial connection uses Cohere Command R; another variant can point the same node at a different model connection. Embeddings are another choice: the demonstration uses an **Ada** embedding model and proposes pairing Command R with Cohere embeddings. Shubhi calls the latter “Command Embed”; the public product name is [**Cohere Embed**](https://cohere.com/blog/embed-compression-embedjobs), and the exact proposed deployment identifier is not established here.

Previously run evaluations compare Cohere Command R, **Llama 3**, and Mistral Large. The purpose is to compare outputs from the same application flow across evaluation parameters. No numeric scores or winning model are reported, so the useful result is the evaluation setup: preserve the workflow, vary a model connection, and inspect how the application’s outputs change.

### Control data handling, model integrity, and access

[16:25](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=985s)

The enterprise discussion organizes the platform’s controls into three areas: data privacy, security and compliance, and responsible AI and content safety. Returning to the earlier inference-data assurance, Shubhi explicitly says it applies to both managed compute and serverless APIs: prompts and completions are not shared with model providers or used to train the models.

Model integrity adds another layer. HiddenLayer scanning is described as checking for embedded malware and backdoors, common vulnerabilities and exposures, and tampering or corruption across model layers. Shubhi ties the required checks to models labeled **curated by Azure AI**. That scope matters: a curated-model scanning process is not a guarantee that every model in the catalog is free of vulnerabilities.

- **Network isolation:** Private networking allows ingress and egress to be routed through virtual networks, with fully qualified domain name (FQDN) rules approving outbound access to non-Azure resources.
- **Model access:** Azure Policy integration supports allow-list and deny-list patterns for model collections.
- **Environment separation:** Different policies can govern development, test, and production environments.

These controls address different boundaries: where traffic can travel, which models an application may use, and which rules apply at each deployment stage.

### From internal assistance to customer-care automation

[18:34](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1114s)

The customer examples return to model choice across applications. EY began with models available through Azure OpenAI Service and was exploring Llama for more specific documentation and summarization tasks. Its EY.ai platform supports professional work where productivity and accuracy matter. Sharmila reports that EYQ chat had been adopted by 275,000 internal employees. The recording does not define whether adoption means access, registration, or active use.

EY’s operational lesson is that deploying AI is not a one-time selection exercise. Performance needs continuous evaluation, and production use must remain aligned with responsible-AI practices. Sharmila cites the catalog and Studio’s evaluation capabilities as reasons EY uses the platform: model selection continues after the first application ships.

CMA CGM, operating across sea, land, air, and logistics, provides a different example. Building on traditional robotic process automation, it used a Mistral model from the catalog for a chatbot scenario supporting customer-care agents. Sharmila reports reduced response latency and increased customer satisfaction for that use case, without numeric measurements. Planned extensions include invoicing, customer-document analysis, interpretation of free-form client text, and email writing.

### Forecasting broadens the catalog beyond chat

[21:16](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1276s)

Bridgestone’s example moves beyond conversational applications to monthly demand forecasting across a large product portfolio. It uses Nixtla’s [**TimeGEN**](https://www.nixtla.io/success-stories/microsoft) time-series model, which Sharmila describes as having launched on Azure at Build the month before the recording. The goals are to simplify forecasting pipelines, improve accuracy, and reduce operational complexity.

Sharmila reports that Bridgestone reduced forecasting errors by nearly 30% on average using TimeGEN. The recording does not specify the error metric, comparison baseline, or evaluation procedure, so the figure describes this reported customer result rather than an expected improvement for other forecasting workloads.

![Bridgestone logo beside a large 30% reduction in forecasting errors statement and three sections titled Results and Impact, Lessons Learned, and Future.](https://ai.engineer/api/talk-essay-media/063ada15b57563a6f79460d94f12d6ed0dd9df14968f0d94335304857b2e4f19)

Bridgestone case study: 30% reduction in forecasting errors.

The closing claim connects these customer applications to the opening lifecycle. Sharmila attributes a significantly shorter path from initial model use to business impact to the catalog and Studio’s integrated tools, without giving an elapsed-time measurement. The connection is practical: discovery and prototyping establish feasibility, optimization adapts the model to the task, and operational controls support production use. Keeping those activities connected lets an application continue evolving as its data, requirements, and model choices change.

## Resources

- [Azure AI Model Inference REST API](https://learn.microsoft.com/en-us/rest/api/microsoft-foundry/modelinference/) — Reference for shared inference requests, SDKs, model-specific parameters, and capability errors.
- [Cohere Embed](https://cohere.com/blog/embed-compression-embedjobs) — March 2024 announcement introducing compressed embeddings and asynchronous embedding computation.
- [Mistral Large launches on Azure](https://azure.microsoft.com/en-us/blog/microsoft-and-mistral-ai-announce-new-partnership-to-accelerate-ai-innovation-and-introduce-mistral-large-first-on-azure/) — Historical partnership announcement with customer accounts, including CMA CGM's internal assistant.
- [Nixtla TimeGEN-1 on Azure](https://www.nixtla.io/success-stories/microsoft) — Nixtla's overview of deploying its TimeGEN-1 forecasting model through Azure.

## Slides

### Presenting sponsor: Microsoft

Microsoft is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=5s) [Microsoft logo beneath the heading Presenting Sponsor.](https://assets.ai.engineer/sites/aiecode2025/sha256/9f227e03d5413a3f145480f3998b2dc3d19988d231e026e1f23d34ccc7102bb5.webp)

PRESENTING SPONSOR

Microsoft

### Innovation and platinum sponsors

AWS is the innovation sponsor; MongoDB, Google Cloud, and Neo4j are platinum sponsors.

[0:10](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=10s) [AWS is the innovation sponsor; MongoDB, Google Cloud, and Neo4j are platinum sponsors.](https://assets.ai.engineer/sites/aiecode2025/sha256/be5ed2653a9b9723aa68c2d793baccf2836206b4520b5b6323510fd4bb21db16.webp)

INNOVATION SPONSOR
aws

PLATINUM SPONSORS
MongoDB®
Google Cloud
neo4j

### Introducing Azure AI Model Catalog

The session introduces model choice, inference APIs, app building, enterprise readiness, and customer stories. The presenters describe the API as a way to swap models without disturbing the rest of the codebase.

[0:28](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=28s) [An introduction to the model catalog lists model choice, inference APIs, generative AI apps, enterprise readiness, and customer success stories.](https://assets.ai.engineer/sites/aiecode2025/sha256/e04f71cc057140c965090d815b8a1bac42bce62debcd0a0d0140ed5fa268e7ca.webp)

Introducing Azure AI Model Catalog

“Offering the best collection of foundation models on Azure”

Model choice matters
Azure AI Model Inference APIs
Building Generative AI apps
Enterprise readiness
Customer success stories

### Why model choice matters in the real world

A prototype loop connects feasibility, prototypes, use cases and a flagship model. The next questions are which model best fits the use case and whether it can scale to real-world workloads.

[1:49](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=109s) [Three model-choice questions appear alongside a prototype cycle for establishing feasibility, building prototypes, defining use cases, and picking a flagship model.](https://assets.ai.engineer/sites/aiecode2025/sha256/c515ad4b0537451f6616c6e31f285949944be33611251da57ad97f3e95691359.webp)

Why model choice matters in the real world

Establish feasibility
Build prototypes
Define use cases
Pick a flagship model

Can AI solve my use case?

What is the best model for my use case?

Can I scale for real-world workloads?

Prototype

### Azure AI offers the best model selection

Azure’s catalog pairs premium models such as Mistral Large with hundreds of open models from Hugging Face, alongside options for image generation, embeddings and forecasting.

[3:13](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=193s) [A model-selection slide lists examples of flagship large language models and small language models.](https://assets.ai.engineer/sites/aiecode2025/sha256/1f3f706e367cb04f716c42f8d6f1a51f09e0024398f54852219558ad480f1fa1.webp)

Azure AI offers the best model selection

LLMs and SLMs

Flagship LLMs
• GPT-4
• Mistral Large
• Llama3 70b
• Command R+

Small Language Models
• Phi3
• Mistral OSS models
• Llama3 8b

[4:08](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=248s) [Four columns summarize language models, capabilities, regional and domain-specific models, and open and proprietary offerings.](https://assets.ai.engineer/sites/aiecode2025/sha256/e0e0930d13c491acdd5c5bfe820e02ca3f5a4714b1bc5f7f588c5d82b07c7d52.webp)

Azure AI offers the best model selection

LLMs and SLMs

Flagship LLMs
• GPT-4
• Mistral Large
• Llama3 70b
• Command R+

Small Language Models
• Phi3
• Mistral OSS models
• Llama3 8b

Modalities, tasks and tools

Multi-modal
• GPT-4o, Phi3-vision

Image generation
• Dalle3, Stability AI

Embedding models
• Ada, Cohere

• Function calling & JSON support

Regional and domain specific
• Core42 JAIS Arabic language LLM
• Mistral Large is focused on European languages
• Nixtla TimeGEN-1 - Timeseries forecasting

Open and propriety

Premium models first on Azure: OpenAI, Mistral Large, Cohere Command R+

100s of Open models from HuggingFace

Open models from Meta, Databricks and Snowflake, Nvidia

### Azure AI Studio model catalog

The catalog lists 1,674 models and lets users narrow the selection by collection, deployment option, and inference task.

[4:34](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=274s) [The model catalog shows announcements, model listings, collections, and deployment and task filters.](https://assets.ai.engineer/sites/aiecode2025/sha256/87bf95d67b6dc0f3a334341ce6d68aa49c828a73333ad95a670438f47344f66c.webp)

Find the right model to build your custom AI solution

Announcements

AI21 Jamba-Instruct is here!
AI21's cost-effective model optimized for long context, now available on Azure.
View models · Read blog

Phi-3 Serverless APIs are now available!
Microsoft's latest Phi-3 SLMs offer groundbreaking performance at a smal...
View models · Read blog

GPT-4o is now available!
GPT-4o, the most advanced multimodal model from OpenAI, now available on...
View models · Read blog

All filters · Collections · Deployment options · Inference tasks · Fine-tuning tasks · Licenses
Search
Models 1674

gpt-4o — Chat completion
gpt-4 — Chat completion
dall-e-3 — Text to image
gpt-35-turbo-instruct — Chat completion
davinci-002 — Completions
text-embedding-ada-002 — Embeddings
gpt-4-32k — Chat completion
gpt-35-turbo-16k — Chat completion

Filter by
Collections
Curated by Azure AI · Azure OpenAI · Microsoft · Meta · Mistral · NVIDIA · AI21 Labs · Deci AI · Nixtla · JAIS · Cohere · Databricks · Snowflake · Hugging Face
Less

Deployment options
Managed compute · Serverless API

Inference tasks
Conversational · Fill mask · Question answering · Summarization
More

Fine-tuning tasks

Interface text:
Azure AI Studio · All hubs · santiagxf-demo-eus2 · All projects · intelligent-apps · Model catalog
To all projects
Current project: intelligent-apps
Project overview · Settings
Get started: Model catalog · Model benchmarks · Prompt catalog
Project playground: Chat · Assistants PREVIEW · Images · Completions
Tools: Code PREVIEW · Prompt flow · Tracing PREVIEW · Evaluation PREVIEW · Fine-tuning PREVIEW
Components: Data · Indexes
Hide

### Exploring Cohere Command R in Azure AI Studio

The demo filters the catalog to four Cohere models, then opens Command R’s model card. The card lists starter notebooks for inference and begins a section on RAG and tool use.

[5:06](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=306s) [The catalog is filtered to Cohere and shows two embedding models and two Command models.](https://assets.ai.engineer/sites/aiecode2025/sha256/665588a4ee6b9f2e7cbf13a47ad4d1cd404f7f7fb20d036257be9329f9ea3a97.webp)

Find the right model to build your custom AI solution

All filters
Collections: Cohere
Deployment options
Inference tasks
Fine-tuning tasks
Licenses
Clear all

Search
Models 4

Cohere-embed-v3-multilingual
Embeddings

Cohere-embed-v3-english
Embeddings

Cohere-command-r-plus
Chat completion

Cohere-command-r
Chat completion

Prev
Next

Filter by
Hide

Collections
Curated by Azure AI
Azure OpenAI
Microsoft
Meta
Mistral
NVIDIA
AI21 Labs
Deci AI
Nixtla
JAIS
Cohere
Databricks
Snowflake
Hugging Face
Less

Deployment options
Managed compute
Serverless API

Inference tasks
Conversational
Fill mask
Question answering
Summarization
More

Fine-tuning tasks

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Model catalog
To all projects
Current project: intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes

[5:26](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=326s) [Azure AI Studio model card showing inference sample notebooks and the beginning of RAG and tool-use samples.](https://assets.ai.engineer/sites/aiecode2025/sha256/ebcb1be8da018e75f0e312dd328c9acc08c27fe04b84f3226ce78367460785d1.webp)

temperature (and even greedy decoding) for code-generation related instructions.

For full details of this model, release blog post

How to use the model

Follow this article to deploy Cohere model with pay-as-you-go.

Learn more about the Cohere model's request schema here.

Inference samples

Packages | Sample Notebook
CLI using CURL and Python web requests - Command R | command-r.ipynb
OpenAI SDK (experimental) | openaisdk.ipynb
LangChain | langchain.ipynb
Cohere SDK | cohere-sdk.ipynb
LiteLLM SDK | litellm.ipynb

Retrieval Augmented Generation (RAG) and Tool-Use samples

Description | Packages | Sample Notebook
Create a local (FAISS) vector index using Cohere embeddings - Langchain | langchain, langchain_cohere | cohere_faiss_langchain_embed.ipyn
Use Cohere Command [illegible]

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Model catalog / Cohere-command-r
To all projects
Current project: intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes

### Command R RAG samples

The model card shows LangChain notebooks for creating a local FAISS index with Cohere embeddings and answering questions using either that index or an AI search vector index.

[5:36](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=336s) [Model card showing notebook examples for local vector indexes and retrieval-augmented generation.](https://assets.ai.engineer/sites/aiecode2025/sha256/5bd1ae2b607138915418f811ebbc96b448f0887245a29f58c6901c0b9bac2a32.webp)

LangChain — langchain.ipynb
Cohere SDK — cohere-sdk.ipynb
LiteLLM SDK — litellm.ipynb

Retrieval Augmented Generation (RAG) and Tool-Use samples

Description | Packages | Sample Notebook
Create a local (FAISS) vector index using Cohere embeddings - Langchain | langchain, langchain_cohere | cohere_faiss_langchain_embed.ipyn
Use Cohere Command R/R+ to answer questions from data in local (FAISS) vector index - Langchain | langchain, langchain_cohere | command_faiss_langchain.ipynb
Use Cohere Command R/R+ to answer questions from data in AI search vector index - Langchain | langchain, langchain_cohere | cohere-aisearch-langchain-rag.ipyn

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Model catalog / Cohere-command-r
To all projects
Current project: intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes

### Command R model overview

The presenter checks Command R’s serverless pricing, then selects Deploy. The card lists $0.0005 per 1,000 input tokens and $0.0015 per 1,000 output tokens.

[5:49](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=349s) [Command R model card with model details, serverless API pricing, and a deploy control.](https://assets.ai.engineer/sites/aiecode2025/sha256/3195e8117d5ad2b69a396d44dcfffaf01694bf675b0f5ec056f0e5367ee841fb.webp)

Cohere-command-r
Overview
Task: Chat completion
Languages: EN
Refresh
Deploy

Description
Model Details
Command R is a highly performant generative large language model, optimized for a variety of use cases including reasoning, summarization, and question answering.

Model Architecture: This is an auto-regressive language model that uses an optimized transformer architecture. After pretraining, this model uses supervised fine-tuning (SFT) and preference training to align model behavior to human preferences for helpfulness and safety.

Languages covered: The model is optimized to perform well in the following languages: English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Simplified Chinese, and Arabic.

Pre-training data additionally included the following 13 languages: Russian, Polish, Turkish, Vietnamese, Dutch, Czech, Indonesian, Ukrainian, Romanian, Greek, Hindi, Hebrew, Persian.

Context length: Command R supports a context length of 128K.

Input: Models input text only.

Output: Models generate text only.

Tool use capabilities
Command R has been specifically trained with conversational tool use capabilities.
These have been trained into the model via a mixture of supervised fine-tuning and

Serverless APIs
Provision an inference API within seconds through Models as a Service. Explore the model in the playground. Pay only for tokens consumed.

Pricing
paygo-inference-output-tokens: $0.0015 per 1000 tokens
paygo-inference-input-tokens: $0.0005 per 1000 tokens

Model ID
azureml://registries/azureml-cohere/models/Cohere-command-r/versions/3

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Model catalog / Cohere-command-r
To all projects
Current project: intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes

### Serverless API deployment

The selected project already has an Azure Marketplace subscription, so the demo can continue to deploy. Marketplace handles billing based on token usage.

[5:54](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=354s) [Serverless API deployment dialog showing project selection and an existing Marketplace subscription.](https://assets.ai.engineer/sites/aiecode2025/sha256/da873f632c7f079db742333af1862a28df31afaf295aafb428b35073818500c0.webp)

Serverless API deployment for Cohere-command-r  PREVIEW

Overview
Pricing and terms

Cohere Command R is offered by Cohere through the Azure Marketplace. View the pricing and terms tab to learn about pricing and terms of use.
Learn more about Models as a Service.

Select the project *
Create a new project
intelligent-apps
You already have a Azure Marketplace subscription for this project.

Continue to deploy
Cancel

Background text:
Description
Model Details
Command R is a highly [illegible]
variety of use cases incl[illegible]

Model Architecture: This [illegible]
transformer architecture [illegible]
and preference training [illegible]
and safety.

Languages covered: The [illegible]
English, French, Spanish [illegible]
Simplified Chinese, and [illegible]

Pre-training data additi[illegible]
Turkish, Vietnamese, Du[illegible]
Hebrew, Persian.

Context length: Comma[illegible]
Input: Models input text only.
Output: Models generate text only.

Tool use capabilities
Command R has been specifically trained with conversational tool use capabilities.

[illegible] in seconds through Models as a Service. Explore the [illegible]
[illegible] only for tokens consumed.

paygo-inference-input-tokens:
$0.0005 per 1000 tokens

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Model catalog / Cohere-command-r
To all projects
Current project: intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes
Cohere-command-r
Overview
Task: Chat completion
Languages: EN
Refresh
Deploy

### Command R deployment settings

The demo names the Command R deployment and shows the content-filter setting before deployment. The dialog says filtering uses the default configuration and is billed through Azure AI Content Safety.

[6:13](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=373s) [Deployment dialog showing a deployment name and a content-filter setting.](https://assets.ai.engineer/sites/aiecode2025/sha256/0b46baa25b64efeb389679802247b754071dd93d9dc043843c70efcb6865c9b1.webp)

Deploy Cohere-command-r  PREVIEW

Deployment name *
Cohere-command-r-utvua

Content filter (preview)
Enabled

Content filtering uses default configuration and is billed through Azure AI Content Safety. Learn more

Deploy
Cancel

Background page:
Cohere-command-r
Overview
Task: Chat completion
Languages: EN
Refresh
Deploy

Description
Model Detail[illegible]

Command R is a highly [illegible]
variety of use cases incl[illegible]

Model Architecture: This [illegible]
transformer architecture [illegible]
and preference training [illegible]
and safety.

Languages covered: The [illegible]
English, French, Spanish [illegible]
Simplified Chinese, and [illegible]

Pre-training data additi[illegible]
Turkish, Vietnamese, Dutch, Czech, Indonesian, Ukrainian, Romanian, Greek, Hindi,
Hebrew, Persian.

Context length: Command R supports a context length of 128K.
Input: Models input text only.
Output: Models generate text only.

Tool use capabilities
Command R has been specifically trained with conversational tool use capabilities.
These have been trained into the model via a mixture of supervised fine-tuning and [illegible]

[illegible] in seconds through Models as a Service. Explore the [illegible]
[illegible] only for tokens consumed.

paygo-inference-input-tokens:
$0.0005 per 1000 tokens

[illegible]-cohere/models/Cohere-command-

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Model catalog / Cohere-command-r
intelligent-apps
To all projects
Current project: intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes

### Model benchmarks overview

Azure AI Studio compares model accuracy across models and datasets. The chart rises from left to right, with colored points scattered around the line; filters help narrow the comparison.

[6:31](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=391s) [Azure AI Studio benchmark page with filters and a model accuracy chart.](https://assets.ai.engineer/sites/aiecode2025/sha256/d0ffb026b4e480f2a53eff31e4da56ba11ec8d887c0197928c4c78f5008dac51.webp)

Assess model performance with evaluated metrics

Quality benchmarks
Embeddings benchmarks

Compare benchmarks across models and datasets available in the industry to assess which one meets your business scenario.
Learn more about how model performance is scored

All filters
Tasks
Collections
Models
Datasets
Dashboard
List

Charts

Model accuracy
Accuracy scores are presented at the dataset and the model levels. At the dataset level, the score is the average value of an accuracy metric computed over all examples in the dataset. Learn more about accuracy.

Y-axis: Score
Y-axis ticks: 1, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3

Filters
Tasks
Question answering
Text generation

Collections
Azure OpenAI
Meta
Microsoft
Mistral
Cohere
Databricks

Models
databricks-dbrx-base
databricks-dbrx-instruct
llama-2-13b-chat
llama-2-13b
llama-2-70b-chat
llama-2-70b
llama-2-7b-chat
llama-2-7b
microsoft-phi-2

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-fr / All projects / intelligent-apps / Model benchmarks
To all projects
Current project: intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes
Hide

### Choose a playground deployment

In Azure AI Studio’s chat playground, the presenter selects an existing Mistral Large deployment to try the model before moving to an IDE.

[7:18](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=438s) [Chat playground with the deployment menu open and a Mistral model selected.](https://assets.ai.engineer/sites/aiecode2025/sha256/341ff7a28dc473888a0de536b1293bc4a18216fa4286ddb961a0c66ffaa806a0.webp)

Chat playground

View Code

Deployment *
mistral-large-ajmih

cohere-cmdR-plus-gyahe
gpt-35-turbo-16k
gpt-4
jais-30b-chat-gcbcv
Llama-2-13b-chat-blewg
llama-2-70b-chat-zrcgg
llama-3-70b-instruct-qgxvd
llama-3-8B-Instruct-fmyay
Meta-Llama-3-70B-Instruct-aggsn
mistral-large-ajmih
mistral-small-ofgon
phi-3-mini-128k-instruct-pkjcr
phi-3-mini-4k-instruct-5
Phi-3-mini-4k-instruct-iscuv

Chat session
Start typing here

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Chat playground
To all projects
Current project: intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes

### Chat playground: adding data

The Mistral Large deployment first gives a general answer about Microsoft’s giving culture. The presenter then selects the microsoft-give index under Add your data to ground a later response in project documents.

[7:44](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=464s) [Chat playground showing a submitted question and controls for selecting a project index.](https://assets.ai.engineer/sites/aiecode2025/sha256/4b6795ea1c9779600ddf4de9f62790dcb0d7e5e5af9ccabeccb954ecfceebd37.webp)

Chat playground
View Code

Deployment *
mistral-large-ajmih

System message
Add your data PREVIEW
Parameters

Ask questions about your own data. Your data is stored securely in your Azure subscription. Learn more about how your data is protected.

Select available project index *
Select available project index
Add a new data source

Chat session
How does Microsoft promote the culture of giving? Explain in short
...
Start typing here

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Chat playground
intelligent-apps
To all projects
Current project
intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes

[7:50](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=470s) [The chat playground shows a general answer while a data index is being selected.](https://assets.ai.engineer/sites/aiecode2025/sha256/366d3d8b34baed5f87e3fd06a8de5a6241371df806c060b49d8c897e7b0de0b7.webp)

Chat playground
View Code

Deployment *
mistral-large-ajmih

System message
Add your data PREVIEW
Parameters

Ask questions about your own data. Your data is stored securely in your Azure subscription. Learn more about how your data is protected.
Loading...

Select available project index *
microsoft-give
Add a new data source

Chat session
How does Microsoft promote the culture of giving? Explain in short

Microsoft promotes the culture of giving through various initiatives. Firstly, it has an employee match program where it matches employee donations to nonprofits up to $15,000 per year. Secondly, it offers a volunteer time off program that allows employees to take up to $25,000 worth of paid time off annually to volunteer. Thirdly, Microsoft provides technology and software donations to nonprofits and offers discounted rates for eligible organizations. Lastly, it runs various philanthropic campaigns and partnerships aimed at addressing societal issues such as digital skills training, accessibility, and environmental sustainability.

Start typing here

Interface text:
Azure AI Studio / All hubs / santiagxf-demo-eus2 / All projects / intelligent-apps / Chat playground
intelligent-apps
To all projects
Current project
intelligent-apps
Project overview
Settings
Get started
Model catalog
Model benchmarks
Prompt catalog
Project playground
Chat
Assistants PREVIEW
Images
Completions
Tools
Code PREVIEW
Prompt flow
Tracing PREVIEW
Evaluation PREVIEW
Fine-tuning PREVIEW
Components
Data
Indexes

### Model catalog deployment options

Managed compute deploys hundreds of open models to Azure VMs, with users managing GPU capacity and quota. Serverless offers 30+ flagship models through ready-to-use APIs, billed per token or provisioned capacity.

[9:35](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=575s) [A two-column comparison of managed compute and serverless model APIs, including deployment and billing approaches.](https://assets.ai.engineer/sites/aiecode2025/sha256/14c66b1c8f541d2ab0c33a6d66ccd6c5077f1953d303cf6543dbaac77dd9b1a3.webp)

Model catalog offers flexible deployment options

Managed compute
• Deployed to managed Azure VMs
• One click deployment that boots up an optimized container and initializes the weights.
• Pay-per-GPU billing
• 100s of open models

Serverless APIs with Azure Open AI Service and Models as a Service
• Ready to use model APIs
• Pay-per-token or pay-for-provisioned-capacity billing
• 30+ flagship models

### Azure AI model inference API and SDK support

A shared API and SDK structure lets developers swap model endpoints and compare results. The API also supports model-specific parameters, function calling and JSON, with integrations for common development tools.

[10:23](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=623s) [A slide presents a consistent API schema, advanced capabilities, and integrations with development tools and model-provider SDKs.](https://assets.ai.engineer/sites/aiecode2025/sha256/8e980b6e1afb09b5d37af22b3110d2957e5e6e1024631d539a720d96906a93a2.webp)

Azure AI model inference API works with favorite SDK

Consistent schema
• Unified API schema that works across all models

Advanced capabilities
• Custom parameters that are unique to each model
• Advanced capabilities like function calling and JSON support

Rich client support
• “Model inference SDK” for multiple languages
• Integration with LLM App development tools like prompt flow, Lang Chain, Semantic Kernal and more

Azure AI Prompt Flow + azure-ai-inference SDK (coming in early June)
Popular OSS tools like Lang Chain and Semantic Kernel
Model provider SDKs
Visual Studio Code

### Shopping assistant prompt flow

The shopping assistant rephrases a user’s question, looks up context, and generates an answer. The diagram shows all three steps completed.

[11:07](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=667s) [A shopping assistant workflow diagram shows input, rephrase, lookup, generate, and output steps.](https://assets.ai.engineer/sites/aiecode2025/sha256/3fd9cbedc217989ae701749dec49c12015b3e5dad5efebb920497575c8740878.webp)

Shopping assistant chat bot demo...

Inputs
Name: question
Type: string
Value: is there any slim fit jacket?

Outputs
Name: answer
Value: ${generate.output}
Name: content
Value: ${lookup.output}

rephrase
Completed
lookup
Completed
generate
Completed
inputs
outputs

rephrase
Connection: [illegible]
Api: chat
temperature: 1
stop
max_tokens: [illegible]
response_format: Please choose an option
Advanced
Function calling
Prompt — Referring to: rephrase.jinja2

# system:
You are a system designed to rephrase user questions in the form of intents. Given a question, rephrase and expand it in a way that it helps to provide a better answer later. Maintain all information in the original question. Do not engage on any type of conversation.

Interface text:
Azure AI Studio
All hubs
santiagoxt-demo-eus2
All projects
intelligent-apps
Flows
outdoor-demo-maas
Project overview
Settings
Model catalog
Model benchmarks
Prompt catalog
Chat
Completion
Assistants PREVIEW
Images
Code PREVIEW
Prompt flow
Tracing
Evaluation PREVIEW
Fine-tuning PREVIEW
Data
Indexes
Deployments
Content filters
Cloud
variant_0
View batch runs
View test results
Clone
Flow
Files
Show description
Show variants

### Function calling and tools notebook

The notebook defines a bill tool for a stationery-shop chatbot, with item name and quantity as inputs. The sample calculation uses the stapler price and a 10% discount.

[11:15](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=675s) [A notebook explains a stationery-shop function-calling example and shows the beginning of its tool definition.](https://assets.ai.engineer/sites/aiecode2025/sha256/60d142e74faefcc1509e7287c1947c5107051ca2043c75ea634e359e4d52d38c.webp)

Function Calling and Tools

Generally when users query an LLM, it returns responses based on it's existing knowledge, but sometimes users need context specific information or access to external APIs that the LLM does not have by itself. In this case function calling using tools allows LLMs to interface with user defined functions, APIs and external systems to generate these responses. First users define a set of functions or tools along with descriptions and usage instructions and then they send a prompt. Based on the user prompt LLMs can intelligently select and invoke the appropriate functions to accomplish these tasks.

Let's try to understand this using an example: Here we have the use case of a shop that sells several stationary items. The store sometimes has ongoing discounts and varying prices set by the owner. The store owner uses a chatbot for the user to inquire about the total price they have to pay if they want to buy certain items. In general an LLM by itself would not have the knowledge of the context specific store prices and discounts, and this is where we can make use of an external function. The LLM will recognize the need to use this external function will return the total price while adhering to the system message instructions we set.

[4]: shop_prices = {
    "pen":4,
    "pencil":3,
    "envelope":1,
    "ruler":2,
    "stapler":5
}

from mistralai.models.chat_completion import Function, ToolChoice

tools = [
    {
        "type": "function",
        "function": Function(
            name="get_bill_amount",
            description="Get the bill amount user has to pay for buying certain items. Ongoing discounts are taken care of",
            parameters={
                "type": "object",
                "properties": {
                    "item_name": {

Interface text:
Jupyter
Function_calling_demo
Last Checkpoint: last month
File  Edit  View  Run  Kernel  Settings  Help
Trusted
Markdown
JupyterLab
Python 3 (ipykernel)

[11:56](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=716s) [Jupyter notebook showing stationery prices, a function-calling tool schema, and a discounted bill calculation.](https://assets.ai.engineer/sites/aiecode2025/sha256/27dbf4a5d8ac73620869704b96767be8cbfa0cbf4d9255f2a14bb1803a6620a8.webp)

[4]: shop_prices = {
    "pen":4,
    "pencil":3,
    "envelope":1,
    "ruler":2,
    "stapler":5
}

from mistralai.models.chat_completion import Function, ToolChoice

tools = [
    {
        "type": "function",
        "function": Function(
            name="get_bill_amount",
            description="Get the bill amount user has to pay for buying certain items. Ongoing discounts are taken care of",
            parameters={
                "type": "object",
                "properties": {
                    "item_name": {
                        "type": "string",
                        "description": "Name of the item eg. glue",
                    },
                    "item_quantity": {
                        "type": "string",
                        "description": "Quantity of the item e.g. 5",
                    },
                },
                "required": ["item_name", "item_quantity"],
            },
        ),
    }
]

def get_bill_amount(item_name: str, item_quantity: str):
    price = shop_prices['stapler']
    discount = 0.1
    return str((price*float(item_quantity))*(1-discount))

Interface text:
Jupyter
Function_calling_demo
Last Checkpoint: last month
File  Edit  View  Run  Kernel  Settings  Help
Trusted
Markdown
JupyterLab
Python 3 (ipykernel)

### Requesting a stapler price

A request for 10 staplers produces a call to get_bill_amount with the item and quantity as arguments. The system also asks the assistant to give environmentally friendly responses and safety tips.

[12:13](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=733s) [Notebook instructions and chat request for ten staplers, followed by a tool-call result.](https://assets.ai.engineer/sites/aiecode2025/sha256/07487018b79d75852fc20d202721e79a2a6eb1397714795bd93b2691504ddf45.webp)

We instruct the system to:
1. Help users find how much they have to pay based on the item name and quantity
2. Keep the responses environment friendly
3. Give tips on how to use the items safely

And then we send a user message inquiring about 10 staplers.

[5]: messages = [
        ChatMessage(
            role="system",
            content="You are a helpful assistant that helps users find how much they have to pay to purchase certain items from a stationary shop. You care a[illegible]
        ),
        ChatMessage(
            role="user",
            content="When is the cost of 10 staplers including any ongoing discounts?",
        )
    ]

    chat_response = client.chat(
        model="azureai", messages=messages, tools=tools, tool_choice=ToolChoice.auto
    )

[6]: print(chat_response.choices[0].finish_reason)

    response_message = chat_response.choices[0].message
    tool_calls = response_message.tool_calls
    print(tool_calls)

FinishReason.tool_calls
[ToolCall(id='[illegible]', type=<ToolType.function: 'function'>, function=FunctionCall(name='get_bill_amount', arguments='{"item_name": "stapler", "item_quantity": "10"}'))]

[7]: tool_call = tool_calls[0]
    function_name = tool_call.function.name
    function_args = json.loads(tool_call.function.arguments)
    tool_call_id = tool_call.id

Interface text:
Jupyter
Function_calling_demo
Last Checkpoint: last month
File  Edit  View  Run  Kernel  Settings  Help
Trusted
Markdown
JupyterLab
Python 3 (ipykernel)

## Full transcript

[0:00](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=0s) [upbeat music] Hi everyone. Thank you for joining us for the session on how you can accelerate your AI journey with Azure AI Model Catalog.

[0:20](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=20s) This is Shubhi, I'm one of the product managers on Azure AI.

[0:23](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=23s) And I'm Sharmi. I'm one of the s- product marketing managers on Azure AI.

[0:29](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=29s) Great, so let's get started. So today, we'll be talking about how we offer the best collection of foundation models on Azure, why the model choice matters so much when you have such a huge collection of models, and how does Azure AI model inference API make it standard and easy for you to switch between multiple models, swap one

[0:47](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=47s) out for another without disturbing your rest of the code base, and then how you can build generative AI apps on top of it, and how the platform makes sure that all your enterprise readiness needs, like data privacy, security, and content filtering are met.

[1:01](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=61s) And then let-- we'll talk a bit about some of the customer success stories.

[1:05](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=65s) So Shubhi, as you mentioned, uh, there are a lot of large language models that are being, uh, released pretty much every day, every week. And, uh, I know in the Azure AI Studio and Model Catalog, we are bringing a lot of models, like large language models, foundational models.

[1:22](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=82s) I want to understand why are we bringing all, all these models, and why does it matter to a customer?

[1:29](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=89s) Yep, absolutely. That's a very valid question. So let's talk about why model choice matters in the real world, right? As Sharmila asked. So the three questions that we all try to answer when we try to build generative AI apps is, can AI solve my use case first, and then what is the best model for my use case,

[1:47](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=107s) and then how do I go about scaling this for the production workloads? So the first step is called prototyping, where you try, try out multiple models that are available to you.

[1:56](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=116s) You try to establish feasibility, build that prototype, do compare model benchmarks, and find the right one for your use case, or maybe at least shortlist some of them. And that's when you move to the next stage where you try to optimize it for your use case, where you try to optimize for cost, latency, regional nuances, et cetera.

[2:13](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=133s) And that's where the Azure AI platform comes into picture, where you can use the techniques like prompt engineering, RAG, and fine-tuning to make sure that you've optimized it. Once you're done through this loop of prototyping with the model catalog and optimizing with the platform, you can go ahead and operationalize it.

[2:27](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=147s) So you don't have to worry much about the capacity and cost trade-offs. You have to-- You have got monitoring, you've got scalability, you've got data privacy, content filtering, and all your enterprise needs are met.

[2:39](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=159s) So once you're through this, you have your generative AI application in production.

[2:44](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=164s) So another question I had here is, uh, have you seen ca- situations where a customer is using more than one model for a specific use case and even for multiple use cases?

[2:54](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=174s) Yeah, absolutely. Like, that's the whole point of the model catalog, that you have this wide range of models. You have your use case, and you're able to plug in any model in from that model catalog.

[3:06](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=186s) So you can swap out one for another without disturbing anything with quick prototyping and quick comparison.

[3:14](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=194s) So as we mentioned, let's talk a bit about the model selections that we offer on our platform. So we offer a wide range of flagship LLMs and SLMs. Recently, we launched the Azure OpenAI models.

[3:24](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=204s) The GPT-4o is already on the platform. We launched Mistral models, Llama models, Cohere models, as well as small language models like Phi-3 and the Mistral OSS models. Along with that, we make sure that your multimodal requirements, image generation requirements, and specific needs such as embedding model requirements are also fulfilled on this platform.

[3:44](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=224s) So GPT-4o, for example, has function calling and JSON support, so you can make sure you use that for your agent-centric workflows. Along with that, we also make sure that we cater to your region or language-specific needs.

[3:55](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=235s) So for example, the Mistral language model is really good with the European languages or, uh, the Cohere embedding multilingual ma- model is very good for your multilingual requirements. And we also recently launched Jais on our platform, which is an Arabic LLM.

[4:09](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=249s) Along with all these flagship and premium models, we also have hundreds of open models from Hugging Face, and we've been actively partnering with Meta, Databricks, Snowflake, and NVIDIA to make sure that we get their models on our platform as soon as possible.

[4:25](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=265s) Okay, so now that we have, uh, seen what all we can do with the catalog, let's try to see a live example of how to actually go to the catalog and deploy your models.

[4:37](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=277s) So once you type into your URL ai.azure.com, it's as easy. You land on this AI Studio page where you can go to the model catalog on this left nav bar, and you land on this page that has the list of all the models that we offer.

[4:50](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=290s) It, it has sixteen hundred plus models right here. And to make it easy for you to filter it for your use case, you can filter by the dif-different deployment types, different inference tasks that you want to do, or even if just by the model collection families.

[5:03](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=303s) So let's start out by filtering for the Cohere models. Like click on Cohere right here. Let's try to see Cohere Command R, for example. Once you click on the model, you land on this model card page where you have all the information about the model, how, uh, you can customize it for your own use case, tool use

[5:20](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=320s) capabilities. This one specific talks about RAG capabilities because the Cohere Command R model. It-- all model catalog pages also have these inference samples that you can use as starter codes to get started with, for example, a LangChain SDK or a LiteLLM SDK.

[5:37](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=337s) As we mentioned, like, we try to standardize the APIs across all use cases, so you can just plug in our APIs into any third-party application like the LiteLLM and have it working in no time.

[5:49](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=349s) So once you've gone over this, check the pricing, we go ahead and click on Deploy. And this is where we make sure that we are connecting to the Azure Marketplace.

[5:59](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=359s) So we use Azure Marketplace just for the billing side of things to make sure that you are billed correctly based on your token usage. And this is the step where you actually subscribe to that offer.

[6:09](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=369s) Here, I've already subscribed to that Microsoft subscription, so it's giving me the option to continue to deploy. It's as simple as choosing a deployment name, checking if you want to enable content filter or not, and clicking on Deploy.

[6:22](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=382s) So under a minute, you'll have your URL and key ready to get started. So while this is happening, let's look, uh, at other capabilities that we have, like model benchmarks.

[6:33](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=393s) So when you're in the Azure AI Studio, you also have the ability to check model benchmarks, which is right here on the left under Model catalog. Once you go in here, here I'm showing all the models that we have.

[6:44](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=404s) You can see that we try to benchmark on certain common characteristics like the model accuracy, model coherence, groundedness, et cetera. And this is the perfect place for you to filter out which model you want to choose based on the extreme selections of models that we offer.

[7:00](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=420s) So let's go back to check... Yeah, and we see... Go back to the deployment that we created, and it got created within a few seconds. We have our target URL right here and the key ready to use in any code base that you already have.

[7:12](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=432s) So now you may be thinking that before I move on to using my IDE, I want to try it out a bit, right? Is there something like a playground?

[7:19](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=439s) And that's why we also have this playground capability. So once... It's also in the Azure AI Studio on the left if you see we're right under the Chat Playground.

[7:27](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=447s) So let's see a live example right here. You can choose the model that you want to use in the playground. So in this deployment section, I've chosen a Mistral Large deployment that I already have in this project.

[7:38](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=458s) Let's try to, uh, chat with this model. Right here, it's not customized on any, any data. I'm just directly asking the model. So yeah, I'm trying to ask, how does Microsoft promote the culture of giving?

[7:48](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=468s) So this will, in general, give me a generic response about how it has a culture of giving through various initiatives. It has employee match programs and some generic information that's available online.

[7:59](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=479s) But what if you want to s-specialize it for your-- for our own data? So here we can go ahead and use the add your data functionality, where you can choose an a-available index.

[8:09](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=489s) So let's choose an available index that's called Microsoft Give. That tells it in specific that what, uh, what are the specific things that are very particularly known internally or may not be available generic circumstances.

[8:24](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=504s) If we send out the same question right here, we should get a more targeted response based on the documents in that index.

[8:33](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=513s) So just wait for a few seconds.

[8:37](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=517s) So Shubhi, while this is happening, I had a quick question. It's great that we are doing all this. I'm just curious because you mentioned data and data source and everything.

[8:46](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=526s) Are we using any of the data from our customers to train the models, or is Microsoft using it? Are our model providers using any of the data that a customer brings in?

[8:57](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=537s) So that's a great question, Sharmila, because that's a very common question we get from our customers. And no, we have very strict data privacy and policy rules in place.

[9:06](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=546s) Your prompts and your completions are not shared with the model provider, nor your data is used for training any of the models.

[9:15](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=555s) So yeah, looking back at the results, we see that it gave us a very specific response that says, "You get fifty USDs to start off with the new hire credit for the giving program."

[9:24](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=564s) And that wasn't in the response earlier. So with just the click of a button, we were able to link it to an index and get that response. So that's how the playground works.

[9:36](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=576s) So talking about the different ways of deployment, the one that we just saw was a serverless API option. So in the model catalog, there are two ways you can deploy a model.

[9:45](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=585s) One's called the managed compute, and one's called the serverless API option. With managed compute, the user is responsible for getting their own GPU. So you basically pay for the VMs per hour, and you're responsible for the quota management, capacity management, and you can use hundreds of open source models with this.

[10:02](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=602s) The second, uh, way you can deploy models is by getting a serverless API, and this is available with both Azure OpenAI service and Models as a Service. And this is what has about thirty-plus flagship models, premium models that you pay for based on your usage.

[10:17](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=617s) So you get ready-to-use APIs, and you only pay for the input tokens or the output tokens that you use.

[10:24](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=624s) We've also put in a lot of effort to make sure that we standardize the schema and the APIs of these models for you. So we've worked with the model providers to make sure that we build an SDK on top of a very standardized REST API system.

[10:37](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=637s) And such an, such an SDK works with common, uh, open sourced applications, things like LangChain, as well as the model provider specific SDKs. So all you have to have is a different endpoint, and every endpoint has the same API structure and the same SDK structure.

[10:55](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=655s) So you can just swap in one for another, evaluate, create multiple evaluations, compare the results, and choose the one that's perfect for your use case.

[11:07](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=667s) Yeah.

[11:08](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=668s) Awesome. So, um, we've been seeing everything about the model catalog and models. Can you show us an actual use case example?

[11:17](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=677s) Yeah. So, uh, let's briefly talk about how you can actually use these APIs in your IDE. Let's talk about the function calling example, and let's take the Mistral Large model for, for that use case.

[11:28](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=688s) So here I have a simple function calling example where I'm trying to que- use this model as a chatbot, for example, of an shop. The shop has-- sells certain stationery items, it has certain specific pricing, and may have certain ongoing discounts.

[11:41](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=701s) So if you just use a model as a black box, you will not get a specific pricing for the model, uh, for the shop or any of the ongoing discounts.

[11:49](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=709s) But what I'm doing here is using the function calling capability of this Mistral Large model to define a function called Get Bill Amount that can take in the specific information that we fed to it, recognize that it needs to call this external function based on the prompt, and smartly make that call, query that result, and give you

[12:07](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=727s) the exact information. So right here I've defined that function, I've defined the tool for that model for Mistral Large. We send in a prompt that says, "You're a helpful assistant that helps users find how much they have to pay."

[12:19](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=739s) And we also make sure that we tell it that you also care about the environment, and you also have to help users understand possible things they should be careful of when using these items.

[12:28](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=748s) So this is just to, uh, add more context to the response and see how the model can adjust based on the requirement. We go ahead, we send this response in.

[12:37](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=757s) We can see that the model has intelligently identified that it is calling the function get bill amount with the right arguments to identify that we queried for a stapler, and we'd asked-- tried to ask what is the price for the 10 staplers.

[12:49](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=769s) And if we see the chat response, it says, "The cost of 10 staplers, including any ongoing discounts, is $45," which is very specific, and it also makes sure that it res-re-reminds the users to be mindful of the environment and try to use staples when possible.

[13:03](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=783s) And this is end result of the system message that we sent to it when we asked it to be environmentally friendly and give users the right context. So similarly, if you see right here, you can swap any code base with, uh, any endpoint that you have, and without putting mu-much time into it or much effort into it,

[13:22](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=802s) you have a running API app right here. We also make sure that we use, uh, uh, model provider, um, fields like the safe_prompt setting to true. So on top of...

[13:34](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=814s) We always build on top of what the model provider capabilities are already existing.

[13:39](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=819s) Awesome. Thank you.

[13:42](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=822s) So now that we've talked about how we can set it up in the IDE, let's talk a bit about how you can set it up in the UI and how we can create a generative AI app using prompt flow.

[13:51](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=831s) So when you try to... Here I'm trying to create a shopping assistant chatbot using prompt flow, where it's a simple RAG application where we take in the user prompt, we try to get retrieval, uh, we retrieve context-specific information from our index, and then, uh, send it to the LLM to generate an output.

[14:07](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=847s) Here we've created the lookup step for it, which is basically doing the RAG part of it. The generate part is going to generate the output from the LLM. But we've added this extra step of rephrasing, where we're using the query transformation technique, where we take in the user con, uh, prompt, which is generally very succinct, but we

[14:25](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=865s) try to make it more verbose by rephrasing it, because we've seen better results of RAG with that.

[14:31](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=871s) So let's look at a live example of this right here. I have this prompt flow running right here, my compute session's running. We see that the first question that we ask is, "Do you have any new hiking shoes?"

[14:43](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=883s) But the rephrase step rephrases into a longer verbose output that says, "I'm looking for hiking avail-- hiking shoes available, and if so, what materials and features?" So it basically elongated that question.

[14:56](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=896s) We check the output of the lookup step, and we see that the prompt was able to get the context-specific information from the index that we provided to it, so it identified certain amount of information that we can now send to our LLM in the next step, and the LLM generates a response.

[15:11](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=911s) So based on that specific information, we were able to get this output that recommended the Fleece Fit, uh, Flex Jacket for the women. So we can see that we are able to generate a prompt flow end to end.

[15:24](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=924s) But you may be wondering that how do I make sure that I'm able to plug in different models into this flow? And this is where you can try to create variants.

[15:31](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=931s) So here you can see that in the generate step, I'm using a connection from the Cohere Command R model, but you can go ahead and choose any other connection to any other model and use evaluation to try to compare the results from the sa-- for the same flow for different models.

[15:46](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=946s) So example, for the, um, first step when we're trying to create the embeddings, here I've used the Ada model, but you can go ahead and try to see, okay, how does the Command R model work with the Command Embed model?

[15:57](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=957s) So you can create these variants and try to see the evaluation results. In the interest of time, I already ran some evaluations, as you also saw in the previous, uh, demo, and here we're trying to compare the Cohere Command R versus Llama 3 versus the Mistral Large.

[16:11](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=971s) And the evaluation capability helps you to compare the same model for the same flow on different parameters, and you can see how one fared against another.

[16:25](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=985s) So this is all great, and I think you touched upon data privacy a little bit. So can you go a little bit more into the details of what else do we have in the AI Studio or, um, Model Catalog to ensure customers' privacy and data security?

[16:40](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1000s) Yeah, absolutely. That's the key. So let's talk a bit about how we ensure that the data privacy and security compliance needs are met. So as I mentioned, there are three pillars to this.

[16:49](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1009s) So one's the data privacy part, second is the security and compliance, and the third is the responsible AI and the content safety. Talking about data privacy, for both managed compute and serverless APIs, your prompts and completions are not shared with the model provider.

[17:04](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1024s) Your prompts and completions are not used for training the models. No data is shared for training or with the model providers, so you can be assured that the AI platform makes sure that your enterprise needs for data privacy are met.

[17:18](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1038s) We also have this additional feature of adding HiddenLayer to our model scanning. So we make sure that we are finding the embedded malware and backdoors. We-- It scans for common vulnerabilities and exposures and detects tampering and corruption across model layers.

[17:32](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1052s) So for any model that's labeled curated by Azure AI, you can be assured that it's passing through the required checks.

[17:40](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1060s) Talking about security and compliance, in addition to the data privacy norms that we mentioned, we also offer the capabilities of adding private networking so that your data is not exposed to the internet.

[17:50](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1070s) So you have the control over routing your ingress and egress traffic through the VNets, and you also have the ability to set up FQDN rules so you can approve outbound access to non-Azure resources.

[18:03](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1083s) Um, in addition to this, you can also regulate access to models with Azure Policy integration. So you can have allow list or deny list patterns, and you can split out which model collections you want access to or not.

[18:15](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1095s) And you can also use these different policies for separating out the dev, test, and production environments.

[18:25](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1105s) Awesome. So, um, Shubhi, thank you so much for going through the model catalog and AI Studio, what, what are the features available in there and all, all these great demos.

[18:34](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1114s) So now I just wanna go into a few customer success stories and, uh, the... One of the main, uh, kind of underlying theme for all the customer success stories that I'm gonna show is that these customers are not just using one model in-- for their use case, they're using multiple models from the model catalog, and it could

[18:52](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1132s) be, uh, in one use case or across multiple use cases, similar to what Shubhi has shown in the demo. [lips smack]

[18:58](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1138s) And the first customer we're gonna talk about is [REDACTED:url] And, uh, they've been using, uh, our large language models. They've started off with the OpenAI models that was available in the Azure OpenAI service.

[19:09](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1149s) Um, they are doing that today, and they're also looking into Llama models, uh, where they are looking into Llama models for, like, really, uh, uh, task-specific use cases, like for documentation and for summarization and all that.

[19:23](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1163s) And, um, they're using our model catalog. They built [REDACTED:url]ai, which is a generative AI platform for EY professionals, which addresses the need for enhanced pro-- uh, productivity and accuracy in professional tasks.

[19:36](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1176s) And one of the key things that we wanna show is what's the result of what they've been doing. The EYQ chat that they built has been adopted by two hundred and seventy-five thousand employees internally and allowed the employees to perform a wide range of tasks efficiently and with great accuracy.

[19:53](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1193s) And what... Some of the lessons that they have learnt is it's not AI... Using AI in their use cases is not, like, a one-time thing. They need to do continuous evaluation of AI performance, and they wanna stick to all the responsible AI practices.

[20:07](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1207s) And that's one main reason why they've been using Azure AI, uh, uh, Model Catalog and Azure AI Studio. It's because they feel that they can easily do this evaluation and, um, make sure whatever they're doing, putting in production is going to be, uh, really an, uh, adhering to safe and responsible AI. [lips smack]

[20:27](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1227s) And then the next customer I want to talk about is CMA CGM. Again, they are a big, uh, global player in sea, land, air, and logistics solutions. And they are also building a kind of like a robotic process.

[20:40](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1240s) Uh, they've been doing traditional robotic process automation, and they've decided to use Mistral model from a model catalog, and they have built, again, a similar chatbot-like scenario, uh, for their customer care agents.

[20:52](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1252s) And, um, one thing that they have seen is they have seen a reduced response latency and increased customer satisfaction in their chatbot, uh, use case. And they plan to, again, extend the application of LLMs to encompass specific products for core business activities like invoicing, customer document analysis, uh, interpreting free client text, and, uh, writing emails and all

[21:16](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1276s) that. And finally, the last customer story I want to share is Bridgestone. Again, Bridgestone is a very popular name. Um, their, their use case is, uh, they have been using the Nixtla model from a model catalog.

[21:29](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1289s) We launched Nixtla, uh, time series model at Build last month, and it's a time series forecasting model, and they're using it specifically to predict monthly demand for a vast portfolio of products.

[21:41](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1301s) And, um, one of the things that they want to do is streamline their forecasting pipelines, enhance accuracy, and reduce operational complexity. And again, what they have seen is that they have seen that using the forecasting model like TimeGEN from Nixtla has helped them reduce, uh, errors by nearly thirty percent on average in forecasting errors, uh, which is

[22:02](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1322s) huge. And, uh, again, in all these customer use cases, the time it took for them to start using LLMs in their applications to seeing the results and impact has been reduced significantly because they were being able to use a model catalog and Azure AI Studio where we provide, as Shubhi showed in the very first slide, we do--

[22:23](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1343s) we have tools for prototyping, um, optimizing, and operationalizing. So whether you're just starting off with, "Let me try this for a prototype project," to realizing, "Okay, I need to put it into production," the time it takes from going from that to the last step has been reduced significantly, mostly because we have streamlined all the different, um, foundational

[22:43](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1363s) models. We have provided the right tools for all our customers to kind of go through that whole, uh, LLM life cycle. I think that's pretty much it. Uh, thank you.

[22:54](https://www.youtube.com/watch?v=gL9kfxt6uo0&t=1374s) Okay. Thank you. [upbeat music]
