AI Engineer
← All AI Engineer talks

AI Engineer World's Fair 2025

Building Agentic Applications with Heroku Managed Inference and Agents — Julián Duque and Anush DSouza

Julián Duque· HerokuAnush DSouza· Heroku52:35

Read the talk

From a browser notebook to managed agents on Heroku

Build up from a hosted Jupyter notebook to streaming inference, executable tools, read-only database analysis and MCP servers that other clients can discover and call.

From a talk by Julián Duque and Anush DSouza

Before you start: Basic familiarity with Python, HTTP requests and environment variables will help; following the deployment steps also requires a Heroku account with access to the relevant services.

010:18 ↗

Start with a notebook that needs only a browser

How far can you get building an agentic application without installing anything locally? This workshop starts by deploying Jupyter on Heroku, loading a notebook into it and attaching managed inference. The same notebook then becomes a client for model responses, tool execution and MCP servers. Heroku Managed Inference and Agents supplies the AI services; the browser supplies the working environment.

Julián Duque introduces himself as Heroku’s principal developer advocate, joined by Heroku AI product manager Anush DSouza. Participants get the slides and links through the Workshop Heroku AI channel in the AI Engineer Slack workspace. The workshop site’s QR code leads to the content and a form accepting the email associated with a Heroku account. At the event, that form provides temporary access to deployment and services without entering credit-card information. The hands-on work will happen inside the deployed notebook.

0:180:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:18 · section reference included

023:54 ↗

Make the agent loop an application resource

Heroku’s precedent is the Ruby on Rails application that was easy to write but difficult to deploy, operate and scale. git push heroku main compressed much of that operational work into a familiar developer action. DSouza applies the same framing to AI: getting a model response is only the beginning. Operating the application means choosing models, scaling execution, controlling tool access and understanding whether the system behaves as intended.

The proposed unit of integration is a managed agent loop attached to an application. Curated models sit inside a control loop with access to code execution and data; Model Context Protocol provides extensions. Model selection, evaluation and tracing remain operational concerns, but the platform aims to supply useful defaults rather than require developers to assemble every component. DSouza presents heroku ai models create as the CLI entry point for attaching AI as a resource.

Two columns list model selection, app integration, secure data access and ongoing operation alongside an integrated platform, Heroku DevEx and OpEx, curated AI models and MCP extensibility.
AI application challenges alongside Heroku’s proposed platform capabilities.

The platform combines three kinds of primitives:

PrimitiveRole in the application
Managed inferenceAccess a curated model from application code
MCP serversExtend agents with remote or stdio tools
Postgres with pgvectorStore embeddings for vector retrieval

Heroku dynos provide the compute for tool execution. A first-party tool can run there and stream its result back to the agent, while eligible tool execution scales to zero when idle. Developers can bring their own MCP tools into the same arrangement. At the time of the workshop, first-party web search is planned and memory is described as a possible future offering.

3:544:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:54 · section reference included

038:02 ↗

Deploy Jupyter with notebook persistence

The workshop website supplies step-by-step instructions for continuing later. Accepting the event invitation adds participants to the AI Engineer World’s Fair team in the Heroku dashboard; access is described as lasting through the seventh, over the weekend, with an extension possible. The team already contains a Jupyter workshop application, a Brave Search MCP and ai-engineer-data, whose PostgreSQL database will support the later analysis example.

To create the notebook application, Duque uses the Heroku-Jupyter template:

  1. Open the repository and select its Deploy to Heroku button.
  2. Enter a unique application name. jduque-jupyter is already taken in the demonstration, so Duque adds a workshop suffix.
  3. Select AI Engineer World’s Fair as the application owner to use the event’s funding.
  4. Set the notebook password through the deployment form’s environment variable, then deploy. Duque uses lab for the demonstration; a publicly reachable notebook needs a strong password.

The deployment creates a dyno for Jupyter and a Heroku Postgres database for notebook persistence. It fetches the template source and builds its dependencies. While deployment and the conference network take time, Duque uses an existing deployment to continue. The persisted object here is the notebook, not every piece of transient runtime state.

8:028:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:02 · section reference included

0413:03 ↗

Attach inference and load the execution context

The next resource is Managed Inference and Agents. Duque describes its appeal as keeping inference within Heroku’s infrastructure instead of directly calling an external model-provider API. That should not be read as a guarantee that inference runs inside the application’s own network: the later documentation describes Amazon Bedrock hosting and a boundary of secure AWS accounts.

In the dashboard, open the application’s management page, select Resources, and add Heroku Managed Inference and Agents. The catalog presented in the recording includes Claude 3.5, Claude 3.7 and Claude 4 for text, Cohere Embed for embeddings, and Stable Image Ultra for image generation. Duque selects Claude 4. This historical provisioning flow supplies an API URL, API key and model ID, usable through HTTP or an OpenAI-compatible SDK. Later standard-plan documentation instead lists the connection URL and key, with the model selected in requests.

Once Jupyter is ready, File → Open from URL imports the workshop notebook. In Heroku, Settings → Reveal config vars exposes the workshop’s three connection settings: INFERENCE_URL, INFERENCE_KEY and INFERENCE_MODEL_ID. The notebook proceeds through environment setup, basic and streaming inference, native agent tools, and finally MCP deployment and client access. Tool execution uses a one-off dyno: it starts, performs the operation and shuts down. Duque describes this as paying for the tool’s compute rather than keeping a tool application running continuously.

The Python setup also needs the application targeted by tools:

python

import os

inference_url = os.environ["INFERENCE_URL"]
inference_key = os.environ["INFERENCE_KEY"]
model_id = os.environ["INFERENCE_MODEL_ID"]
target_app_name = "ai-engineer-workshop"

Replace ai-engineer-workshop with the deployed application’s name. This selects where commands and compute run; naming an application does not itself grant access. The managed tools documentation requires the inference add-on to be attached to the target application before it can start dynos there. Database access will require its own attachment setup.

13:0313:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:03 · section reference included

0520:18 ↗

Inspect a completion, then consume its deltas

Basic chat completions take a message or an array of messages and return a model response. This endpoint can also request custom tool calls, but the caller owns their execution: application code must interpret the function call, execute it and provide the result. Frameworks such as ADK or LangChain can handle that loop. The first notebook example avoids tools and makes a direct HTTP inference request.

The payload contains the provisioned Claude 4 model ID and a user message asking for a one-sentence explanation of managed inference. Parameters such as temperature, maximum tokens and top_p tune generation, subject to the endpoint’s documented support. The response is a JSON chat-completion object: choices contains an assistant message explaining that managed inference handles deployment, scaling and maintenance of models for predictions.

Streaming changes how the response arrives. Instead of waiting for one complete JSON answer, the client receives server-sent events containing completion chunks. Duque first prints the raw events so their structure is visible. Each chunk carries a message delta, and an initial delta may have no text. The next example extracts those content fragments, prints them incrementally and renders the accumulated response as Markdown when generation finishes.

For decoded completion chunks, the notebook’s accumulation step can be written as a small Python function:

python

from IPython.display import Markdown, display


def render_completion(chunks):
    parts = []
    for chunk in chunks:
        for choice in chunk.get("choices", []):
            content = choice.get("delta", {}).get("content")
            if content:
                print(content, end="", flush=True)
                parts.append(content)

    print()
    answer = "".join(parts)
    display(Markdown(answer))
    return answer

Incremental printing supplies feedback while generation is underway. The final Markdown pass formats the completed answer, including any code blocks the model returns.

20:1820:31
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

20:18 · section reference included

0624:46 ↗

Move tool execution into the managed loop

The agents endpoint adds managed execution around inference. The available tools cover several distinct operations:

ToolOperation
Dyno Run CommandRun a Unix command or an existing application script
Postgres Get SchemaRetrieve the database structure
Postgres Run QueryExecute SQL and return data
HTML to Markdown / PDF to MarkdownExtract text from a supplied URL
Code ExecutionGenerate and run Python, Node, Ruby or Go code

Dyno Run Command is useful for trusted code already deployed with the application. The database tools form a natural sequence: discover the real schema before generating a query. Document conversion provides text the model can work with, while code execution returns a computed result to inference.

Generated-code tools also accept a packages array, allowing dependencies such as Pandas or NumPy to be installed before execution. Duque describes the native tools as MCPs maintained by Heroku; custom MCPs extend the same endpoint. In the demonstrated API, agent responses are streaming only because tool work may take time. Application and database permissions depend on the tool being invoked.

24:4624:57
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

24:46 · section reference included

0728:27 ↗

Ask the server for its current time

The first agent question asks for the server’s current date and time. A model’s training data cannot establish the live clock of this particular server, so the request enables Dyno Run Command. The payload includes the model ID and message, identifies the tool as a Heroku tool, selects the target application, and supplies date as the command with a description of its output.

The notebook sends the payload to the agents endpoint and extracts tool-call information from the stream. Execution targets ai-engineer-workshop; the notebook waits for the command, receives the date and time, and then displays the inference response based on that result. Replacing date with a deployed script would use the same mechanism for data extraction or access to an external service. The tool supplies current evidence before the model composes the answer.

Jupyter notebook output shows agent tool execution, a message about checking the server time, and the returned date and time above the next Code Execution demo.
The notebook displays the server date and time returned by the agent’s tool call.
28:2728:36
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

28:27 · section reference included

0830:56 ↗

Keep the computation, change the runtime

Code execution lets the model produce the program as well as request its execution. Duque enables CodeExec Node and asks for the 13th Fibonacci number. The notebook displays the generated JavaScript with syntax highlighting, then receives the result from execution on Heroku. The Node demonstration returns 233 for the 13th Fibonacci number.

Using the convention F₀ = 0 and F₁ = 1, the computation fits in a short JavaScript program:

javascript

function fibonacci(n) {
  let a = 0;
  let b = 1;
  for (let i = 0; i < n; i++) {
    [a, b] = [b, a + b];
  }
  return a;
}

console.log(fibonacci(13));

Duque then changes the selected tool from Node to Go. The model generates Go code, Heroku compiles and runs it on a dyno, and the response gives the same result with an explanation. Python and Ruby are further runtime choices; participants are encouraged to change the requested operation as well as the language.

30:5631:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

30:56 · section reference included

0933:19 ↗

Retrieve an implementation before running it

The next example gives the agent two tools: HTML to Markdown and Code Execution Python. The request is specific about both the source and the task: use the Python snippet from Wikipedia’s Euclidean algorithm page to calculate the greatest common divisor of 252 and 105. That instruction makes retrieval part of the computation instead of leaving the model to produce an implementation from memory.

The managed loop now crosses a dependency between tools:

  1. Fetch the Euclidean algorithm page as Markdown.
  2. Read the page and extract its Python implementation.
  3. Prepare and execute Python using the requested inputs.
  4. Return an explanation grounded in the retrieved implementation and execution result.

The notebook shows page retrieval, then code generation and execution. Duque points out the additional wait involved in multiple tool runs, invites participants to inspect and modify the code, and finally receives an explanation describing the implementation and calculation. The important extension is that the output of retrieval becomes input to execution.

33:1933:31
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

33:19 · section reference included

1035:46 ↗

Attach a read-only database before querying it

The notebook application initially lacks access to the shared solar-energy database. Before enabling database tools, open the team’s ai-engineer-data application and its Resources page. Heroku’s maintained Postgres tools operate only on read-only followers. Duque explicitly leaves the primary database untouched; write access through a separately deployed MCP would be a different capability with risks borne by its operator.

On the follower, select Manage Attachments and add the Jupyter application. Then inspect the consuming application’s Resources page to find the database’s attachment name. One application uses DATABASE; Duque’s ai-engineer-workshop uses HEROKU_POSTGRESQL_AQUA. These application-specific names identify the connection configuration that the tool should use. The selected dashboard frame shows this attachment mapping.

Heroku Resources page with a Postgres attachment popover listing four application attachments, including HEROKU_POSTGRESQL_AQUA and DATABASE.
The Heroku dashboard shows database attachments and their application-specific names.

An attachment makes the database available to the application; a name selects that available database. Tools cannot reach another application’s database merely by naming it. If the appropriate database is already attached locally, no additional attachment is needed. This keeps resource access separate from the natural-language question the agent will receive.

35:4636:00
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

35:46 · section reference included

1139:16 ↗

Discover the schema, then investigate energy savings

Postgres Get Schema and Postgres Run Query receive the target application and database identifiers. Duque describes the example dataset as roughly two months of hourly energy consumption and production for a solar-energy company. The question is how much energy has been saved in the last 30 days. To answer it, the agent must discover the relevant tables and fields, generate SQL, execute the query and interpret the returned data.

The first tool retrieves the schema, revealing tables including metrics, products, systems and users. The agent then generates and runs its query. Duque expresses complete confidence that schema knowledge will make the query run, but schema discovery grounds table and column references; it does not guarantee correct SQL or a correct interpretation of energy savings. The demonstration proceeds from returned data to further investigation.

The agent asks for additional queries to break the result down across three systems with different performance. Its final report includes savings, per-system results and an energy deficit. This is the useful behavior of the loop: one user question can trigger schema discovery and several queries before the final answer. Duque then proposes adding Python execution to generate a Matplotlib chart from the data; that chart is an extension, not a completed part of this demonstration. More tool calls also increase the total time required. A further exercise combines PDF extraction with code execution.

39:1639:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

39:16 · section reference included

1346:07 ↗

Give an existing MCP a Heroku entry point

For deployment, Duque uses a fork of the official Perplexity Ask MCP. The main adaptation is a Procfile, Heroku’s declaration of how application processes start. He adds an MCP entry point and explains that process types beginning with mcp are recognized as MCP processes. The server continues to communicate over standard input/output. A deployment button packages this into a Node.js application that can be attached to the consuming app.

Duque does not have a valid Perplexity API key and enters a placeholder for the deployment exercise. That lets him demonstrate building and making the application available, but it does not establish authenticated Perplexity execution. Participants are left with an exercise: attach the deployed server and write the code to invoke it with working credentials.

46:0746:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

46:07 · section reference included

1447:58 ↗

Call the same tools from an MCP client

After the earlier Brave news response completes, the workshop turns to consumers outside the managed agent endpoint. Cursor, Claude Desktop and agents built on other platforms can use the hosted tools through Toolkit Integration. In the recording, this gateway aggregates attached MCPs behind a server-sent events endpoint authenticated with a bearer token; OAuth support is described as work in progress.

Duque adds the newly deployed Perplexity server to the toolkit and uses the same token as inference. The notebook installs the Python MCP package, creates a client for the toolkit URL and sends the key in an Authorization header. The client then lists the tools and calls Brave Web Search. Although the notebook itself is hosted on Heroku, this demonstrates access through an ordinary MCP client rather than through Heroku’s managed agent loop.

The client connects and discovers Brave Local Search, Brave Web Search and Perplexity Ask. It executes Brave Web Search and prints the returned search content without further processing. Perplexity’s appearance in discovery shows that the server is exposed through the toolkit, while the successful invocation shown here is Brave’s. Duque also describes using the same gateway approach from Cursor.

Jupyter notebook output shows “MCP Client connected!”, a list of available tools and a block of search-result text beneath the client code.
The MCP client displays its available tools and a returned search result.
47:5848:03
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

47:58 · section reference included

1550:50 ↗

Use the OpenAI SDK, then continue experimenting

The final technical example returns to inference through the OpenAI SDK compatibility interface. Duque estimates chat-completion compatibility at 95%, with 99% as a goal; he supplies no measurement method. Some parameters are unsupported, so field-level compatibility matters more than that estimate when porting an application. The demonstrated integration configures the OpenAI SDK with Heroku’s API key and URL.

In Python, the client configuration and basic request have this shape:

python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["INFERENCE_KEY"],
    base_url=os.environ["INFERENCE_URL"].rstrip("/") + "/v1",
)

response = client.chat.completions.create(
    model=os.environ["INFERENCE_MODEL_ID"],
    messages=[{
        "role": "user",
        "content": "Explain the concept of Managed Inference in one sentence.",
    }],
)
print(response.choices[0].message.content)

The SDK call in the recording returns an inference response. By this point, the notebook has exercised both interfaces: familiar chat completions for model output, and a managed agent endpoint that runs tools and feeds their results back into inference. The toolkit additionally exposes those tools to independent MCP clients.

The workshop closes with continued experimentation rather than another deployment step. Duque extends the event team’s access through the weekend and explains that Heroku has no free tier at the time of the talk, with a return under consideration. He points participants to Dev Center documentation, the Heroku AI website, the Heroku community on Twitter/X and the slides shared in Slack, with questions welcome through social follow-up.

50:5051:03
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

50:50 · section reference included

Resources

From the talk

  • Heroku-JupyterRepositoryTry it ↗10:11

    Deploy Jupyter Notebook or JupyterLab on Heroku with password protection and PostgreSQL-backed notebook storage.

Updates since the talk

Read the complete timestamped transcript
  1. 0:00

    [upbeat music] Welcome, welcome you all to this Lunch & Learn.

  2. 0:18

    I hope you are enjoying lunch right now that the conference brought you. Um, we are going to be talking about our recent, uh, product that we announced at Heroku, which is Heroku Managed Inference and Agents or part of the Heroku AI offering.

  3. 0:35

    And we will be talking about the fundamentals of building agentic applications with this service. This is not a session, this is a workshop, so if you wanna follow along, just bring your laptop.

  4. 0:47

    You only need a browser. You are not going to install anything in your computer. I swear that it's going to be, uh, easy to follow along. Also, this content is going to be available for you if you wanna continue learning at home and, um, having access to the Heroku platform during the event and the weekend.

  5. 1:09

    So initially, uh, we are going to do the setup while you get ready by signing up to an Heroku account, or if you already have an Heroku account, you can just get access into the platform.

  6. 1:22

    I will give you a link for you to follow. Then my friend here is going to give you an overview of what we release, and then we will go and do the hands-on.

  7. 1:32

    My name is Julián Duque. I'm a principal developer advocate for Heroku. I'm here, I am with Anush.

  8. 1:37

    Yeah. Hi, my name is Anush. I work as a product manager for Heroku AI. Uh, thank you everyone for joining us. Very excited to talk to you guys as well as walk you through how simple Heroku is.

  9. 1:48

    Of course. And let's get, uh, started. You can join the, uh, Workshop Heroku AI channel in the AI Engineer Slack, uh, workspace. There I share the slides and some links.

  10. 2:02

    Also, this is the QR code that is going to take you to the workshop site. So in the workshop site you are going to find a link that has the content and a form when you enter your Heroku account email.

  11. 2:18

    If you already have an Heroku account, just enter your email that you used to sign up to Heroku. If you don't have an Heroku account, go ahead, sign up for one.

  12. 2:28

    You don't need to put, like, any credit card information. And with that form you are going to get access to Heroku. You will be able to deploy applications, use our services during the duration of this week.

  13. 2:42

    If you have any problems with the sign-up process, let us know. We are here to help. And this is a follow-along workshop. What are we going to do is deploy a Jupyter Notebook to Heroku, and then we are going to load all of the workshop contents into this Jupyter Notebook, and from there I will take you, uh,

  14. 3:03

    with the workshop material. So let's take a picture of this. Also make sure you join the Slack channel. There is the information in there. And while you are doing the setup, my friend Anush here is going to giving...

  15. 3:16

    give you an overview of Heroku AI and what we release.

  16. 3:21

    Okay. Um, thank you for joining us, friends. Uh, how many of you are familiar with Heroku? Have you used Heroku before? Show of hands. Oh, nice. So, um, to set the stage,

  17. 3:36

    this is the most exciting time to be building right now, and especially building with AI.

  18. 3:42

    And this point has come before, right? There have been inflection points with technology that have fundamentally changed the way we build things. There was the internet, there was the cloud, there were web apps, and now there's AI.

  19. 3:54

    Previously when Heroku started, we took a similar challenge. People wanted to build and deploy their apps, especially with Ruby on Rails, but it was hard. It was hard to operate it, it was hard to deploy it, it was hard to scale it.

  20. 4:06

    Heroku made that super simple with Git push heroku main, and a whole new host of developers could easily build, push, and scale their apps. We are doing a similar thing right now where we have taken on the challenge with AI.

  21. 4:19

    We have seen that people wanna build with AI. They wanna build agentic applications. They wanna build agents that scale and operate in a way that is very simple. We wanna make sure that every software engineer right now is an AI engineer, and it is simple as attaching agents and AI to your apps.

  22. 4:37

    So how are we doing that? Currently a lot of solutions are only for the day one problems. But what happens on day two? How do you operate it? How do you scale it?

  23. 4:47

    There are so many models out there. How do you know this is the right model for your problem? How do you know your tools are running safely? Heroku has taken a very opinionated and curated set of models that we believe work best for our customers and developers will enjoy.

  24. 5:04

    We have expanded that further by deeply entrenching these models in an agentic control loop that runs on Heroku that has access to tools like code execution, access to your data, all under the trust layer of Heroku.

  25. 5:16

    And for these extension of agents, we are using the Model Context Protocol.

  26. 5:22

    You might have seen online that people keep asking that who's gonna build the Heroku of AI or who... uh, this is the Heroku of X. So the question for who's building the Heroku of AI, it's Heroku of course.

  27. 5:34

    Why wouldn't we? Okay. So what are the challenges right now people are facing? One of the things that I can see is that how do you, how do you figure out that this model works best for you?

  28. 5:47

    How do you know that it's evaluated and traced and has the right technologies to make sure that it is performing the way you want it to perform? So these are the challenges that we're taking on.

  29. 5:57

    We are curating these things such that whenever you work with the agentic applications you don't feel like it's a bunch of knobs and bells and whistles like a plane, but it's actually s- pretty simple.

  30. 6:07

    By taking on, uh, the opinionated approach of having those defaults for you, you can get started with as simple as one CLI command. You, you just do He-Heroku AI models create, and that attaches AI to your app as a resource.

  31. 6:21

    Uh, moving on to the next slide.

  32. 6:23

    Of course.

  33. 6:24

    Okay. So we have three major things that work great for our AI application or building agentic applications. One is we offer primitives like inference, where you can curate and take a set of models and access this in your apps.

  34. 6:38

    We also have Model Context Protocol to extend your apps. You can build remote MCP servers on Heroku simplified way, and you can also build standard IO MCP servers that run in Heroku's trusted compute.

  35. 6:50

    And they'll... They can also scale to zero so that you do not pay for things that you're not using. And we also have had pgvector, which is a great vector database for embeddings.

  36. 7:00

    So all, all these together give you all the primitives that you need to build agentic applications.

  37. 7:08

    So how do we do this? Heroku has a trusted compute layer called dynos, and the-these compute is w- the one that runs tools for you. So ex-example, we f- we provide first-party tools, like code execution, that run on Heroku's compute, and these can run on the compute and stream the data back to you and solve your problems.

  38. 7:32

    We plan to offer our additional, uh, first-party tools, such as web search for grounding. Uh, memory. Memory is really important right now, so in the future we'll probably offer memory and other compute tools that can run on Heroku's compute and provide to you.

  39. 7:46

    You can bring your own tools as well using MCP that can run on our compute and stream things back to agents.

  40. 7:53

    Okay. With that, I'll hand over to Julián for a more hands-on workshop, and you can build all the things I spoke about within the next 30 minutes.

  41. 8:02

    Okay, let's go back to this last, uh, picture for the folks that just joined so you get access to the workshop content. And remember, we have the Slack channel in the ai.engineer workspace where I already shared these slides and the link so you can follow along.

  42. 8:19

    You can follow along with me right now, or you can follow along at home with more time.

  43. 8:25

    So what are we going to do is the following, so let me show you. That link is going to take you to this page. In this page, you have access to the workshop resources.

  44. 8:36

    This is a website that has the instructions step by step of what you are going to be doing today. I'm going to be following these same steps right here, right now.

  45. 8:45

    And second, we have a way for you to get access to Heroku. If you already have an Heroku account, just put your email, the email that you use to log into Heroku.

  46. 8:57

    If you don't have one, go and sign up first. You can sign up for Heroku. You don't need to enter your credit card information. With just this invitation, you are going to get access to the workshop.

  47. 9:09

    These services are going to be enabled until, uh, the seventh, like this weekend, and I can extend it if you like the content we have for you. This is going to send you an invitation to a team.

  48. 9:21

    The, the, the team is called AI Engineer World's Fair. Go accept that invitation, and then you have access to this dashboard. So from the Heroku dashboard, you go to dashboard.heroku.com, you log in, and you should get access to this view.

  49. 9:39

    We already have, like, some pre-deployed applications here. The workshop that I will be showing you, the Jupyter Notebook, the Brave Search MCP. We are going to use a preexisting MCP to give you an example of how to call that MCP from the Managed Inference and Agents endpoint.

  50. 9:59

    And I also have another application here, which is the AI Engineer Data that has an Heroku PostgreSQL database, and we are going to see how can we build agents that has access to this database as well.

  51. 10:11

    But we start the workshop first by deploying an Heroku application. So we have this Heroku Jupyter template that you can deploy to Heroku. Go to the repository.

  52. 10:26

    After you have the account and logged into Heroku, just click to deploy. Click to deploy it.

  53. 10:37

    You are getting this, uh, page that will ask you for an application name. Make sure that this application name is unique.

  54. 10:47

    And the internet situation is a little bit slow. Let's hope it gets better

  55. 10:54

    to show you the things working live. Otherwise, I, I need to start getting access somewhere.

  56. 11:02

    If you join this link, please go to Slack channel. It's called workshop Heroku.

  57. 11:07

    Heroku-

  58. 11:07

    Yeah, AI. Yeah

  59. 11:08

    ... Heroku AI workshop. And in the AI Engineer Slack channel, you can find the how to get started instructions. If you guys are stuck somewhere or you don't know how to get started, please raise your hands and I can come help you out.

  60. 11:20

    Beautiful. This is the deployment page. We are going to create a new application. Remember to have a, like, a unique name. Let's do... This is [REDACTED:username]. This is going to be my Jupyter.

  61. 11:32

    It is not unique. So if it is not unique, you are going to get an error. So let's do workshop to get a, a unique. And for the app owner, if you want this workshop to work without you paying anything, make sure to select the AI Engineer World Fair, which is the team we have invited you to.

  62. 11:55

    These, uh, Jupyter Notebooks are password-protected, so there is an environment variable here for you to define the password to this workshop. I'm going to use a super secure password, lab, and then click to deploy.

  63. 12:10

    So this is going to deploy the application with a, a dyno where the Jupyter Notebook is going to run. The dyno is pretty much the- Container unit where your application runs, like the virtual machine on Heroku.

  64. 12:25

    And it also has Heroku Postgres, so all the work that you are going to do in the Jupyter Notebook is going to get persisted into this database. And this is going to take a little bit of time to, to install.

  65. 12:38

    This is pretty much, uh, fetching all the source code of this Jupyter Notebook, uh, template to Heroku, building that application and all of the dependencies, and then you have this available for you to use.

  66. 12:52

    So basically you are going to get something like this. I already have this deployed, but you don't have any workshop here. So we are going to be loading that notebook.

  67. 13:03

    And to load that notebook, you go and keep following the instructions. Oh, the third step that I miss is how can we provision Managed Inference main- mainly how can we provis- provision an AI model to an Heroku application?

  68. 13:22

    So Managed Inference is a service that let you run AI models within the same infrastructure where your application is running. So the data is not going to third parties.

  69. 13:33

    If you are using, let's say, like, OpenAI or Anthropic APIs, the data is going outside your application. This is keeping, uh, everything running inside the same network that your application is running.

  70. 13:47

    And to provision this you can use the Heroku CLI, but we are going to make things easy. Once the application is deployed... This is taking, uh, time to deploy, so let's go to something that is already deployed.

  71. 14:00

    Let's say this one that I, I already, uh, created before. You go to your application in the dashboard, you click to, uh, the management page, then you go to Resources.

  72. 14:14

    And in Resources is where you can provision add-ons. Add-ons like Heroku Postgres, or the Key Value store, or a- any other third party add-on that we support on our marketplace.

  73. 14:25

    The one that we are going to provision is called Heroku Managed Inference and Agents. This is our AI offering. So you are going to click on this add-on, and then you will select the model that you want to use in your application.

  74. 14:40

    We support text-to-text- models from Anthropic, Claude 3.5, 3.7, and 4. We support Cohere Embed for embeddings, and we support, uh, Stable Image Ultra for image generation. We are going to be working mostly with text-to-text with inference models, so let's do Claude 4.

  75. 15:01

    We submit, and that's it. Now my application can access AI services. That's the only thing that you need to do. And how this works, this is going to enable your application to have access to environment variables that contain the API URL, the API key, and the model ID.

  76. 15:22

    And with those three things you can use, like, an existing SDK that supports the OpenAI specification or the OpenAI API, or you can perform, like, an HTTP request or build your own solution to, uh, run these applications.

  77. 15:41

    So now we provision AI into our app.

  78. 15:45

    Now our another app is deployed, so we can do the same with that, so I'm going to keep following. Then let's go and load the workshop. So I'm going to be copying this URL.

  79. 15:57

    That's the Jupyter Notebook we are going to load on our Jupyter template that we deploy. And here you go to File, Open from URL. You paste that URL, click on Open, and voila.

  80. 16:13

    We have our, uh, notebook ready to start.

  81. 16:19

    So remember that we provision the Managed Inference and Agents service into our application. That is going to give us three environment variables. If I go here to Settings,

  82. 16:33

    Reveal config vars, we see the environment variables there. The INFERENCE_KEY, the MODEL_ID, and the INFERENCE_URL, the API key that we will use to call the service.

  83. 16:47

    So obviously the first step that we will do is to load those environment variables on our Jupyter environment. So we will do this workshop in four parts. The first one is set up the environment.

  84. 17:01

    We are going to just load those environment variables so we can continue doing our activities. Second, we are performing managed inference. This is your basic chat completions endpoint. And we will see two examples, one doing, like, basic chat completions, and the other one doing a streaming version of those chat completions endpoint.

  85. 17:24

    The third, which is one of my favorite ones, is the Heroku tools or the agents part. We have another endpoint that lets you run agents that we support natively.

  86. 17:36

    We have code execution agents, we have database access agents, document conversion, uh, execute commands on a dyno, and we are going to continue adding more to those. And this is just an endpoint that performs the execution of the agent in the Heroku infrastructure in our dynos, and it is just an one-off dyno.

  87. 17:58

    What is a one-off dyno? It just spins up, runs the code, and it scales back to zero. So you are not paying for an application that is constantly running.

  88. 18:07

    You are just paying for the amount of compute that that tool took to execute. And that endpoint also supports MCPs. MCP is that you can deploy to Heroku

  89. 18:19

    And then attach them to your, uh, inference agent endpoint. And last but not least, we are going to take a look at that MCP support that we have. How can you deploy an MCP to Heroku, attach that MCP to the agent's endpoint, and also use those MCPs externally, remotely through the MCP gateway or MCP toolkits that we

  90. 18:43

    have? I know it sounds like a lot, but it's going to be easy. I have a couple of exercises for you if you wanna write some code. But basically, what I will be doing here is just, like, running the code that I already have implemented here.

  91. 18:58

    My good friend, thank you for coming. Okay, so first thing I told you, we need to load these three environment variables, inference URL, inference key, and inference model ID.

  92. 19:08

    And there is a fourth one that we will use for certain, uh, tool execution on Heroku, which is the target application name. So what are going to put you here in target application name, the name of the application you deploy, the name of the application, the Hero- the, the Heroku Jupyter application that you deploy.

  93. 19:33

    In this case, my worsh- my Jupyter Notebook is called AI Engineer Workshop. This is the one that I'm going to use here. I am giving permission to my tool to run commands and perform compute operations on this dyno, on this application.

  94. 19:51

    So here you just replace to the name of your application so you can do those examples. And later, I will show you also how can you give access to the application that has the database to run the other examples that we have for you.

  95. 20:07

    So let's, uh, load the environment variables. Now the environment variables are ready, so I can continue with the examples.

  96. 20:18

    Then let's go with Managed Inference. I mentioned that for Managed Inference we have the basic chat completions endpoint. This is an endpoint that you will find on services like OpenAI or Cloud Anthropic.

  97. 20:31

    Basically, you ask a question, you send a message or an array of messages, and you are going to get a response. This endpoint also supports custom tool execution, but this tool execution is something that is happening on your code.

  98. 20:47

    So you'll need to specify the function and capture, like, the information to execute those. Or if you are using an agent framework like ADK or LangChain, it also has, like, custom tool execution internally.

  99. 21:01

    But for the example, we are going to perform a basic inference. To perform the basic inference, we are, we are going to do a basic HTTP request. So we have the inference key, the model ID on the payload, so we are going to use Cloud Four, which is the model that I provisioned here on my app.

  100. 21:19

    And the message that I'm going to send as an user is, "Explain the conte- the concept of Managed Inference in one sentence." You can adjust the parameters that the OpenAI API supports like temperature, maximum tokens, uh, top P, et cetera.

  101. 21:36

    You can take a look at the documentation by clicking on the, uh, path endpoint, and it will take you to the documentation of that endpoint and all of the parameters we support.

  102. 21:50

    And then after performing the HTTP request, we get the response as JSON, and let's run this.

  103. 21:58

    And let's see how this work. So we get a response like any other AI endpoint works. So we have a chat completions object with the different choices, like the amount of messages that it returns.

  104. 22:11

    Basically, we have just one message here as an assistant, and the response is Managed Inference is a cloud service that handles the deployment, scaling, and maintenance of machine learning models for real-time predictions, et cetera.

  105. 22:25

    So that's a basic example. So this is something that you can do everywhere, getting inference. It became a commodity. And also to show that we also support the streaming.

  106. 22:37

    Of course, we wanna have, like, a better experience, uh, while building our applications. We need to support streaming, meaning that it's not going to return the whole answer as just one JSON object.

  107. 22:49

    It's going to start giving you chunks so you can get that real-time feedback when getting the response from the inference service. So here I'm just consuming chunks and printing out those chunks on the s- uh, on the screen.

  108. 23:05

    The first ex-sample is going just to show you the raw version of this request. So this is using a server-sent event approach. It's sending you message events. Every message events has data, and each data is a piece of that completion object.

  109. 23:25

    But in this case, you are getting like a, like a delta, like a version of that message. So you see here in the content we get nothing, then streaming is crucial for this, and then it continues giving me the response.

  110. 23:42

    We get a bunch of different responses, but let's take a look how that looks like on an application when you are rendering the chunks. This is just the pure objects to understand how the API works.

  111. 23:54

    Here I have another example where I am doing some parsing of that information, extracting the delta content of each response, and then at the end rendering everything as a beautiful markdown.

  112. 24:07

    When I execute this, the inference service is going to start responding on real time, chunk by chunk, all the messages that I'm getting from the service. As you can see, it is a markdown response.

  113. 24:19

    So at the end when we finish, we use the markdown, uh, display object here in Jupyter to get the beautiful answer here. Sometimes it will give you like a, a code example depending on what the inference, uh, service is giving you, and that's how this works and how...

  114. 24:39

    This is how you can build an application to consume these real-time streams.

  115. 24:46

    But now let's talk about the good part, which is the agents. Who has built an agent before, like an application that executes a tool or run, run some code?

  116. 24:57

    Perfect. We have like couple hands there. Amazing. So the tools that we support today, and Anush here as product manager can tell us what we are going to be supporting in the future, are, uh, dyno run command.

  117. 25:10

    Dyno run command allows you to execute a command on a dyno, like a Unix command or a script that you are, you, you already have pre-deployed on your application.

  118. 25:21

    It will execute that and give you the response. So this is pretty good to run trusted code, code that you have written, that you know that it works, that it has a predictive, um, uh, predictive result, and you can just have that code running on your dyno and execute it through this, uh, this agent, this tool.

  119. 25:43

    We have query databases. We have two tools here. One is Postgres get schema, and the other one is Postgres run query. Postgres get schema, mean, the LLMs doesn't know the shape of your database.

  120. 25:56

    If you ask for data, it will hallucinate an SQL query, or if you have some sort of retrieval augmented generation, or if you are grounding the prompt with the shape of your database, it will generate something that is close to the shape of the data of your, uh, database.

  121. 26:17

    But with this tool, it will get exactly what you have on the, on the database, and then we can pass that schema to the next tool, which is run query.

  122. 26:26

    It will generate a query that will run on your database, get the result, and make inference over those, uh, data. Then we have a couple other tools for document transformation, HTML to markdown, PDF to markdown.

  123. 26:43

    You pass an URL, like a website or a URL of a PDF that is hosted somewhere, and it will give you the m-markdown response so the inference can work over that text content.

  124. 26:55

    It's going to perform that text extraction for you. And my favorites are the code execution one. We support for today Python, Node, Ruby, and Go. So the LLM will generate code that will run on Heroku on a one-off dyno, and it will return that response back to the inference service.

  125. 27:15

    And these also support dependency installation. So the input parameter has like a, a packages array parameter where the LLM automatically say, "Okay, my Python script is going to use, uh, Pandas or NumPy.

  126. 27:32

    Let's install it before I attempt to execute the tool." And all of these are just MCPs that we natively support. But you can also extend this agent endpoint by deploying your own, your own MCPs.

  127. 27:48

    So now let's take a look. The agent's endpoint is different to the chat completions. It works similarly with, uh, some minor differences. For example, all of the responses are a stream, so it doesn't support like the synchronous call where it waits to give you like the full response because code execution, access to database, and these tool execution

  128. 28:10

    take time. So we prefer to have it like a stream. And, um, depending on the tool you are using, you might need to give access to the tool to your database or to an application, for example, to execute a command.

  129. 28:27

    So let's try that first one. Let's ru- run a demo to run a command on an Heroku dyno.

  130. 28:36

    So here we are setting up the payload similar to chat completions. We have the model ID. We have the message. We are asking,

  131. 28:46

    "What is the current date and time on, on the server?" LLMs doesn't know anything about like the real time. They need to use tools to be able to have context about what's going on right now.

  132. 28:58

    I have asked the LLM about like information, and it gives me like information from twenty twenty-four because that's the date when they stop training that model. But with this, it knows exactly what time is 'cause it's running this command on the server.

  133. 29:12

    So the tool is an Heroku tool, so I'm specifying the type to run this on Heroku. The tool that I'm executing is the dyno run command, and then the parameters for this tool are, okay, the target app name.

  134. 29:28

    Remember I told you use the name of the application you deploy. And then the command to run is date and a description of pretty much what's going on with the, with the output of this, uh, command.

  135. 29:46

    And I'm calling it. I'm just calling the agent's, uh, Heroku endpoint. I'm passing that payload that has the tool. This is also a, a stream endpoint, so I am having the, the stream output here.

  136. 29:58

    And I'm just extracting the tool calls, and this is a standard also shape, like API shape that other, uh, API support for tool calling. But this is specifically running the tools on Heroku.

  137. 30:13

    So let's execute and see the example. And I'm hoping you can execute this tool in your, in your Jupyter Notebook. So it is running this on my AI Engineer Workshop application.

  138. 30:26

    It's going to execute the dyno run command. So now it is waiting until it runs the command on my app. We get the, the date and time, and then the inference response.

  139. 30:39

    Cool. I mean, it's a basic example but powerful. Instead of running data, imagine if you deploy your own screen that does your data extraction or connects to external services, you can just call it, call it from here from this endpoint as well.

  140. 30:56

    The other one, code execution. Uh, with code execution, I am asking the inference service to perform a operation, and I am passing that code execution tool. So now that it has that tool enabled, it will try to generate code in that language, run it on Heroku, and give you the response back.

  141. 31:18

    For this example, I will invite you to change Node to Python, Ruby, or Go to see different, uh, responses. I'm a Node developer, even though I have a bunch of Python here.

  142. 31:31

    Some of you might recognize this is LLM generated. So sorry, not sorry. That's what tools are for. And I'm going to be executing that, uh, CodeExec Node to perform this operation.

  143. 31:44

    What is the 30th Fibonacci number? This is a basic algorithm. I just wanted to do something, uh, easy. I will invite you to change this to perform a different operation and see what you can do, like break this thing.

  144. 31:58

    And then for the execution, I am just parsing the response so I get like beautiful, uh, markdown code highlighting and everything. So let's perform the operation.

  145. 32:11

    And, um, it will generate, uh, the algorithm.

  146. 32:18

    Let's see. Okay, I will execute CodeExec Node. This is the code that I'm passing as the input. This is the beautiful markdown syn- syntax highlight code that it will execute.

  147. 32:29

    That's JavaScript. Yes, that's JavaScript. And at the end, it's going to execute that code on Heroku, and then I am getting the response back. The Fibonacci number is two, three, three.

  148. 32:41

    But I want to do that in Go, so just change

  149. 32:45

    from Node to Go. Execute this, and it will do the same but now the code that we are going to see here is a, is a Go code, and it will do the same.

  150. 32:57

    Get this Go code on an Heroku dyno, compile it, run it, and get you the response back. That looks like a Go code to me. I am not a, not a Go developer, but that definitely looks like Go.

  151. 33:09

    And it execute the response, and it gave me the same thing, and an explanation, like the inference operation over that tool execution.

  152. 33:19

    But that's cool. So far we have just called one tool. The good thing about agents is that we can, uh, chain calls together, mix and match. We, we have like different agents acting together.

  153. 33:31

    So now let's complicate things a little bit more, and we are going to use two tools. One is the HTML to markdown. So go to a website, do something, and then get that result and use it on this other agent.

  154. 33:46

    For that, we are going to use two tools, HTML to markdown and code execution Python. So the prompt here is use the Python snippet from the Wikipedia page for Euclidean algorithm to calculate the common divisor of two hundred and, uh, two hundred and fifty-two and one hundred and five.

  155. 34:10

    So it doesn't know the algorithm. I am telling directly, "Go fetch the exact one that is on Wikipedia, then run this code on Heroku and give me the response."

  156. 34:22

    I'm enabling those two tools, HTML to markdown and code execution Python,

  157. 34:28

    and let's run this. And now we are going to see multiple tool calling. Getting the page in markdown. So now it recognize that the Euclidean algorithm page is this one.

  158. 34:39

    Now it is reading the whole content of that page in markdown, attempting to get the algorithm from there, then it's going to generate the code, then it's going to run the code, and at the end I'm going to get the response.

  159. 34:52

    This execution takes a little bit more time because it is performing multiple, uh, tool runs at the same time.

  160. 35:01

    Any questions so far? Is anybody doing it? If not, you have access to the workshop. You can do it, uh, at home with more time.

  161. 35:09

    Yeah.

  162. 35:09

    Modify the code, analyze it, and make sure, uh, the concept is understood. So it got the algorithm here,

  163. 35:19

    and it execute the thing on Heroku. Now I'm just waiting for the last inference step, and that execution is done. There you go. I found the Python implementation from Wikipedia.

  164. 35:33

    I calculate that number that you asked me. This is the Python snippet from Wikipedia, and then it implemented in this code, and this is how I did it. These are the explanation.

  165. 35:46

    Beautiful. It works. Now, before I get into Postgres execution, we need to have access to the database, and right now the application that you have doesn't have access to that database.

  166. 36:00

    If you wanna give it a try, you are going to go to

  167. 36:07

    the Heroku dashboard. Where is the Heroku dashboard? Here.

  168. 36:14

    To the AI Engineer World's Fair. Click on AI Engineer data.

  169. 36:26

    Go to Resources. And we have one thing here. The Postgres database tools only work on followers. Followers on Heroku are read-only. For security reasons, we don't wanna give an LLM tool write access to your database because, of course, uh, LLMs are not

  170. 36:51

    trusted for that. If you want to give write access to your production database to an agent, I will invite you to deploy an MCP that does that, like the PostgreSQL MCP, give the access under your own risk.

  171. 37:05

    The ones we maintain, we wanna make sure we don't, like, break production. So we have two different databases here. The master one or the main one is database. Don't touch that one.

  172. 37:18

    The other one that has attachments is the follower. So you are going to expand these, manage attachments, and add your application,

  173. 37:30

    like Julián Jupyter, and that's it. Now I gave my application access to the database. Second, you will need to get the name of that database.

  174. 37:45

    So go to your application. In this case it's, uh, Julián Jupyter.

  175. 37:54

    Go to Resources, and this database here, AI Engineer Data,

  176. 38:07

    in my application is called Database, so that's the name I'm going to use in code.

  177. 38:15

    Since I am working on AI, um, AI Engineer Workshop, the name is different, so I'm going to show you so you can rec- recognize the differences in names. This is just the name of the environment variable on Heroku that has the connection string.

  178. 38:33

    So here I have access to that database. On this application, my database is called Heroku PostgreSQL Aqua, so that's the name you are going to give and change it in your code.

  179. 38:47

    Heroku PostgreSQL Aqua. So now I have access to that specific database on my app. I cannot access databases from other applications. This is for security reasons, so this is why I have to do the attachment first to be able to give permissions to my database.

  180. 39:05

    If the database lives in your application, you don't need to do, to do this because your application already has access to it. That's the only thing that we need to do here to, to set up.

  181. 39:16

    Uh, then we are going to enable two tools, Postgres Get Schema and Postgres Run Query. These are going to run on my application, remember the target app name, and these are going to access that specific database.

  182. 39:34

    So I'm giving permission to go to those two, uh, places. So the database that we have here is a database for a solar energy company. It contains a table full of metrics with, uh, energy consumed and energy produced every hour in kilowatts per hour, and we have metrics for maybe two months.

  183. 39:58

    So now what type of applications can we build with this? Like asking questions like how much energy has been saved in the last thirty days?

  184. 40:09

    It's going to understand the shape of the database. It will see that it has a metrics table with these columns, then it will go and generate an SQL query to be able to get this information, and then run that query and give you the response.

  185. 40:27

    And let's execute that to see how it works.

  186. 40:34

    So first step is, "I don't understand your database. Let me get the schema."

  187. 40:41

    So to get the schema, it executes the tool, and then you are going to see the full schema being printed here on the screen in a moment. So now we have the schema of my database.

  188. 40:53

    It see the different tables, metrics with the fields, uh, products, systems, users, et cetera. So I have the information that I need here.

  189. 41:05

    Beautiful. Now it will run the, the query. So it already generated the query that it is a hundred percent sure will run on my database because it knows the shape.

  190. 41:18

    This is the response. It got the data. And now that after it got the data, it's going to perform the inference. It's going to extract the answer from that data, and at the end it's going to tell me the information.

  191. 41:31

    It wanted to run more queries to get a breakdown of the energy by system. I have three different systems, one system that performs well, the other one that doesn't perform pretty good, the other one that it's totally horrible.

  192. 41:46

    So it is also trying to get more information. Just from one, one question it is doing inference on my data, and I just enabled two tools. I didn't need to do anything else.

  193. 41:58

    And there we go, the report. These are the key metrics. This is how much you have saved, and this is the breakdown per system, the best performer, good performer, and the energy deficit.

  194. 42:11

    And this is ga- getting access to your data and acting over your data. We can add a third tool. Let's say Code Execution Python. I want to generate a graphic.

  195. 42:23

    So it will now do a Matplotlib, uh, graphic with that data, and you can keep adding more, more things. But also the more agents and tools that you add, it will take more time to perform the whole operation.

  196. 42:38

    But that's an example how can you mix and match these tools that we have here and, uh, how can you give access to agents to your data. I have a couple of exercises.

  197. 42:50

    Just try to come up with an example to use a PDF and extract something from that PDF and run code. And you can do this on your own time.

  198. 42:58

    We are running low on code. I want to show you now how can we

  199. 43:04

    deploy and run MCPs with the same endpoint. So those are the tools we maintain, but what about the tools you are building and the tools that already exist on the MCP ecosystem?

  200. 43:20

    So with Manage, Manage Inference and Agents, uh, let's say you go to the dashboard, to the configuration page, and here

  201. 43:31

    you get access to the, um, model configuration, and here it has the toolkit integration and the MCP server list. So basically, this MCP Brave that I have here, the Brave Search MCP, and you might have already have access to this, so you can attach it to your own Jupyter Notebook.

  202. 43:54

    Pretty much to attach an MCP, you click on Manage MCP Servers. You attach it as an application similar to what we did with the database. It's just another local application.

  203. 44:05

    And there you go. We have an MCP server here that is exposing to tools, web search and local search. So now we can use these tools on the agent's endpoint.

  204. 44:19

    So let me go and take a look at that. So these are the instructions, step-by-step, how can you enable an MCP. Then

  205. 44:27

    it's just another tool. This time it is not a Heroku tool, it is just an MCP, and this is the name of the tool that I will execute. MCP Brave

  206. 44:42

    is the name of the namespace. You can have multiple MCPs here, so this is kind of like a na- like a namespace for your MCPs. And I am going to execute Brave Web Search.

  207. 44:57

    This requires an API key, and I already have the API key on my application. If you want, you can go ahead and take it. I'm going to, like, remove that API key in a moment.

  208. 45:08

    Don't perform like, uh, two thousand queries, otherwise I, I'm going to get, uh, charged. But now I have access to it and the security to also enable those, those keys to your MCPs, and I will execute it as another tool.

  209. 45:24

    And the prompt that I'm sending is, what is the most recent news about AI agents? Let's execute this tool.

  210. 45:32

    And this MCP is running on Heroku the same thing as code execution or the Postgres, uh, tools. It will spin off a dyno, run the MCP as a standard input and output, and scale back to zero, so you are not paying for something that is constantly running.

  211. 45:51

    And there we go. That's the search from the Brave Web Search. So this is the response from the tool, and now it's going to, uh, render that after the inference, uh, operation.

  212. 46:07

    So I'm running MCPs on Heroku. A quick example how can you deploy an MCP.

  213. 46:14

    I have, uh, an exercise here. Deploy an MCP to Heroku. I have a Perplexity Ask MCP that I just forked. This is the official Perplexity MCP. I fork it to my repository to just do one thing.

  214. 46:32

    To make it Heroku-compatible, you use the Procfile, which is the file on Heroku that defines how an application is executed,

  215. 46:43

    and I added the entry point as an MCP. So this is a new process type we support. Everything that starts with MCP is going to be recognized as an MCP on Heroku.

  216. 46:54

    It will execute this standard input and out- and output code. It also requires an API key that I don't have,

  217. 47:02

    but you can deploy it. To deploy to Heroku, I added a button to make it super easy. Click to deploy. I deploy this, uh, MCP to my space, and then I attach it to my application.

  218. 47:15

    This is my Perplexity MCP. Let's add available to the space.

  219. 47:27

    I don't have one. I deploy. There are certain MCPs, if you don't have an API key, it fails when you run them, so this is why I'm specifying something.

  220. 47:38

    It doesn't need to be valid. And this is now a Node.js application. It will just deploy this, build this, and keep it available on Heroku, and you will be able to attach it to your, to your app.

  221. 47:49

    So this is an exercise you can do and write the code to run that, that MCP that you just deployed.

  222. 47:58

    Now we got the response from the previous, uh, Brave Search,

  223. 48:03

    and this is pretty much the most recent news about AI agents. But remember that I told you that you can also use those MCPs outside of Heroku, not only for Heroku agents.

  224. 48:16

    Let's say you are using Cursor or Cloud Desktop, or you are writing your own agent in a different platform, but you wanna have those MCPs available remotely, you can use those too.

  225. 48:30

    So here on the management dashboard, you see the Toolkit Integration page. So all of the MCPs that I deployed to my app are going to be available through this endpoint.

  226. 48:42

    So this is a server-sent event endpoint, and it is, um, authenticated behind a bearer token, and we are working on OAuth support.

  227. 48:52

    We're getting, we're getting-

  228. 48:54

    So that you can run it securely. You can build remote MCP servers on Heroku that are accessible without a bearer tokens. We don't like bearer tokens. [laughs] Perfect. So now that the MCP is there, let's add, like, my, my Perplexity really quick.

  229. 49:09

    Let's refresh. I need it. Perfect. I have it. It's available. I go to my toolkit, copy the token. It is the same, the same token that you use for inference, so I don't need to do this.

  230. 49:23

    You just need the URL. And in my Jupyter code, I have a basic MCP client. I'm using...

  231. 49:32

    So I need to install the dependency here. I'm using the MCP package from, uh, from Anthropic.

  232. 49:42

    And I'm creating an MCP client. That MCP client is going to my Heroku endpoint, passing that API key as a, a authorization header, and let's run it to list the tools that are available, and then execute Brave Web Search.

  233. 50:02

    So I'm executing the MCP that I deploy on Heroku outside of Heroku.

  234. 50:08

    Let's run this demo. And it connected. We have the following tools: Brave Local Search, Brave Web Search, and Perplexity Ask, the one that we just deployed and enabled with just two clicks mostly.

  235. 50:26

    And it executed the web search. I am not processing the result here. This is just a very basic example, but that's how can you deploy an MCP and use it externally.

  236. 50:37

    And I think, and I, I have also, like, a cursor, uh, MCP gateway that I use for MCPs that I deploy. I just use the same approach to use my MCPs and cursor.

  237. 50:50

    And last but not least, uh, I told you that the, at least the chat completion endpoint is compatible with OpenAI API, 95%. We are working to bring it to 99%.

  238. 51:03

    There are, like, certain parameters that are not supported, but you can just use the SDK. So let's use the SDK to perform a basic operation. I am just using the OpenAI SDK with the API key and URL from Heroku.

  239. 51:20

    I perform an inference, and we should get a response

  240. 51:30

    in a moment. And there you go. We have the response. And that's pretty much what we had for you today. You get access to this notebook. Keep playing with it.

  241. 51:45

    I extended access to, to the Heroku platform, that team, until the weekend, 'cause right now, unfortunately, we don't have a free tier. We are working hard to bring it back.

  242. 51:56

    Uh, but you can go deploy, try it out, and if you have any questions, please, um, uh, connect with us on social.

  243. 52:06

    Uh, we have, um, the dev center docum, uh, site for documentation, the Heroku AI website. Uh, we created a Heroku community on Twitter or X. These slides are on the Slack, so you can get them from the Slack.

  244. 52:23

    And thank you very much. I hope you enjoyed this workshop. [audience applauding] [upbeat music]