AI Engineer World's Fair 2024
Unlocking Developer Productivity across CPU and GPU with MAX
Read the talk
Beyond the Endpoint: A Shared CPU and GPU Stack with MAX
Chris Lattner explains how MAX combines familiar Python interfaces, an integrated inference stack, and Mojo to reduce the work of deploying and optimizing AI.
From a talk by Chris Lattner
Before you start: Familiarity with Python and basic model inference is helpful; no GPU kernel programming experience is required.
When an endpoint stops being enough
A hosted inference endpoint makes a generative AI prototype unusually easy to build. Choose a model, send a request, and put its response into an application. The harder question comes when that application needs something the endpoint cannot provide: control over where private data goes, integration with internal security, or fine-tuning that keeps critical company information inside the organization.
Customization adds another reason to own more of the system. A proprietary model can incorporate research that better fits a particular application. At scale, inference prices can also justify running the model yourself, while alternative hardware may offer capabilities or economics unavailable through the default provider. Moving beyond the endpoint is a decision about control, customization, cost, and deployment choice. It does not negate the endpoint’s usefulness; it changes how much infrastructure the product needs to own.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Owning inference exposes a fragmented stack
The tools for taking that step have changed. Training frameworks such as PyTorch, TensorFlow, and Caffe gave way to a larger inference ecosystem, including ONNX and TensorRT, followed by a proliferation of generative AI frameworks. Some specialize in a single model. That specialization can be convenient for one deployment, but supporting several models can mean switching technologies repeatedly.
Models then sit inside an already complicated production system. Lattner’s objection is that this fragmentation slows the movement of research into products: an optimization is less useful if adopting it requires another round of integration work. Meanwhile, AI engineers face new models and optimizations every week, pressure to add generative AI across products, and little time to evaluate new hardware. Once a prototype reaches production, per-unit inference costs become a further constraint on scale. The problem is not a shortage of effort from engineers; it is how much infrastructure work surrounds each improvement.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
One stack, adopted gradually
Modular approaches this problem with experience from building TensorFlow, PyTorch, LLVM, MLIR, and XLA. Those systems reflect earlier requirements; the proposal here is to apply what their builders learned to a stack organized around generative AI. The goals are ownership of models and data, control over the product, and freedom to choose where it runs.
The architectural choice is one integrated stack rather than a separate solution for every model. Adoption should nevertheless start with familiar Python and PyTorch workflows. Developers can go deeper as needed, without first becoming experts in the infrastructure beneath their applications.
MAX has two parts: a free AI framework and commercial managed services. This talk concentrates on the framework, with a talk-time focus on inference: deploying PyTorch models and supporting generative AI workloads whose cost and complexity make deployment difficult. The managed service is a separate offering, not a prerequisite for understanding the framework’s architecture.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Familiar interfaces over a rebuilt execution layer
Python is the primary entry point, with a C++ option also mentioned. Lattner describes several paths into MAX, depending on how much of the model implementation a developer wants to control.
| Entry path | Intended use |
|---|---|
| PyTorch ecosystem | Bring models through ONNX, TorchScript, or torch.compile |
| Native APIs | Work directly with KV caches, paged attention, and other inference mechanisms |
The first path preserves existing model workflows. The second exposes more of the machinery needed to develop new LLM and generative AI techniques. These are the integration paths presented in the recording, rather than a current API recipe.
Underneath those interfaces, MAX takes a more invasive approach. Rather than building on cuDNN and the corresponding Intel libraries, it replaces that layer with a consistent execution stack. Compiler technology, runtime support, and high-performance kernels arrive together. The intended simplification is below the API as well as at it: developers should not have to assemble or reconcile those layers themselves.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Quantization makes the CPU case concrete
Developer velocity is a major reason to change frameworks, but it is difficult to quantify. Performance gives the architecture a more concrete test. Lattner introduces a recent quantization release involving INT4 and reports roughly 5× faster performance than llama.cpp on cloud CPUs. The displayed comparison names Llama 3, specifies time to first token on AWS c6i.16xlarge, and shows bars labeled 6.7×, 4.8×, and 4.9×. Those conditions matter: this is a particular latency comparison, not a universal multiplier for every model or every stage of inference.
The product implication is that faster inference can change perceived responsiveness and cost. The implementation argument goes further: compiler and kernel work should make quantization a reusable capability, rather than a one-off optimization attached to one model. Llama 3 supplies the recognizable example; the intended benefit is an approach that generalizes beyond it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A device switch does not unify two implementations
GPU programming raises the ambition. The goal is to make it as approachable as programming a CPU in Python, while improving utilization of hardware already being paid for. Lattner speculates that audience workloads may achieve only 30–50% GPU utilization, framing that as roughly 2–3× excess GPU spending. This is a motivating estimate, not a measurement of those workloads. The desired stack must combine ease of use with CUDA-level power, rather than forcing developers to choose between them.
Rebuilding that capability is difficult: NVIDIA’s software represents years of engineering. A slide showing a one-line change from CPU to GPU does not explain how the difficulty was solved. Such a switch can conceal a fork between an Intel MKL implementation and a CUDA implementation. The interface looks consistent, but the underlying systems can behave differently.
MAX’s response is to replace the underlying matrix multiplication, fused attention, and graph machinery so the components fit together across execution targets. The meaningful claim is therefore not the brevity of device selection. It is that a shared implementation approach can make behavior more predictable beneath that selection.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Custom kernels and compiler fusion
That shared stack offers two routes to GPU performance:
- Custom kernels: Advanced developers can write kernels directly against MAX. Lattner presents this as CUDA-level capability with strong developer tools and a programming experience he considers easier than Triton.
- Automatic fusion: Developers who do not want to write kernels can rely on an auto-fusing compiler, reducing the need to create hand-fused implementations for ordinary cases.
The distinction is how much optimization work the developer takes on, while both routes remain part of the same system.
Ease of programming still needs a performance test. For matrix multiplication, MAX is compared with NVIDIA’s cuBLAS library and its more programmable C++ alternative, CUTLASS. The displayed microbenchmark uses an A100, BF16 inputs and outputs, and FP32 accumulation, with throughput plotted across matrix shapes. Lattner describes MAX as meeting or beating both libraries and cites a 30% win, while emphasizing that results depend on the data. This is a kernel-level comparison, not a claim of 30% faster end-to-end inference.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Mojo supplies the implementation foundation
Rebuilding the execution stack led Modular down another layer: it built Mojo to implement MAX. Using Mojo is optional for MAX users. Its intended role is to make extending Python possible without requiring a move into C, C++, or Rust, while retaining syntax and habits familiar to Python developers.
Lattner describes a downloadable toolchain, Visual Studio Code support, and Linux, Mac, and Windows availability. The historical scope needs two distinctions: the March 2024 open-source release covered the Mojo standard library, not the whole compiler; Windows use was through WSL, with native support still forthcoming in Modular’s 2026 account. He also cites 200,000 people and 20,000 in Discord, without defining the first population.
The design separates Python’s strengths from its implementation. Its developers, libraries, community, ecosystem, and familiar packaging practices are assets to preserve. Mojo combines that familiarity with MLIR and compiler technology intended for a different execution model. Python-like code consequently need not inherit the performance characteristics developers associate with interpreted Python.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What fast ordinary control flow changes
Lattner reports users finding 100–1,000× speedups over Python in unspecified cases, with even larger outliers. Without workloads or measurement protocols, those figures do not establish a general language speedup. The more useful consequence is what developers can write: when performance matters, Python programmers often avoid putting hot loops in Python itself.
The concrete example is tokenization code from Modular’s Llama 3 implementation. It uses a standard algorithm with linked-list traversal, conditionals, and loops. These are ordinary programming constructs, but they matter because token processing is not simply a large matrix operation that can always be handed off to a library. A small Mojo scan illustrates the same freedom to express work directly, here counting adjacent repeated token IDs:
mojo
fn count_adjacent_repeats(tokens: List[Int]) -> Int:
var count = 0
for i in range(1, len(tokens)):
if tokens[i] == tokens[i - 1]:
count += 1
return count
fn main():
var tokens = List[Int](17, 17, 42, 42, 42)
print(count_adjacent_repeats(tokens))
The scan has three matching adjacent pairs. The talk’s tokenizer is more involved—it follows linked structures and merges tokens—but the programming point is the same: loops and branches can remain in a language that feels familiar to Python developers.
This changes the boundary between application work and systems work. Developers who need deeper control can pursue it without immediately switching to C or C++. Lattner connects that capability to lower execution cost, researchers’ ability to modify the full stack, and teams with fewer boundaries between language specialists. He also points to a Modular blog comparison with Rust; that is a claim about particular implementations and workloads, not evidence that one language is universally faster.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From framework adoption to deployment
The adoption decision returns to the requirements that made an endpoint insufficient. Data control, internal security, customization, cost, and hardware portability determine whether owning inference is worth the additional responsibility. MAX is presented as a free, downloadable framework for that path. For production, Modular also offers deployment support on Kubernetes and SageMaker, although the recording does not walk through a deployment procedure.
GPU availability is a roadmap statement in this recording: Lattner forecasts an official September launch and invites developers to seek early access through Discord. That forecast should not be read as a completed release; September’s MAX 24.5 announcement still described a CPU-only release, and the GPU technology preview followed in December. The closing invitation is to explore Modular’s tools, the source and examples on GitHub, and its developer community—starting with the framework and going deeper into the implementation when the product requires it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Current home of Modular’s open-source components, MAX and Mojo examples, and getting-started links.
Further reading
Historical release details for quantization, Llama implementations and MAX on Apple silicon.
Related AWS Summit deck covering MAX architecture, CPU and GPU comparisons, Mojo and cloud deployment.
Explains Mojo’s compiler and SIMD design and the limits of language performance comparisons.
Updates since the talk
- MAX 24.6 GPU previewArticle
The December 2024 announcement of the MAX GPU technology preview and its initial hardware support.
Later announcements covering Mojo compiler open sourcing and planned native Windows support.
Read the complete timestamped transcript
- 0:00
[upbeat music] All right. Good morning, everyone. I'm here to talk to you about Modular and accelerating the pace of AI.
- 0:19
Uh, you know what gen AI is. I'm not gonna tell you all about this. Let me tell you one of the things I think is really cool about it, and very different than certain other technologies, is that it's super easy to deploy.
- 0:30
There's lots of great endpoints out there. There's a lot of good implementations, a lot of ways to make it super easy to build a prototype and get, get going very quickly.
- 0:39
But despite all the availability of all these different endpoints, sometimes you do have other needs. Sometimes you might wanna go, you know, and control your data instead of sending your data to somebody else.
- 0:51
Sometimes you might wanna integrate it into your own security 'cause you're-- got your critical company data in your model and you don't wanna fine-tune it somewhere else. Sometimes you wanna customize the model.
- 1:00
Like, there's research happening all the time, right? A lot of things in building proprietary m-models that work best for your use cases, uh, can make your applications even better.
- 1:09
And of course, the inference endpoints are expensive, and so sometimes you wanna save money. Sometimes there's hardware out there that's really interesting, and you wanna explore out from the, the mainstream, and you wanna go do this.
- 1:21
And if you care about any of these things, what you need to do is you need to go beyond the endpoint.
- 1:27
And so how do you do that? Well, if you have-- many of you have explored this, I'm sure. The answer has shifted. It used to be that we had things like PyTorch and TensorFlow and Caffe and things like this, but as inference became more important, the world shifted.
- 1:41
First we got ONNX, TensorRT, things like this. And today we have an explosion of these different frameworks, some of which are specific to one model. And that's cool if you care about that one model, but if you have many different things you wanna deploy and you wanna work with, it's very frustrating to have to switch between all,
- 1:57
all these different technologies. And of course, it's not just the model. You all know there's this gigantic array of different technologies that get used to build real-world things in production.
- 2:07
And of course, no-none of these are really actually designed for gen AI.
- 2:11
So my, my, my concern about this, my, my objection to the status quo is that this fragmentation slows down getting the, the research and the in-in-innovations coming into gen AI into your products.
- 2:22
And I think we've seen so many of these demos. Last year was really the year of the gen AI demo. But still, we're struggling to get gen AI into products in an economical and, and, uh, good way.
- 2:33
And so whose fault is it? Well, m- is it our fault? Like, many of you are AI engineers. If you don't, let's sympathize with the plight of the AI engineer because y'all, these folks that are building this have new models and optimizations coming out every week, right?
- 2:52
Every product needs to be enhanced with gen AI. This is not, like, one thing. We're getting dumped on, and there's so much to do, and we can't even keep up.
- 2:58
There's no time to deal with new hardware and all the other exciting new features. And of course, once you get something that actually works, the costs end up making it very difficult to scale these things because getting, getting things into production means suddenly you're paying on a per unit basis.
- 3:13
So it's not the AI engineer's fault. We should look at the concerns and look at the challenges faced here. And so I think that we need a new, new approach, right?
- 3:21
We've learned so much. Let's look at what we need to do. How do we solve and improve the world here? This is what Modular is about. And so I'll give you a quick intro of what we're doing and kind of our approach on this.
- 3:33
First of all, who are we? Modular is a fairly young company. We've been around for a couple of years. Um, we, we have brought together some of the world's experts that built all of these things.
- 3:43
And so we've built TensorFlow and PyTorch. We built compilers like LLVM and MLIR and XLA and all, all of these different things. And so what, what I can say about that is that we learned a lot, and I apologize [laughs] because we know why it is so frustrating to use all these things.
- 4:01
But, but really it's-- it was a, you know, the world looked very different five years ago. Gen AI didn't exist. It's, it's understandable. We, we tried really hard, but, but we have learned.
- 4:11
And so what our goal is, is to make it so you can own your AI, you can own your data, you can control your product, you can deploy where you want to.
- 4:20
You can do this and it-- make it much easier than the current systems work today. And so how? Well, what we're doing is really going back to the basics.
- 4:31
We're gonna-- we're bringing together the best in class technologies into one stack, not one solution per model.
- 4:38
Our goal is to lift Python developers, PyTorch users. This is where the entire industry is, and so we wanna work with existing people. We're not trying to like say, "Hey, ditch everything you know and try something new."
- 4:50
We wanna gradually teach and give folks new tools so they can be superpowers, so they can have superpowers. And finally, uh, so I spent a lot of time at Apple.
- 4:59
Like, I want things to just work. Like, you wanna build on top of infrastructure. You do not wanna have to be experts in the infrastructure. And this is the way all of this stuff should work, and unfortunately, it's just not the case today in AI.
- 5:10
And so at Modular, we're building this technology called MAX. I'll explain super fast what this is. Um, MAX is two things. One is an AI framework, which I'll spend a bunch of time about.
- 5:22
Uh, the AI framework is free, widely available. We'll talk about it today. The other is our managed services. This is how Modular makes money, very traditional. We're not gonna spend a lot of time talking about that today.
- 5:33
And so if you dive into this AI framework, well, it's... We see it as two things. It's the best way to deploy PyTorch. It's also the best way to do gen AI.
- 5:43
And both halves of this are really important. And MAX is currently very focused on inference. And so these are areas where, uh, PyTorch is challenging at times. This is where gen AI is driving us crazy with cost and complexity.
- 5:57
And so really focusing on this problem is something that we are, uh, we're all about.
- 6:02
The other thing, as I said before, is Python. So we natively speak Python. That is where the entire world is. We also have other options, including C++, which we'll talk about later.
- 6:12
So how do we approach this? Well, as I said, we work with PyTorch out of the box. You can bring your models, your model works. We can talk to the v- wide array of PyTorchy things like ONNX and, uh, TorchScript and TorchCompile and, like, all this stuff, and so you can pick your path and, and that's all
- 6:27
good. Uh, if you wanna go deeper, you can use native APIs. Native APIs are great if you w- if you speak the language of KV caches and page attention and things like this, and you care about pushing the state-of-the-art of LLM and other gen AI techniques.
- 6:40
That's very cool. And also, um, MAX is very different in that it r- really rebuilds a ton of the stack, which I don't have time to talk about. But, um, we do not build on top of cuDNN and the, the NVIDIA libraries and on top of the Intel libraries.
- 6:56
We replace all that with a single consistent stack, which is really different approach, and I'll talk about what that means later. And so what you get is you get a whole bunch of technology that you don't have to worry about.
- 7:07
And so again, as a next-generation technology, you get a lot of fancy compiler technologies, runtimes, high-performance kernels, like, all this stuff, uh, in the box, and you don't have to worry about it, which is really the point.
- 7:20
Now why would you use MAX? So it's, it's a AI framework. You have one, right? W- And so there are lots of different reasons why people might want to use an alternative thing.
- 7:30
Um, for example, developer velocity, your team being more productive. That's actually incredibly important, particularly if you're pushing state-of-the-art. But it's also very hard to quantify. And so I'll do the same thing that, you know, kind of people generally do, is go and talk about the quantifiable thing, which is performance.
- 7:46
And so I'll give you one example of this. Um, we just shipped a release that has our int four, int six K fancy quantization approach. Um, this is actually five X faster than Llama.cpp.
- 8:00
And so if you're using Llama.cpp today on, in cloud CPUs, this is actually a pretty big deal, and five X can have a pretty big impact on, um, you know, the actual perceived latency of your product and performance and cost characteristics.
- 8:15
And the way this is possible is, again, this combination of really crazy compiler and technology and other stuff underneath the covers. But the fact that you don't have to care about that is actually pretty nice.
- 8:27
It's also pretty nice that this isn't just one model. This is... [chuckles] You know, we have this make it easy to do int four technology, and then we demonstrate it with a model that people are very familiar with.
- 8:38
And so if you care about this kind of stuff, this is, this is actually pretty interesting, and it's a next-generation approach to a lot of the things that are very familiar, but it's also done in a generalizable way.
- 8:48
Now, CPUs are cool, and so, I mean, so far we've been talking about CPUs, but GPUs are also cool. And [chuckles] what I would say and what I've seen is that the, uh, uh, CPUs and AI are, are kind of well understood, but GPUs are where most of the pain is.
- 9:03
And so I'll talk just a little bit about our approach on this.
- 9:07
And so first, before I tell you what we're doing, let me tell you our dream. And this is, this is not a small ambition. This is kind of a crazy dream.
- 9:17
Imagine a world where you can program a GPU as easily as you can program a CPU in Python.
- 9:25
Okay. Not C++, in Python. That, that, that, that is, that, that is a very different thing than the world is today.
- 9:34
Imagine a world in which you can actually get better utilization from the GPUs you're already paying for. I don't know your workload, but you're probably somewhere between thirty percent, maybe fifty percent utilization, which means you're paying for, like, two to three times the amount of GPU that you should be, right?
- 9:49
And that, that is understandable given the technology today, but that's not great for lots of obvious reasons. Imagine a world where you have the full power of CUDA. So you don't have to say there's a powerful thing and there's an easy-to-use thing.
- 10:05
You can have one technology stack that scales.
- 10:08
Well, this is something that is really hard. [chuckles] This is something where, you know, NVIDIA has a lot of very good software people, and they've been working on this for fifteen years.
- 10:16
Um, but I don't know about you, I don't run fifteen-year software on my cell phone. Like, it doesn't [chuckles] run BlackBerry software either. And I think that it's time to really rethink this technology stack and push the world forward, and that's what we're trying to do.
- 10:29
And so how does it work? Well, you know, it's just like PyTorch. You use one line of code and switch out CPU to GPU.
- 10:36
Ha ha. We've all seen this, right? This, this doesn't say anything. I actually hate this kind of a, a demo, um, because the way this is usually implemented is by having a big fork at the top of two completely different technology stacks.
- 10:49
One built on top of Intel MKL, one built on top of CUDA. And so as a consequence, nothing actually works the same except for the, the thing on the slide.
- 10:58
And so what Modular's done here is we've gone down and said, "Let's replace that entire layer of technology. Let's replace the matrix multiplications. Let's replace the fused attention layers.
- 11:08
Let's replace the graph thingies. Let's replace all this kind of stuff and make it work super easily, super predictably, and let's make it all stitch together." And yeah, it looks fine on a slide, but the slide is missing the point.
- 11:21
So if you are an advanced developer, and so many of you don't wanna know about this, and that's cool. If you are an advanced developer, like I said, you get the full power of CUDA.
- 11:30
And so if you want, you can go write custom kernels directly against MAX, and that's, that's great. And for advanced developers, which I'm not gonna dive too deeply into, it's way easier to use than things like the, uh, Triton language and things like this, and it has good developer tools and it has all the things that you'd
- 11:48
expect from a world-class implementation of GPU programming technology. Um, for people who don't wanna write kernels, you also get a very fancy auto-fusing compiler and things like this. And so you get good performance for the normal cases without having to write the hand fuse kernels, which is, again, a major usability improvement Now, you know, it's cool, like,
- 12:08
there's a lot of, lot of things out there that promise to be easy, but what about performance, right? A lot of the reason to use a GPU in the first place is about performance.
- 12:17
And so one of the things I think is pretty cool, and one of the things that's very important to Modular, is that we're not comparing against those standards. We're comparing against the vendor's best.
- 12:26
In this case, NVIDIA, they're experts in their architecture. And so if you go look at, again, there's a million ways to measure things, a micro benchmark. Go look at the core operation within a neural network, matrix multiplication.
- 12:41
This is the most important thing for a wide variety of workloads, and again, one set of data. But we compare against CuBLAS, the hard-coded thing, and then also against Cutlass, the more programmable C++y thing.
- 12:55
And so, MAX is, uh, meeting and beating both of these, you know, by just a little bit. I mean, it's... You know, it depends on your bar, and data's complicated.
- 13:04
But you know, if you're winning by 30%, 30% is actually a pretty big deal given the amount of cost, the amount of complexity, the amount of effort that goes into these kinds of things.
- 13:14
And so I've talked a lot about the what, but I haven't talked about the how. And so the how is actually a very important part of this, and I'll just give you a sample on this.
- 13:23
So we are crazy enough that we [laughs] decided to go rebuild the world's first AI stack from the bottom up for gen AI. And as part of doing that, what we realized is we had to go even deeper, and so we built a programming language.
- 13:38
We have a new programming language that's called Mojo. And so the thing about Mojo is, if you don't wanna know about Mojo, you don't have to use Mojo. [laughs] You can just use MAX, it's fine.
- 13:48
But we had to build Mojo in order to build MAX. And I'll tell you just a couple of things about this. Our goal is that Mojo's the best way to extend Python,
- 13:58
and that means that you can get out of C, C++ and Rust. And so what is it as a programming language? It's a full-- It's Pythonic, so it looks like Python.
- 14:06
It feels like Python. Everything you know about Python comes over, and you can not have to retrain everything, which is a really big deal. You get a full tool chain.
- 14:14
You can download it on your computer. You can use Visual Studio Code. It's open source, available on Linux, Mac, Windows. Two hundred thousand people, twenty thousand people in Discord.
- 14:23
It's, it's really cool. Would love for you to go check it out if you're interested in this. But what is Mojo? [laughs]
- 14:31
Like, what, what, what actually is it? Fine, there, there's a programming language thing going on. Well, what we decided is we decided that AI needs two things. It needs everything that's amazing about Python.
- 14:42
This is, in my opinion, the developers. This is the ecosystem. This is the libraries. This is the community. This is even, sorry, the package managing and, like, all the things that people are used to using already.
- 14:56
Those are the things that are great about Python. But what is not great about Python, unfortunately, is its implementation. And so what we've done is we've combined the things that are great about Python with some very fancy, highfalutin compilery stuff, MLIR, all, all this good stuff, that then allows us to build something really special.
- 15:15
And so while it looks like Python, please do forget everything you know about Python, because this is, this is a different beast.
- 15:23
And I'm not gonna give you a full hour-long presentation on Mojo, but I'll give you one example of why it's a different beast, and I'll pull it back to something many of you care about, which is performance.
- 15:32
And all what I'll say is that Mojo's fast.
- 15:35
How fast? Well, it depends. Like, this isn't a s- slightly faster Python. This is a working back from the speed of light of hardware kind of system. And so many people out there have found that it's a thou- a hundred times to a thousand times faster.
- 15:47
In crazy cases, it can be even, even better than that. But the speed is not the point. The point is what it means. And so in Python, for example, you should never write a for loop. [laughs]
- 16:00
Python is not designed for writing for loops if you care about performance, at least. In Mojo, you can go write, uh, code that does arbitrary things. This is an example pulled from our Llama Three, written in Mojo, that does tokenization using a standard algorithm.
- 16:15
It's chasing linked lists, has if statements, for loops. Like, it's just normal code, and it's Python. I mean, it feels like Python, and that, that is really the point.
- 16:23
And so for you, the benefit of Mojo is, first of all, you can ignore it if you don't wanna care about it. But if you do, you don't have to learn C, C++.
- 16:32
You have lower cost by default versus Python because performance is cost. It means that as a researcher, if you use this, you can actually have full stack hackability. And if you're a manager, it means that you don't have to have people that know Rust on your team, and C++, and things like this.
- 16:48
You can have a much more coherent engineering structure where you're able to scale into the problem no matter where it is. And so if you wanna see something super polarizing, go check the Modular blog, and we'll explain how it's actually faster than Rust, which many people consider to be the gold standard, even though it's, again, a fifteen-year-old
- 17:03
language. So I have to wrap things up. They'll get mad at me if I go over. Um, the, the, the thing that I, I'm here to say is that many of you may want to go beyond the AP- the API, and they're fantastic.
- 17:15
There's amazing technology out there. I'm very excited about them too. But if you care about control over your data, you want to integrate into your, your security, you want customization, you wanna save money, you want portability across hardware, then you need to get onto something else.
- 17:28
And so if you're interested in these things, then MAX can be very interesting to you. MAX is free. You can download it today. It's, it's totally available. Go nuts.
- 17:37
Uh, we didn't talk about production or deployment or things like this, but if you wanna do that, we can also help. We support production deployment on Kubernetes, SageMaker, and we can make it super easy for you.
- 17:47
Our GPU support, like I said, is actually really hard. We're working really hard on this. We wanna do this right, and so it'll launch officially in September. If you join our Discord, you can get early access, and we'd be very happy to work with you ahead of that too.
- 18:01
Uh, we're cranking out new stuff all the time. And so if you are interested in learning more, you can check out modular.com, find us on GitHub, a lot of this is open source, and join our Discord.
- 18:12
Thank you, everyone. [upbeat music]