Copilots Everywhere

[upbeat music] I'm delighted and honored to welcome a very special guest, Thomas Dohmke.

Thomas has been fascinated by software development since his childhood in Germany, and he's built a career building tools dev love and accelerating innovations at, that are changing software development.

Currently, Thomas is CEO at GitHub, where he has overseen the launch of the first at scale AI developer tool, GitHub Copilot. So please join me in welcoming to the stage, Thomas Dohmke. [upbeat music] [applause]

Thank you, Thomas.

Yeah. Ooh, it's bright out here.

Yeah. [laughs] [laughs] Well, thank you, everyone. Thank you, Thomas. Um, let's start with Copilot. Many people have shared their own takes on the Copilot origin story, so but what was your personal experience seeing it in GitHub?

I don't know, you had a sneak preview. Take us back to the start in 2020.

So imagine it's 2020, um, it's lockdown here in San Francisco and Seattle, everywhere where GitHub engineers are sitting. So like all of you probably were on a Zoom call, um, one of us had early access to a new model that, um, OpenAI had just, um, released in preview, um, a version of GPT-3 called Codex and, uh, you

know, one had the, um, Uge, I think, had the keyboard, um, the leader of GitHub Next at the time, and, uh, we were dictating prompts and asked the model to, uh, write some code.

And I think the first aha moment that I had is that you could ask it to write JavaScript code and it put the curly braces in the right places and whatnot, and you could write it, ask it to write Python code.

And the model in a way, you know, do- it doesn't work like a compiler. It doesn't have a syntax tree. It doesn't know these things. Or you could also argue it knows them exactly like we'd know it.

So that was probably the first moment, um... We kept building, uh, uh, we kept exploring the model and then decided we build this auto-completion Copilot that, you know, was the first Copilot and we build it all, you know, while being remote, while being on lockdown, so-

Wow

... if, like, event- if your investors tell you today you need to be in a, in a room and in front of a whiteboard, um, you can innovate if you want to while being, via being, uh, in your home offices around the world.

Um, I think the next moment was, um, that we shipped, um, uh, a preview to our in- internal engineers, and we call it a staff ship at GitHub. And, uh, the NPS, um, survey with those engineers was through the roof, I think seventy-two, seventy-three, something like that.

And-

Wow

... typical our early stage products especially, you know, with the large language model and, you know, all the hallucinations and the UI wasn't really figured out yet, uh, is, is much lower.

So that was kinda like a holy shit moment, uh, that we had. And, um, as the product then shipped in mid 2021 and, you know, COVID was still going, uh, uh, we, we started looking at telemetry and the team came and says it writes about 25% of the code in, in those files where it was enabled.

And I remember saying, "I don't believe this. Like, you know, telemetry is wrong. Please go back and, and validate that." And turned out, you know, that it was actually right and, uh, by now it's about half, you know, the code that's written.

Some languages like Java even has a, have a higher acceptance rate and more lines written. And so I think those kind of this journey that we went through over the first two years really was like one, one moment after another where we saw, uh, the future of AI long before, um, ChatGPT actually opened everybody else's mind.

Amazing, and now it's available to everyone here as well.

Yeah.

So I think Copilot started as an auto-completing IDE, and now it's all over GitHub. I know I have PR bots, et cetera.

Yeah.

What do you do to make Copilot, um, and integrate it across all of GitHub? Like, what are some experiments? What worked? What didn't work?

I think the first thing is to think about, you know, what do I do as a leader, as the CEO of a company, and it's really about constantly reconfiguring our approach.

Um, so much of, you know, the AI world is changing almost daily, um, there's, you know, some news, uh, on the information elsewhere, uh, every morning. And so there is no more a, I have a long-term strategy, uh, I have my features all laid out and let's work through the backlog.

It's really like operating as agile as possible, even in, as we are, you know, 3,000 person company as part of, you know, one of the largest company, uh, on the planet.

The second is that we really try to meet, you know, the developer where they are. Um, we say, you know, we're not trying to build an AI engineer, we are trying to build AI for engineers, a human-centric approach.

You know, that's where the name Co- what the name Copilot ultimately, uh, visualizes. Um, but also, you know, we're trying to make the developers' lives better, um, and because we are developers ourselves and, um, every productivity improvement we can find ultimately helps us at GitHub to build, you know, our AI product.

So that really is the approach, like looking at what, what can we do next, uh, to make, you know, our, our work, um, a little bit easier of building more features for Copilot.

You mentioned a great point. You're trying to meet the developer where they are. So s- for now, we've been bringing the AI to the IDE.

Yeah.

Can you-- Are we gonna try to bring the developer, the IDE, closer to the AI? How are you thinking about that?

You know, the idea of bringing AI into the IDE or really into ghost text, you know, auto-completions, was a way of getting around hallucinations. Um, it was a way of saying, okay, the model's not always going to be perfect, but neither are auto-completions, right?

Like, whether you have auto-completions in your Google Docs or in your email or in your editor in, in the old IntelliSense way, as you're typing, it cannot know what you wanted to type, and so you're used to adjusting your typing, and then you find this moment when you press the Tab key.

And, um, even with auto auto-completions, if you think about what developers do in the editor, well, they write code, and the best developers write a lot of code before they get stuck, and the newbies and, and those that rarely write code like I, you know, get stuck more often.

And then you, you know, Control + Tab or Command + Tab into, um, into your browser and you open, um, Google or-

StackOverflow

... StackOverflow, GitHub. [laughs] Right? And what you do there is you find code and, and you argue with other developers, and then you copy and paste that code into your editor and, you know, modify that as well.

So it's kinda like, in a way, ha- StackOverflow has as many hallucinations as, as, as, um, the model it might have. Uh, and not because the answers are bad, but because the world is changing so, so much.

You know, I code a little bit on iPhone projects in Swift, and there's always a new Swift version, uh, after WWDC or a new Xcode version, and so things have changed of how you use the APIs.

And so it keeps the developer in the flow. That... Really the crucial thing here was we didn't, you know, in a world, you know, 10 years ago, we probably wouldn't even call this AI.

We would just call it, you know, mo- smarter auto-completion. And, um, the AI key piece is not the core piece. The c- the core, the core feature of Copilot is it helps developers to stay in the flow to get the job done and not be in this constant distraction between the editor and, and the browser.

That's a great point, and I think a few months ago you wrote this post about Workspace.

What was the journey to creating Workspace? And maybe for folks who are not unfamiliar with it, what is Workspace?

Yeah. So, you know, you already mentioned auto-completion. That's what- how we started. Um, in, um, November 2022, ChatGPT happened, so early 2023 we added chat and, and GPT-4, uh, to Copilot in the IDE as a, as a separate, um, um, um, sidebar window.

So we have that available, and it has RAG and, and all the information, the context available in the IDE. But ever since we have been thinking, how can we make the developer flow even easier?

And Workspace does exactly that. It takes a GitHub issue or just a task, an idea that you write down on github.com, and it helps you then, as part of your code base and repositories, to figure out how to implement that change.

It bridges from the issue, you know, from the task description into the pull request, into the code. And the, the magic behind this is that, A, the human is still in the center, so e- every step of that way, you know, writing a specification, analyzing the current repo, the current behavior, and then using your description to figure

out how do you modify this. Uh, then writing the plan, um, which cha- shows you how to change all the files to the implementation, which is the diff view, if you will.

The human can interact, can change those bullet points, can change the code. And, um, what that really does is it ha- gives you a, an, a pair programmer that helps you to explore the code base.

Mm.

Right? Because the challenge we all have as engineers is that as l- a- as soon as you get moved onto a new project or you wanna, you know, modify an open source project, um, or you're just, you know, coming back from vacation, you're trying to remember what in, what is implemented where in your 1,000-plus files, that is,

navigating the code base is the first challenge you have, figuring out what's the current behavior and what's the new behavior. So you're having an AI native, um, a Copilot native developer environment that helps you along that journey that you're naturally also doing in your IDE, and that really is the key here.

It's not about, you know, building an autonomous agent. I'm sure you have heard a lot about that in the last three days. It's about building agents that helps us as humans to solve a task and learn along the way as we figure out, oh, you know, there's this test file that I also have to modify if I

want to implement this feature.

I love the point that you mentioned, which is not building autonomous agents and also helping the developers. So how should non-developers use Workspace?

They can, and in fact, you know, once we announced this, um, last year at GitHub Universe in November, I think the first email we got with feedback was from a program manager or product manager saying, "This is awesome because now I can, uh, not only write, you know, a user story or, um, a work item, I can

also see what that would mean to implement in the code base." In many ways, you know, the biggest challenge we have today is can we be as specific as possible when we write down a task, you know, as product managers or as engineers ourselves?

You know, often everything is obvious until it is not. Um, and then, um, you know, uh, uh, you, you kind of need to size the task, right? Like, how long will it take?

And, uh, the Mythical Man-Month, um, I think the pragmatic engineer had that a couple of weeks ago, is still true. Most, uh, most estimates are half as, uh, half the time that the, the job actually takes.

And so we're really bad at estimating how much time it takes to get, uh, uh, work done, whether it's in coding or whether it's building houses or, or roads or infrastructure.

And so, um, Workspace helps you with that as it helps you to figure out what I just described. Is it actually specific enough to write the code for that or to even figure out what the plan would look like?

Can you share a bit about your vision on how you think we will build and code in natural language and how it will help us collaborate better, devs and PMs, coders and non-coders, across languages and across the world?

For me, you know, the very first thing that you say natural language is, and I have it on my T-shirt here, Copilot speaks your language, is because chat, these large language models that we're using today in GitHub Copilot and many other AI applications are the same models that are also helping us in chat agents.

They speak almost any language, or any major human language. And so whether you, um, you know, wanna explore coding in English and you don't understand the concepts of, uh, you know, true/false Boolean logic yet, or whether you want to learn that in German, in Hindi, in, you know, Brazilian Portuguese, in, in Spanish and Chinese, you can do

that now. And, um- If I, you know, look at, uh, kids today in, in school, most of them are growing up with mobile phones. Um, you know, when you go into a restaurant here in San Francisco and Seattle or elsewhere in the world at night, you probably see a family with little kids where the kids have their

phone because the parents wanna enjoy five minutes on their own. And then as then kids grow up, you know, they see Super Mario or, or Minecraft, and they get into gaming, and that naturally that means, "How can I create my own game?

How can I create my own web page?" Copilot enables that, and it enables that in the language that the kids grow up with, which, you know, for the majority of the humans of the, on the, o- on this planet is not English.

Um, so that's number one. It, it democratizes access to technology. It also democratizes access for those that don't have parents at home that have a technical background or that don't have parents at home that have infinite patience, which most parents do not.

I have two kids. I- [laughs] ... speak from my own experience. At some point you're just done, you know, with explaining the world to your kids, and you just wanna, you know, switch on the TV and, uh, and watch your, watch your Netflix show.

And, and, and but that keeps going. If you look into the professional context, one of the biggest challenge we have is, you know, if you would join my company or I join your company, uh, tomorrow, the biggest challenge we have is wh- what's all the institutional knowledge, how are things being done, you know?

And what we don't like as humans is ask a thousand questions.

Yeah.

Um, especially if you're a new employee in a, in a big company, you're, like, having this anxiety in your head that everybody else thinks you're, you're dumb. You-- "Why did you get hired in the first place?"

So a Copilot also democratizes access to all the information in companies, and I think that is going to be changing how we work, and not only for developers in the workforce, but for really every human.

Thank you. Thank you, Thomas, for sharing your vision. I guess the next thing I wanna ask is maybe a little bit more unhinged. [laughs]

Speaking of agents, in your opinion, what makes an agent or a Copilot? What's the defini- what, what's your definition of an agent?

I think an agent, you know, is like an AI dishwasher. Um, you fill it, um, with, you know, the dishes, and you let it, uh, let it do its thing, and then at the end you st- you take the output and you put it back into the shelves, right?

And today, um, we have, you know, we called it, used to call it bots, um, um, you know, or CI/CD. In, in many ways that's an autonomous agent, right?

You push a pull request, and you run your CI/CD GitHub actions or, or, or a similar product. Um, many compute primitives that we have today are agents as they get a job done on their own.

Um, my monitoring, you know, to figure out is GitHub up or down is somewhat autonomous. Um, hopefully it pages somebody without us hearing from you that you cannot access your repository.

So, you know, I think in many ways, um, uh, what we are building is still tools that help us to get the job done, and there's many jobs that developers have to get done, many jobs that now AI engineers need to get done.

You saw it on the slide earlier, all the things that are also still true, you know, even though you can automate things with large language models. Um, and a lot of work in software engineering is, is, is bogging us down.

Um, a lot of boilerplate, a lot of security compliance. You know that Friday evening, uh, when you, when you wanna, you know, enjoy the barbecue because the sun is out, and instead you have to update all your Log4j dependencies, right? [laughs]

Like security tooling, in fact, you know, is creating more work. It's not a dishwasher. It's actually a tool that shows you that the, tells you that the dishes are, are dirty, and then you have to do the dishes yourself today.

And so, uh, that's security tooling, right? It just adds stuff to our backlog while we actually wanna work on the creative side, and we wanna, we wanna build new features.

We wanna build innovative product that, uh, creative things. Um, think many software developers do not understand themselves as a production worker. They understand themselves as artists, as creators.

Yes.

And, uh, but, you know, our companies, our governments, you know, the world is requiring us to do a lot of other work, and we need AI tools, auto-fix, you know, things that, that scans, uh, not only for security issues, but then fixes those security issues.

We need those pieces, um, supported by AI so we have more time for the things we don't wa- we do wanna do, and AI takes over the things we don't wanna do.

And that's, that's where the agents will go.

Fantastic. What's an agent you wanna have, and how far are we from it?

I mean, I want to have these agents that burns down all my security backlog. Um, it's, um, as in any company, the, the challenge is that I have way too many of these items.

Um, and there isn't really a book you can buy, um, that tells you as an engineering manager of how to balance those two things. Um, you cannot do all the, your work into security compliance, um, uh, accessibility and whatnot, and you cannot put all your work in- into innovation because your customers will lose all your trust the

moment you have a security issue that, uh, threatens their, their data. And as such, you have to balance those two things, or you find AI agents that brings the work down.

And I think as any, you know, leader of a software development company, uh, I always wanna go faster. I always wanna get that feature done faster. And, um, I'm sure it's, you know, f- the same, uh-

The same

... for you folks at Amazon. When, uh, when I have an idea and I ask my folks, "How long will it take to implement that?" the estimate I'm getting is like I'm scratching my head and I'm thinking, "I could done that, I c- can do that myself faster than, than waiting for, for my team to do it."

But of course that, that's not the truth. The truth is that there are so many other things in the process these days that, um, we need to find new abstraction layers, um, that help us to, to, uh, get control over our development life cycle again.

That's a great point. So last question: Do you have any advice for devs, both new and experienced, on how they should, they should navigate this new world of tools, this new of, new world of abstractions, um, in what some say is the biggest technology innovation since the internet?

I think, you know, the most exciting thing about this new technology is, and you saw it, uh, hopefully over the last three days at, at this conference, is that we are moving into a, a new world of software development.

And there have been multiple step functions, you know, over my, uh, life. Um, I was born right before the PC was invented. Uh, I remember getting my Commodore 64 and a PC in the '90s.

I remember the open source and the internet and, you know, i- internet open source before the internet was buying, uh, CDs and DVDs, um, in bookstores. The internet came, you know, um, a st- a SourceForge, and then GitHub came.

All of a sudden, developers started collaborating. The mobile wave came, and every time we had those step functions, software development got more exciting. And I think, you know, we are again at that, at that step function.

It means we can embrace our nerditude. We can build new shit, new shit.

Yes.

And I think, you know, the, it's really like, like, you know, for me as the CEO of GitHub, I don't get to touch code often. And so when I get to touch code on a Sunday afternoon, I don't wanna spend all my time off updating all my dep-

Okay, that's all we have. Thank you, Thomas. Please join me.

Thank you so much. [upbeat music] ... that we want to bring the fun back. And I think that's really, AI brings the fun back into software development. And so I wanna, you know, encourage all you back home to build interesting stuff and have a great time developing AI.