Insights on Building AI teams

[upbeat music] Hi, everyone.

I'm Heath Black. I'm the managing director of product at SignalFire. And let's actually take a quick step back and ask, uh, why am I here? Uh, so before I got involved in tech, I actually went and got a master's in Irish literature, of all things.

Uh, both of my sons are named after Irish writers. And, uh, you're gonna see how this actually weaves into the presentation a little bit later. When you get a degree like Irish literature, you have to get creative in how you actually use it.

In 2009, I got involved in, uh, some startups, and I helped ship the first ever conversational chatbot at a company called Chirpify. Uh, I then went and worked at a company called Imzy, where we were trying to build a Reddit competitor, but a lot nicer.

And then I actually went and joined Reddit, where I worked on experimental business lines and trust and safety tools. And I followed that experience up by going to Meta, where I shipped Meta's first assistant.

It was called M. It lived within Messenger. And then I was the, uh, first product manager on the AI assistant for their Ray-Ban glasses. But now I serve as the managing director of product at SignalFire.

So what does SignalFire do? I like to say that SignalFire is actually the first VC built like a tech company. And what I mean by that is, the same way that you all go out and interview customers to figure out whether you're building the right thing for them before you ship your product, SignalFire interviewed five hundred founders

to understand the things that keep them up at night and make them bang their head against the wall all day.

We then built AI and ML tools and our portfolio success teams entirely around those problems: going to market, recruiting, building your leadership skills, and the ability to launch your product.

But today, we're actually here to talk about, uh, some things that we've learned from our proprietary AI ML platform, Beacon. Beacon tracks over six hundred and fifty million employees, twen- or eighty million companies, and two hundred million open source projects.

And with all of that information, we build a variety of proprietary ranking systems and market insights that we can then use to power our firm so that we can move at startup speed, but then also to support the companies that we invest in.

Today's focus, we're gonna be using some of the data from Beacon to figure out how to filter the right people,

to find them in the right locations, to nail the right timing, and then finally, to close them with the right narrative. So let's first start with filters.

When I think about recruiting, I think about it in terms of, like, what filters would I apply to the people that are on my team and that I want on my team?

Beacon gives people the tools to apply these filters as they search. But the reality is, if you don't know what filters you need to apply, you're not going to find the right people.

So here are some interesting trends that we've seen that change how we filter.

Over the past, you know, decade or so, we've seen a stark de-credentialization in AI. AI startups are hiring more engineers without PhDs or prestigious schooling than ever before. In 2015, twenty-seven percent of engineer hires were from top schools, and sixteen percent had PhDs.

In 2023, those numbers were fifteen percent and seven percent. This is about a fifty percent decline for both of these numbers over that period of time.

And you're probably saying, "That doesn't sound right. What about for things like research scientists? They've got to have PhDs, right?" And, you know, you're not entirely wrong. About forty percent of research scientists have advanced degrees.

Now, this is not PhDs. This is simply advanced degrees. And even still, it makes up less than half of the people that serve in research scientist roles today.

From my standpoint, this isn't too surprising because there's been a slight shift in the market since 2015. In 2015, we were really focused on the, the kind of ML research side of things, the foundational side of things, whereas today, a lot of the work is about applying that to the real-world usage of that model.

It's MLOps, pro- like, product, uh, like, software experience. It's understanding how users interact with the thing that you're building.

And with the shift from credentials, we've also seen this really interesting people mobility over this period of time. If you look on the left side here, historically, a lot of the AI talent was centered on these companies, the Googles, the Ubers, the f- the Metas, the Apples.

And over this period of time, they've shifted to nine companies that we call the AI V League. The companies on the right here have seen a massive concentration of talent over this period of time.

It's really interesting because we generated this last year. Shortly after that, Inflection was ac- uh, acquired. So one of the key things that I want us to take away from here is that the market is constantly moving, so we have to constantly be assessing

Where that market is shifting. And the interesting thing as well is that all the companies on the left side of the screen are now fighting viciously to get people from the right side of the screen rather than the other way around.

But one of the key things is not just knowing where people are going. It's about where they're coming from. This graph shows you net employee movement between different AI V League companies, so to speak.

As you can see at the top, OpenAI has a positive flow of people from DeepMind, whereas Cohere actually has an, a negative trend. Knowing where people come from and where they're going is essential in ensuring that you are filtering for the right people as you look for peop- like, you know, building out your teams.

So the takeaway here is that work experience has always been important, but it now far surpasses education in terms of the main aspect that you should be looking at here.

Don't just rely on the credentials that someone has. You should instead look at the body of work that they've compiled. For new workers, you can still look at their body of work.

What are their open source contributions? What have they built outside of class? The reality is experience and what you're building matters more than where you get a degree. Secondly, you should be asking yourself, "Do I need a PhD researcher for the role that I'm hiring?

Or will a really awesome engineer with experience suffice?" And then the third is that you should actually consider removing academic requirements from your job postings, or maybe making them soft, so to speak, because this will ensure that your top of funnel is getting the people that have the experience you need more so than the education.

Now let's talk about the next aspect, which is location. I'm sure many of you have seen the debates on Twitter that San Francisco is clearly dead. And so we wanted to know, is it?

Interestingly, the answer is no, it's not dead. San Francisco makes up about twenty-nine percent of all startup engineers. Now, this is slightly down from highs of 2013 when it was at thirty-three percent, but it's upticking again since 2021.

New York and Seattle have also been pretty impressive as they've both doubled the market share of engineers that they have over that period of time. If we were to zoom out and look at big tech, fifty percent of big tech engineers still reside in the San Francisco Bay Area.

But what about AI specifically? Well, San Francisco is still leading the pack.

Twenty or thirty-five percent of all engineers in AI reside within San Francisco. Seattle makes up about twenty-two percent. New York makes up about ten percent. So San Francisco makes up more than both of those cities combined.

But if you actually look at this slide and compare it with the data that I showed in the previous slide, you'll see that

these, these markets are all punching well above their weight in terms of AI hiring and AI talent. Where they had a smaller number in the previous slide, they have a much larger number in this one.

So c- the talent is concentrating in these three key, key markets today. Now, this isn't terribly surprising for me because San Francisco makes up nearly thirty-eight percent of all early-stage funding into AI startups.

And the interesting thing about this is that San Francisco only has twenty-six percent of all early-stage funding in the United States total. So not only is San Francisco punching above its weight in terms of AI talent, it's punching above its weight in terms of the funding that is going to AI companies today.

So the takeaway here is that Twitter doesn't determine whether a market is dead. Data does. Location still matters, even in the highly distributed world that we live in today.

San Francisco, Seattle, and New York are the premier locations for AI talent today. And so your job is to watch the location and funding markets to see where talent and capital is flowing as another way to filter and find the right people.

Now let's talk about timing. Finding the right person has as much to do with time as it does talent. At the airport on the way here, it was a mere matter of minutes that separated me catching my flight or sitting in the lobby sad like Charlie Brown.

A fraction of a second is the difference between a home run to right field or a foul ball in the bleachers. For me, timing means two different things. First, it means finding people when they are most likely to leave.

Secondly, it means finding people that really are going to have a penchant to join a company at your stage and uplevel your team for where you are today. So let's talk about timing.

We analyzed some of those AI V League companies to see their retention rates. If you're on this slide, I'm sorry if the, if the information, uh, offends you at all.

But Anthropic is leading the pack with about a sixty-six percent four-year retention rate, while Perplexity hovers around forty-four percent or so, forty-three percent. This is just a small slice of the world as it's constantly changing.

But the reality is understanding retention helps you know when you're likely to be able to get someone to answer that message that you send to them. It effectively creates a poachability score, so to speak, of your ability to land that person.

But in addition to retention, we actually studied the behavior of different generations. They act differently.

In 2023, nearly 27% of all Gen Z left their job.

If you compare this with Gen X, that's actually more than two times as much as Gen X. And if you were to look at the fact that within four years after graduating, Gen Z has about 2.2 jobs whereas Gen X has 1.1.

And so some of this has to do with the fact that Gen Z is actually getting promoted at a slower rate. Some of it might be causing those slower promotions.

Some of it has to do with the layoff market that took place over that period of time. But if you ask me, a lot of it has to come down to the penchant for people to take risk and bet on themselves.

Gen Z likes that risk. But it's not just retention, and it's not just the generation you were born into. You need to know when people work at different companies.

Being on the New York Knicks in 1973 is very different from being on the New York Knicks in 2018. One of those teams held up a championship trophy, and the other team had the worst record in their franchise's history.

So at SignalFire, we built this cool tool. We call it historical composition. And it actually shows all of the startups that we invest in a snapshot of the companies that they admire at different points in time.

What did their org structure look like at that point in time? Who were the sales leaders that took them from one million to 10 million? Who were the first three engineers on their team when they shipped that key product that I'm trying to beat now?

These things are going to help you identify the risk profile people have. Are they gonna join a company at your stage? It's gonna mo- uh, like, help you understand the motivations that they have, but it's also gonna help you understand whether they're a potential 10X hire, which you need to make in order to take your company to

the next stage. So the takeaway here is that you have to understand timing, both from an outreach and an impact standpoint. You should know when your competitors or the companies that you admire most are likely to lose other people.

You should track the people that work at those companies' profiles to see whether there are changes made to it over that period of time. Studying the patterns of different generations or segments of the population will help you understand how they change jobs.

And finally, y- you need to know when people join and leave companies because that will help you identify your 10Xers and help you identify people that are likely to join a company at your stage.

Now, this is where I finally get to use that literature degree narrative. One of my favorite writers, Kurt Vonnegut, has this awesome visualization for the shape of stories. If you look at the bottom left here, on the X axis you have beginning and end, and on the Y axis you have ill fortune leading up to good fortune.

So the bottom left here is Franz Kafka's Metamorphosis. Gregor Samsa wakes up a bug, and everything goes downhill from there. On the top right, you have a man walking down the street, and he falls into a pothole.

But then he works him- uh, works his way out. But my favorite's Cinderella. She's on the bottom right over here. Things start pretty crummy for her. Her sisters are evil.

She has to do a bunch of work. Then, uh, some magic happens, and she gets invited to this ball, meets a beautiful man, they fall in love, and then what happens?

The clock strikes 12. She falls off a cliff, but then through a series of fortunate events, things lead to eternal bliss. Now, I'm not telling you that you need to preach the depths of despair that your company is in or has gone through, but you do need to understand the triumphs that you've had, why you are where

you are today, where your arc is going. The reason for that is because historically, pay and equity were the two components that we used for narrative, but we can't rely on those solely anymore.

Why? From November 2022 to November 2024, we saw a 1.6% increase in the average tech salary and a precipitous decline in the amount of equity granted. But I have some really bad news for the folks in this room.

It's even worse for AI. AI engineers are the hot ticket for this year. They command a 5% salary premium and 10 to 20% equity premium over other engineering roles.

So what was already expensive is getting even more expensive for us. So if we rely our entire narrative on that, we're relying on things that we might not be able to afford.

So salary as a sole selling point has gotta go.

Equity was that other thing that we used to dangle to get people to buy into what we were doing as a company. But we've seen a precipitous decline in the amount of people that are exercising vested shares.

In Q2 of 2024, 33% of people exercised the shares that they had vested. This is down from 55% a couple years earlier. A lot of this is driven by concerns over, uh, valuations that might be a little bit too high, concerns about the cost of liquid capital to exercise these shares, and concerns about the

market shifting, which it does every three weeks in AI. Equity can't be the only other thing that we're relying on.

We have to get to a point where we're not just focusing on money and equity. We have to have things like- A, a close-knit environment with working with the founders, collaborative teams, speed and the lack of friction to actually get stuff done.

A big mission, the ability to grow your mind and your career opportunities. Markets that are exploding, and solving complex problems. You need to understand what all these things are for your company in order to not rely wholeheartedly on salary inequity as a narrative.

So to summarize, in a world where so many companies are fishing in the same engineering pond, recruiting data can give you an edge. These are just a few examples, but in the same way that you use data to build your product, both your models and the kind of analysis of that product, you should be assessing data to

build your team. Your team is your most valuable product that you have. What we've seen is that de-credentialization is happening, so you need to filter accordingly. Location still matters, so watch where people move.

Data can help you identify the right time to reach out to people, and it can help you f- identify the right time that people have been at different companies.

And all of this is going to help you craft a better narrative. So if you can filter, if you can time, if you can find the right location, and you can have a good narrative, you're gonna do a much better job.

If you're on the other side of the coin and you're actually looking for work, you should know where the people you admire go, not just the companies, but the space.

You should watch how long they stay there. This will help you know how they're treated, whether they think the space is going to be fruitful. And then finally, you should know what you want in that arc of your career.

I'll be out in the lobby a little bit later. Thanks for your time. [clapping] [outro music]