How to Hire AI Engineers When Everyone Is Cheating With AI

Talk transcript

1. 0:00

   [upbeat music] Hi everyone.

2. 0:16

   So I'm Beth Glenfield. I actually flew in from Ireland, so, uh, I will try and slow down my accent for everybody. I have heard that I do get a bit American sometimes, so, uh, yeah, it may not be the [REDACTED:origin] accent you're expecting.

3. 0:30

   So I'm gonna talk to you today about how I believe AI is breaking how we hire technically.

4. 0:38

   So I know everyone's very busy, but just a few questions to think about. Is that okay? Perfect. Is, are you using AI today in your recruitment process, both as an interviewee and an interviewer?

5. 0:54

   Are you competing against Google or Meta employees, both for positions but also for candidates? And are you recognizing that LeetCode puzzles just aren't really up to scratch like they used to be for knowing if people will build great AI products?

6. 1:13

   So welcome to twenty twenty-five, where the technical recruitment process is very broken and small companies are getting crushed in the talent war.

7. 1:23

   So just to refresh our memories, let's look back at what happened over the last eighteen months. So we had Clue come to the table. Uh, they have raised a five point three million round, and it is a AI cheating service from a ex-Columbia student.

8. 1:37

   Um, they're currently heading towards one million in ARR. And, um, we're also seeing ninety-three percent LeetCode wizard success rates for Goog- Google and Meta interviews, and we're seeing, like, one in three interviews now are having AI assistants.

9. 1:54

   So really whenever you're interviewing candidates, you're just interviewing for who has the best AI coding assistant.

10. 2:03

    So Sam Altman said this very plainly, "The tactical thing to do is to learn the best AI tools." We also saw that Marc Benioff announced that Salesforce will no longer be hiring software engineers this year because according to them, they have seen a thirty percent boost in their productivity since they've replaced people with AI.

11. 2:24

    I would like to see the data on that. But really, jobs have fundamentally changed.

12. 2:31

    And you're not just competing on compensation anymore. You're competing on brand recognition, career prestige, and perceived stability. There's a lot of people being laid off, and people are looking for security.

13. 2:43

    So when a candidate has to choose between your Series A startup and Google's AI division, we kinda know what really happens there.

14. 2:53

    So the candidates you actually want for AI development, the creative problem solvers, the collaborative leaders, and the ones who can work with AI instead of re- being replaced by AI, they are not being optimized for LeetCode performance.

15. 3:09

    Instead, they're building AI tools and using AI libraries. They're contributing to open source. They are understanding how the business impact will be impacted, and that is just not what LeetCode is indexing for.

16. 3:24

    So what if instead of asking candidates to solve puzzles that they will never actually use in a, a job, you could observe how they collaborate with AI teammates on actual business scenarios?

17. 3:41

    What if instead of measuring their ability to memorize algorithms, you can see how they delegate tasks, how they handle ambiguity, and how they deal whenever requirements change mid-sprint? So you're not competing on brand recognition.

18. 3:55

    You can demonstrate to a candidate at the beginning of a process what your engineering culture is and see how they would perform in your environment.

19. 4:06

    So at DevDay, we are completely reimagining the technical hiring process in the AI era. Here are some examples of what we're seeing today, like I mentioned earlier, and then we'll get into what we're considering is the new kind of process you should be looking for in a hiring process.

20. 4:26

    What we, what we do instead of getting candidates to code interviews is we create real-life workplace simulations. Candidates work alongside AI agents with very different personalities. You can have the perfectionist, the pragmatist, the security expert, and even the junior developer that needs extensive mentoring, so that now they have to make trade-offs on day in, day out situations.

21. 4:50

    They're not solving contrived problems. They are building features specifically for your business domain.

22. 4:57

    So instead of measuring how c- they can code, we're actually, actually measuring them on skills that matter. How do they collaborate with AI? How do they handle ambiguity? How do they communicate these technical decisions in pull requests, in comments on tickets?

23. 5:14

    And how do they mentor others and adapt when everything changes on a daily basis?

24. 5:20

    Now, if you do work at Google or Meta, you can just brute force your hiring process. You interview a hundred candidates, you hire five, and you give them great paychecks.

25. 5:32

    But for the majority of companies, we don't have the luxury. You cannot afford to make one bad hire, and you can lose up to twenty to sixty K in that bad hiring process.

26. 5:43

    And you definitely can't afford to hire someone who doesn't know how to ship AI products.

27. 5:49

    So look, the future is coming for us whether we like it or not. Uh, Mark Zuckerberg says that AI will handle mid-level engineering work by later this year. TechCrunch is also reporting that we're wiping out engi- entry-level engineering jobs.

28. 6:05

    But that doesn't mean fewer engineering jobs. They're just very different engineering jobs, jobs that require creati- creativity, collaboration, and the ability to work according to business judgment and not just code, jobs that require working with AI instead of being replaced by it.

29. 6:26

    So we're currently working with select design partners. If you're interested in thinking about this process or learning more or just want to chat about how you're thinking about hiring, let me know.

30. 6:36

    My details are on the screen, or just come find me at the event. Thank you. [upbeat music]