The Weekend AI Engineer

Talk transcript

1. 0:00

   [upbeat music] In this talk, I'm gonna walk you through, uh, some of my projects that I've built and all of the lessons that I learned along the way to build great AI apps that can scale to millions of users.

2. 0:25

   So let's get right into it. So to, to set the stage with some context, I've been building side projects, uh, pretty consistently for about two years now. Uh, and so last year I built about 11 side projects, and they got about 20,000 visitors total, uh, so not too shabby.

3. 0:40

   Uh, so my goal for this year was to try to double that number and get to 40,000 visitors. And, uh, happy to announce that I did hit that goal and slightly exceeded it as well.

4. 0:50

   Um, and, uh... Thank you. Um, and, uh, basically here today to talk about how this happened and, you know, very thankful and, and very lucky that, that I managed to, to hit such a good number, over 8 million unique visitors, uh, across all of my projects, 20,000 GitHub stars, and about 2.8 million people, uh, that signed up.

5. 1:11

   And fun fact, every single one of these projects that I launched was built on the weekend. So, um, I'm gonna pick through some of these projects, and we're gonna go through them and, and talk about some, some lessons learned.

6. 1:23

   I also want to mention that everything I do is open source, so you can check out all of my projects at [REDACTED:url]. Embarrassing [REDACTED:username] username from like 10 years ago that I can't get rid of. [laughing]

7. 1:34

   Um, but, uh, yeah, no, I love building in open source and, and it makes me so happy to see people, uh, use my projects. Uh, but it's also a very good growth lever when you launch.

8. 1:44

   Um, and I get a lot of genuinely helpful PRs from, from a lot of people, uh, that are, uh, better at prompt engineering than I am, so that's always helpful.

9. 1:53

   Uh, disclaimer, I, I do have a bit of an audience on Twitter, uh, which is very helpful, but honestly, I don't think it's as important as people make it out to be.

10. 2:01

    Uh, a lot of people, um, a, a lot of people can kind of attribute having a lot of followers to having successful projects, but I, I've seen plenty of people have very successful side projects with little to no Twitter following.

11. 2:13

    And in fact, less than 5% of the traffic of those 8.5 million people that have visited all of my projects, less than 5% of that traffic actually comes from my Twitter account.

12. 2:23

    So you may be thinking, where does this traffic come from? And, uh, honestly, it's a lot of word of mouth and Google and SEO and, uh, other influencers sharing it.

13. 2:31

    So I'm gonna get to that, uh, in a bit as well. Um, so today I wanna talk to y'all, uh, like friends, and when I talk to my friends about my projects, I kind of just share, um, my laptop and go through a bunch of things.

14. 2:43

    So I'm gonna, I'm gonna switch over to, uh, my laptop here and, and go through a bunch of, uh, my side projects.

15. 2:51

    So let's do that. Wonderful. So this is kind of the-- my, my first AI project, how I got into AI last December. And really it stemmed from, uh, this, this problem that we had where we had just run a conference last year and we had several hundred photos, uh, out there in an image gallery.

16. 3:07

    And, and right before we published it, my CEO, uh, came up to me and was like, "Hey, we probably need to add alt tags for a lot of these images."

17. 3:14

    And, uh, that would've been a very painful process going through several hundred images. So I, I looked stuff up and I found a nice image-to-text API that ended up working really well.

18. 3:23

    You know, I went and I, and I checked a lot of these, um, a lot of the alt tags and maybe fixed like two of them and published. But this was really my big like light bulb moment of like, oh my God, AI can really, really help you save a ton of time.

19. 3:37

    Like this isn't some Web3 hype from last year. You know, this is real. [laughing] Um, no, I'm kidding. Web3 has its place for sure, but this was really the, the, the big thing when, when it came out.

20. 3:47

    So, uh, I built this little open source project. I put it out there, and then I just started having fun and building other stuff. So I built another project called QRGBT with my friend Kevin at, at a hackathon.

21. 3:57

    Uh, and so the idea is that you just generate, um, just pretty nice QR codes. So we can actually go and, and, uh, generate a QR code for AI, um, ai.engineer.

22. 4:06

    I forgot the domain name. Uh, and we can, can select a prompt here. I'm gonna just click, uh, one of the pre-generated ones, uh, a forest overlooking a mountain.

23. 4:13

    And hopefully in like five or six seconds it should generate a QR code that links to the conference that just looks a little bit better than the black and white QR codes.

24. 4:21

    Um, and so we built this and we, we weren't expecting way too much, um, because people really don't have to generate QR codes every single minute. Um, so yeah, we put it out there.

25. 4:31

    Got about 8,000 visitors, about 8,000 QR codes generated, and so we were like, "Okay, cool." Um, and I was like, all right, I wanna try to build something that has more like daily active users or people that will use it consistently.

26. 4:43

    So I built this little, uh, tool that summarizes, um, TechCrunch articles. So the idea is that you go to techcrunch.com. You can click any article that you want, and all you have to do is add summary to the end of the URL over there, and it'll redirect you to my website and kind of summarize the whole article

27. 4:59

    using GPT, uh, 3.5 in a couple bullet points. Um, and so the reason I'm showing you a video here and not a live demo is because I got a very nice email from the TechCrunch lawyers- [laughing] ...

28. 5:10

    uh, when I launched this telling me to take it down. So that was, that was a lot of fun. Uh, but yeah, anyway, I, I took it down and I moved on.

29. 5:18

    That one did, it did pretty good when I launched it, and then they made me take it down and, and, uh, it kind of, it kind of died off from there.

30. 5:24

    And then I started just like replying to random people on, on Twitter. So Samina here asked like, "Can someone help me build an AI to help me pick glasses?"

31. 5:31

    So I was like, "All right, bet. I got you." [laughing] And, uh, built this little thing in like a couple hours where it takes some information about yourself, your face shape and your, your gender, and you can add some relevant context, and it uses a combination of LLMs and the Amazon API to find the ideal glasses for you

32. 5:47

    and actually links them on there so that you can buy them. Uh, so yeah, just started replying to a bunch of tweets. Another one was by my friend Theo, who said someone should make an app that kind of auto-generates commit messages for you.

33. 5:59

    And, uh, and then my CTO tagged me and was like, "CC, uh, Sasan, I love that idea," which translates to

34. 6:05

    build this as soon as possible. Um, so I was like, "All right. I got you." And, uh, I, I built a little, uh, I built a little tool. So essentially, you could run Git add, you run the CLI tool that I built, AI commit, and it analyzes your Git diff and produces a little commit message for you

35. 6:20

    that you can then, uh, use to commit. Um, and these are v- like very small hackie solutions. You know, my CTO tagged me at 7:53 PM on February 11th, and then less than two hours later, I replied [laughs] with that little, wi- wi- with that little script.

36. 6:35

    Um [audience cheering] thank you. And after I saw it get some attention, I was like, "Okay, I need to clean this up. I need to figure out how to bundle it into an npm package."

37. 6:45

    Uh, and so, uh, that's what I spent my Monday morning on. I hope my manager isn't watching. But, uh, that, that was, that was a fun Monday. And yeah, kind of bundled it out there and posted it as, uh, an npm package.

38. 6:57

    And now I think over, uh, 30,000, uh, developers, uh, are, are now using it to, to commit their messages, and it's one of my more popular open source repos.

39. 7:06

    Like, there's some PRs that I need to take a look at. But, um, yeah, a bunch of, uh, 6,000 stars and about 25, uh, contributors. And so this was kind of my exploration with, with LLMs.

40. 7:16

    And, and so actually I, I have one more project called the Twitter Bio Generator. And essentially, uh, also open source like most of my other projects. But, um, you just put in some context, uh, about you, so we can do like engineer at Microsoft, and we can say, "Loves volleyball."

41. 7:31

    And, uh, pick a vibe and it'll make your Twitter bio for you, and kind of stream in text from, uh, GPT-3.5. Um, spiking code bugs and volleyball balls. You, you can't get any better than that.

42. 7:44

    Um, but you might, you might take a look at some of these projects and think like, "This is so simple," like, "Nobody's gonna use this." This is just like this little ChatGPT wrapper, like everybody in this room can build this thing.

43. 7:55

    Um, but, but, but I think we, we constantly underestimate, like the, the majority of the world are not AI engineers. Nobody can build this. A lot of people haven't even used ChatGPT yet.

44. 8:05

    Like, it's, it's crazy. So even the simplest apps can do really, really well. And so that's a common theme that you might see is like all of these are very, uh, very simple apps.

45. 8:13

    So I launched it and, and it got about 200,000 visitors, uh, that, that used it. I got about 100,000 people in a single weekend, and then, um, I hit my, uh, OpenAI bill and had to shut it down for a little bit, so. [audience laughing]

46. 8:26

    It's always a good sign. Um, and so after this I kind of switched into image-to-image models. So I built this, um, photo restore website that basically unblurs old photos.

47. 8:35

    And the motivation behind this actually was, uh, my parents sending me these old photos. So I'm actually gonna put in a picture of my dad doing karate when he was like [REDACTED:age].

48. 8:43

    And he sent me this photo, and his face was really blurry. Um, and, uh, you'll see, yeah, he's flexible. I do not, I did not inherit that. Um, but you see his face is a little bit blurry.

49. 8:54

    You can't see it too well, but hopefully in the space of a, of a few seconds we should see, um... And so this uses just a, a GAN model.

50. 9:01

    It's called GFPGAN. Uh, it sends it to that model, and it will basically, um, scan like all the faces in, in a picture and restore it. So we'll see, uh, if the internet is working out.

51. 9:14

    Um, we'll hopefully see [laughs] the image come in in a few seconds. And if not, I can move on and, and come back to it.

52. 9:21

    All right, I'll come back to it. Um, so again, open source repo, um, and this one like really, really did well. Um, and it kind of is my most consistent project.

53. 9:31

    It, it still has about 250,000 people that, that use it, uh, every month. Um, mostly actually in India and Indonesia, which makes a lot of sense because the, uh, the phone cameras there are a lot lower quality.

54. 9:42

    Um, so it makes sense that they would use it. But shortly after it went viral, I got a lot of, uh, inappropriate images being uploaded. [audience laughing] And so I had to, [laughs] uh, I used actually TensorFlow.js, uh, and I published this as a library as well.

55. 9:54

    Um, but yeah, I just ended up using this to, to scan the image and make sure it was safe before I processed it. So let's go back. Okay, so it looks like it, it was restored.

56. 10:02

    I'll actually put them side by side and zoom in a little bit so you can see his face before, a little bit blurry, and then after the transformation you can see it- [audience cheering] ...

57. 10:08

    it really, really clears up. Thank you. And, um, really ano- another thing I wanna stress here is that this is one single API call to this GFPGAN model, and that's it.

58. 10:21

    And, and it's really getting that and displaying it back to the user. Um, so there's i- it's, it's such an exciting time to be an AI engineer and to build this stuff 'cause it's so easy and it's so impressive to other people as well.

59. 10:32

    Uh, and so my, my la- I'm gonna talk about one more project and then I'm gonna start to talk about some takeaways. Uh, but before that actually, this is like the architecture for, for most of my apps.

60. 10:41

    Uh, really I, I use Next.js on the front end and the back end. And you saw for restore photos there's this little upload component that I use, and so that it, uh, the user uploads an image, it gets sent to cloud storage, and then I send that image URL to my Next.js API route.

61. 10:54

    Or you can think of it as just like a Lambda function. Um, and then that sends it to my machine learning model, to GFPGAN, to get restored. It gets back the image, sends it back to the client, and display it to the user.

62. 11:05

    So this is kind of the architecture I use for a lot of my image-to-image, um, side projects. Uh, by my last one, which um, I'll, I'll restart. But my, my last one, uh, that did the best is actually called RoomGPT, and it's that idea of, um, if you give it a room, I'm just gonna give it a

63. 11:22

    random living room from the internet and we're gonna select a couple themes. But if you give it a, a room and some themes, uh, the idea is that it'll use this and it'll help you redesign your room.

64. 11:34

    It'll, it'll give you different variations of that specific room, different color themes, different like couch styles and stuff like that. So we can see it just finished. You can see, uh, it, it really respects the structure of the room.

65. 11:44

    So it looks the same, but it gives you, you know, different ideas for these like tables and backgrounds and tiles and, and everything like that. Um, so really the, the motivation behind this project was that I, I saw somebody else built this before, but they used Stable Diffusion.

66. 11:57

    And Stable Diffusion actually does a notoriously bad job at maintaining the original structure of a room. Like you can give it a room, you can tell it, "Okay, redesign this in this theme," and the image it produces looks nothing like the original room.

67. 12:11

    Like the dimensions are messed up, the depth is mess- is messed up. And then I saw this new model called ControlNet that came out. Uh, and ControlNet does r- really well at maintaining that structure of the room.

68. 12:21

    So I saw that and I was like, "Oh, this, this could be cool to build." Um, so I, I put it out there and I launched it on, uh, on Twitter, and obviously it's also, it's also open source.

69. 12:30

    But I, I launched it on Twitter and, uh, it, it did pretty well on there and, and kind of, um... Kept tweeting about it. 'Cause the thing about Twitter, when you tweet about something, 24 hours later it's kinda dead.

70. 12:41

    Uh, so what I like to do is I like to kind of post updates over and over again. So, uh, we had about 10,000 people that used it in the first 12 hours, and then, um, 30,000 in, in the first day, and then I added some testimonials.

71. 12:55

    Um, may or may not have paid these people. And then, yeah, two days later it had, like, 90,000 people, and then three days tw- 270,000 people. And so it kinda just, it kinda just blew up, and I feel like it was just, it was mostly because I was one of the first people to kind of prod- uh,

72. 13:10

    productionize this, this ControlNet model that had just, that had just come out. So a lot of people were seeing it, uh, for, for the first time and using it.

73. 13:16

    And, uh, most of these users, again, I can show you the, the analytics chart. So I have about six million people that have visited the site, and about a little over two million that, that have registered and, and used it.

74. 13:25

    And you can see the vast majority of the traffic is, is just Google, is just, uh, s- straight up from Google. You know, it ... A, a lot of people kept sharing it, and, you know, part of that I think is because it was open source and a lot of developers liked it and re-shared it.

75. 13:38

    Uh, but also, uh, the fact that I kept it free. So I'm gonna, I'm gonna talk about how I did that kind of, uh, when I transition back to, uh, slides.

76. 13:46

    So those are some of my side projects. Um, one thing I wanna call out is, um, it's a really good idea to use AI-enhancing tools when building a lot of this stuff.

77. 13:53

    So use GPT-4 for your code. Uh, we have an AI SDK that you can use, uh, over at Vercel. And we also have this product called v0 at Vercel, and so it helps you kind of generate UIs.

78. 14:04

    Uh, and what's really cool is you can kinda see other people generating UIs. We can click on, uh, this one for example, which looks like the Apple Notes UI.

79. 14:13

    Um, and we can actually fork ... We, we can look at the code, which is cool. So we can ... I can copy all this code. But what's also cool is I can look at these templates or look at other people's code, and I can fork it, similar to how I can folk- fork a GitHub repo.

80. 14:24

    So now this is mine. I can kind of add additional prompts to change it, or I can click this button over here and actually select different elements within the page.

81. 14:32

    So I can select this div and tell it, like, uh, add, uh, three more notes and alternate their colors. And I can press enter and update, and what it'll do is it'll just re-render this specific div, and it'll stream in the data using our Vercel AI SDK.

82. 14:48

    It'll stream in these React components. Um, and, uh, yeah, hopefully it'll, it'll, it'll keep going and, and add all this stuff in. And again, as it streams in these components, it adds them inside of this, uh, code box over here.

83. 15:02

    Um, so we'll ... I, I think it's still generating. But eventually, you know, it'll, it'll add all, all of the notes here, and we can go into the, the code and kind of copy and paste it.

84. 15:13

    Uh, and we can also run a CLI command. You can see it scrolls down because it's still generating. Here's, yeah, note three, note four, note five. There we go.

85. 15:19

    So added the five notes. I can go take all this code or run this command and, uh, get all the code and kind of iterate on UIs that way.

86. 15:25

    So it's just a way to kinda prototype a very early, uh, UIs. So I'm gonna go back to slides right now to talk about some, uh, takeaways.

87. 15:37

    So use AI tools to move faster. I, I mentioned that. I mentioned the AI SDK. I mentioned v0. But there's a lot of really amazing libraries. I, I love using, uh, Replicate and Hugging Face and Modal and, and a lot of these other tools, and Brev.

88. 15:49

    Uh, there, there's a lot of really cool stuff you can use, uh, to, to kind of train your models or, or move faster, uh, when you're coding. So this is a bit of a spicy one.

89. 15:57

    Um, I always tell people, "Don't do any fine-tuning, and don't build your own models." And this is specifically for launching MVPs, 'cause, uh, again, the purpose of this talk and everything is, like, building projects very quickly on weekends.

90. 16:10

    So you don't have time to fine-tune. You wanna keep things very, very simple. If you can't describe your idea to me in five words, like, i- it might not do great, you know?

91. 16:20

    I have friends that come up to me that are like, "Oh, I wanna build this platform for developers where they can connect them to clients, and they can have their portfolios there, and they can have a chat, and they can have this," and I just, like, stare into them, and I'm like, "That's, that's not gonna happen."

92. 16:32

    Like, that's not ... You can't build that in a weekend, you know? If you can't build a qu- ... So what I tell them is just basically down-scope to an MVP and then launch it.

93. 16:40

    And even RoomGPT, when I launched that, I had so many machine learning engineers that DM'd me on Twitter and were like, "Oh my God," like, "What models did you train?

94. 16:47

    What parameters did you use? How did you get the data? How did you clean your data?" I'm like, "Dude, I just used, like, an API off the shelf," you know? [laughs]

95. 16:54

    Uh, you don't, you don't need ... You can do so much with, uh, off-the-shelf APIs. Another one is use the latest models. I mentioned a big part of RoomGPT's success is, is using, um, ControlNet, which had just come out a couple days before.

96. 17:05

    Uh, launching early and iterating is so, so important, uh, 'cause you don't know what's gonna do well. So if you can de-risk your projects, if you can get a project out in one or two weekends, and if it fails, so what?

97. 17:17

    You can pivot. You can move on to a new idea, and, and you can just, just ... Yeah. You, you can just move on to other things. Um, and so ...

98. 17:24

    And if it does well, then you can double down on it. Then you can add, uh, additional things to it. So I, I've found that to be, to be great.

99. 17:30

    Another one is making it free and open source. Making things open source is, is always great because, uh, people learn from it and are incentivized to share it, and will open PRs to your project, um, and will also get you a bunch of followers.

100. 17:40

     You know, I gained, like, 25,000 Twitter followers this year just from posting a bunch of these open source projects, and they're just all developers, uh, wanting to learn and, and, and help me out.

101. 17:49

     So open source is amazing. Uh, making things free is a little bit hard, right? 'Cause as we know, AI workloads are really, really expensive. And so, uh, there's a few ways you can do this.

102. 18:00

     I kinda play to my strengths. You know, I have a Twitter audience, so I can go to companies and be like, "Hey, I wanna launch this project. I think it'll get X amount of users.

103. 18:07

     Um, please give me some credits, and I'll shout you out in the footer, and I'll put you in the README and all this stuff." But I've seen a lot of other people replicate this with no followers, and the key is to just build a very high quality open source project, put it out there, put a, like, a

104. 18:19

     $50 limit on it, and when you run out, you can reach out to the company and say, "Hey," like, "my project went viral on Twitter, and it's featuring you, and the, the GitHub, uh, repo is open source."

105. 18:28

     So when, when companies see this, they're, they're kind of willing to, um, give you some, uh, credit. So shout out to Replicate and Bytescale and Neon and, and a bunch of my other, uh, sponsors that help me keep a lot of my AI projects free.

106. 18:41

     And the last lesson that I have for you is making sure your UI looks good. Nobody's gonna use your product if it doesn't look good. Uh, that's just something that's been learned.

107. 18:49

     And so I actually spend, like, 80% of my time on the UI. Even though these are, like, AI projects, most of the time is on the UI 'cause you need to make it look good.

108. 19:00

     Uh, so, um, and if you're not a designer, you can just take inspiration from a bunch of different websites. Um, and that's what I do. I'm not a designer, so I just look at, like, five other websites and I kind of, uh, steal a little bit of each site to make it look good.

109. 19:13

     'Cause, uh, I don't know how to just come and make a website that makes ... that, that looks good, but I know when something looks good when I see it.

110. 19:19

     So, uh, that, that's kinda what I do. So very quick, uh, summary. Um, if you do these five things, I, I think you can go very, very far. And lastly, like, I, I tell people to use whatever tech stack, uh, they wanna use.

111. 19:31

     I like the tech stack of, like, Next.js and TypeScript and Tailwind. Uh, lets me move really quickly, and then using, uh, Vercel for deploying my apps. Two final ideas and then I'm gonna get off the stage so better speakers can come and, and tell you about their projects.

112. 19:43

     But, um, I don't work 24/7, despite what you might think with, with, with all of that. I actually spend most of my weekends relaxing. Uh, but what I do is I work in sprints.

113. 19:52

     So I'll take a single weekend and I'll just drop everything and go and try to put out a project. And, and then for the next, like, two or three weekends, I'll just binge Netflix shows and hang out with friends and live my life.

114. 20:05

     Um, so this has worked out for me, but when I say, like, I work all weekend, I mean, like, 12 hours Saturday, 12 hours Sunday kinda deal. You know, I kinda drop everything and do that.

115. 20:13

     And so if you have flexibility in your life to do that, you can go ahead and try it. If you're [REDACTED:marital_status] or have kids or have other responsibilities, you can experiment with what works for you, you know?

116. 20:22

     You can spend a couple hours every weekend here and there. Um, but, but that's what I do. Basically a weekend a month where I sit down and I put out a project and then relax, uh, for a little bit.

117. 20:32

     Um, so yeah. Moral of the story is I think, like, do what works for you. I'm just kinda sharing what, what's worked for me. And the final thought I wanna put out there is that you need to, like, put in the hours.

118. 20:42

     I think pe- a, a lot of people DM me and are like, "Hey," like, "I'm feeling really unmotivated 'cause I'm trying to build these projects and they're taking me so much time," and like, uh, you know, "How do you do it?"

119. 20:52

     Like, "What's your secret?" And, um, the first thing I ask them is like, "Oh," like, "I'm sorry to hear that. Uh, how many projects have you built?" And more often than not, they're like, "Oh, this is my second project."

120. 21:04

     And I just stare at them and I'm like,

121. 21:07

     you can't go to the gym for the second time ever and then look down and be like, "Where are my biceps?" Like, "Where, where..." It doesn't work like that, you know?

122. 21:15

     You have to go to the gym consistently over months to see progress. And so the same thing happens with, with side projects and coding in general. And if you're an engineer, that's even better.

123. 21:23

     I, I'm not an engineer, actually. I, I don't do ... I don't write code for most of my time at, at work, and I just learned to code a few years ago.

124. 21:30

     So I think genuinely anybody can do it. Um, you just have to, to kind of, uh, put out the, put in the hours and, and build and good things will happen.

125. 21:39

     So thank you so much for having me. [audience cheering] [upbeat music]