Git push, get an AI API.

Talk transcript

1. 0:00

   [upbeat music] At Hypermode, we believe iteration is everything, right?

2. 0:17

   That's why we built a platform that makes it really easy to iterate over AI features. And over the course of today, over the next 50 minutes really, you're gonna be able to build a couple of AI features and incrementally iterate on them, right?

3. 0:28

   And we think that incremental iteration is everything, and so that's why we're gonna walk through it.

4. 0:33

   So let's get started. As we dig in, there's a few things to be aware of. We are just launching, so I'm sure there'll be a couple things that break, so bear with us on that, but we'll work through it.

5. 0:43

   Um, we are gonna skip some steps to just condense this. So you're gonna push right to main. That's okay. That's always fun. Um, but let's, uh, let's really just kind of start to dig in.

6. 0:55

   And overall in the agenda, we're gonna start with really kind of showing you a, a few different ways where this plays out. So we're gonna start by playing a familiar game, where we're using AI to scale the format.

7. 1:07

   Then we're gonna understand how that game's constructed and kind of break it down to the building blocks, and then start to apply those into triaging GitHub Issues. We picked that because there's open data that we can easily use on that.

8. 1:17

   You can think about applying that to a lot of different contexts, whether that's customer records or product records or other things in your application. But we'll start with GitHub Issues and talk about how we can generalize those concepts to add AI to your application.

9. 1:29

   So we've created a game that we call Hyper Categories. It's like the familiar game Scattergories, but it works for a bigger format of people, right? That, the traditional game doesn't really work, uh, for more than six or eight people.

10. 1:41

    So we're gonna invite you all to play with us, so you'll need your phones in a second for a QR code to kind of join into this. But the prompt really is, is pretty simple.

11. 1:48

    Given a starting letter, provide the most unique entry that matches each category. You'll get a point if you actually match the category and you start with the right letter, but you'll share that point with anyone else that says something similar.

12. 2:00

    So if the category was furniture and the letter was T, and you said table, anyone else that says table, you're gonna share that point around that, that way. So you need to be unique, right?

13. 2:09

    And we'll talk about how we're using AI to kind of power this. But it also helps you to, um, start to think of act- think about the building blocks that you're gonna use, right?

14. 2:17

    So let's actually jump in. We'll play and then we'll kind of break it down afterwards with how it's actually running.

15. 2:26

    We do. We have a, uh, a Hypermode jacket for whoever wins, so...

16. 2:33

    So Matt's gonna get that built and running.

17. 2:35

    There we go. All right.

18. 2:40

    Oh.

19. 2:41

    And let me... Can everybody see that? All right. We are playing with things that start with the letter T, so please, uh, scan that QR code if you can.

20. 2:52

    Hopefully everybody can. If not, there is a link to it in the, uh, in the doc that accompanies this workshop.

21. 3:02

    Which they can get to how?

22. 3:04

    Yep. [laughs]

23. 3:07

    Uh, hype.fu/workshop, right?

24. 3:10

    Yep.

25. 3:10

    Okay, great. We'll let it go a little longer than the, the countdown timer here.

26. 3:17

    Yeah. It won't, it won't cut off, so if you haven't joined in yet, feel free.

27. 3:34

    There is a bot playing against you as well, so...

28. 3:38

    I'm curious to see, uh, the creativity that people come up with here. [laughs]

29. 3:44

    The bot tends to pick the o- obvious answers, so it doesn't do very well in this one.

30. 3:56

    Need some music. Little Jeopardy music. [laughs]

31. 4:14

    There we go.

32. 4:15

    Thank you. [laughs] A little improv there. [upbeat music playing] Yeah. I see people typing still, so we'll give, we'll give it a little bit more time.

33. 4:40

    All right, we can end it whenever you like.

34. 4:48

    I see most phones down, so another five seconds?

35. 4:53

    Five-

36. 4:54

    Why don't we see the results?

37. 4:55

    ... four, three, two, one. [snaps fingers] All right.

38. 5:00

    Where is Kirk Byers? Congratulations, come see us after. We'll g- grab you a jacket.

39. 5:07

    Awesome to see.

40. 5:07

    Oh, yeah.

41. 5:07

    So you see the bot there in, in fourth place, so great job for three of you for beating the bot.

42. 5:12

    And we have a tie for fifth.

43. 5:14

    Oh yeah, sorry. I admit- look at it. Uh, Francisco,

44. 5:19

    awesome. So come see us after. We, we have two jackets. We can make that work.

45. 5:22

    Shall, shall we look at the results?

46. 5:23

    Yeah, let's look at it.

47. 5:24

    All right. Things you wear.

48. 5:27

    Tevas. Oh, that got an invalid. So we'll, we'll dig into that. AI's not always perfect in the evaluation. Okay. Still won.

49. 5:35

    We, we need to train the model better.

50. 5:36

    Yeah. Cool. So why don't we jump over to show you how this is actually working behind the scenes. This is a d- a template that you can deploy. You can play it back at, at your company or wherever you're at.

51. 5:45

    But really, we start to think about these type of AI applications.

52. 5:50

    Can we get the screens lined up?

53. 5:53

    Sure can. The one on the left is a little off there.

54. 5:54

    Oh, there we go. Um, as these building blocks, right? That you're starting to put, to assemble these into your application. Right. So when you submit that response or when the bot submits the response, we score that as well.

55. 6:04

    We validate the starting letter, we validate the category, right? And so we are using a model to make sure that you are hitting that category as we saw in the results there.

56. 6:11

    It's not always perfectly right. We can continue to train that. But one of the things we learned as we were building this was if the letter was P and the thing was occupation, you could type in P doctor and the model would think that that was valid, right?

57. 6:24

    So that's why we added a dictionary validation as well. And we really see this as illustrating th-the way we see the world of AI applications are developed. It's this mix of models and traditional programming paradigms that come together that actually make it useful, right?

58. 6:38

    So we have that filtering stage, and then we go into the scoring stage, and we start to cluster those responses. We look at it and say, "If you said table and someone else said tables, we consider those the same answer," right?

59. 6:48

    So we're using, um, those processes to make sure that those things are matched, and it's not just an exact text search match. These are ways you could start to scale this game to a size of this group, right?

60. 6:58

    And then that responds to the leaderboard. And we show all this just so you start to get a feel for how these things are assembled. It's a, a set of building blocks.

61. 7:06

    Behind this is a set of functions, models, and data sources, right? So you start to think about this at a, at a deeper level. It's basic programming concepts, right?

62. 7:17

    We have a set of functions. We have a collection of user responses that we're doing similarity search across, a set of models that help power both those functions directly as well as the embeddings for the models or for the collections.

63. 7:28

    And we're gonna walk through all this in the n-next example, and you're actually gonna get hands-on it and build these pieces into some connections, right? Whether that's within Hypermode itself, to out to your database, to a traditional API, to a model host.

64. 7:40

    We make it really easy to stitch all that together.

65. 7:43

    And so that's what we wanna show you now is really to apply these concepts into triaging Gi- Gi- GitHub issue. So if anyone has ever worked on an open source project, you s-- you know that you get GitHub issues from a wide variety of sources, and quality is not always great, right?

66. 7:57

    So we thought, what if we could actually apply AI to make it easier to triage those issues, to understand what type of issue it was, were there similar issues that may be already reported or trends that you're starting to see, and actually be able to summarize trends coming out of the repo, right?

67. 8:11

    So that's the part that we would love for you to build with us.

68. 8:15

    So-

69. 8:16

    You guys ready? We're gonna, we're gonna build this. Actually build it. Uh, if you've got laptops out, that'd be good. Uh, I'm gonna get out of here, and we're gonna, we're gonna kinda work together.

70. 8:25

    The, uh, all this is documented, by the way, um, kinda chronologically the way we're gonna follow in the workshop. Um, and to get to there, there's a, uh, a short link, which is, uh, what is it?

71. 8:35

    hype.fu/workshop. That'll just take you into our di- docs link, or you can go to docs.hypermode.com and, and go find the AI World's Fair tab here on the left. Um, and feel free to browse around the rest of the docs and learn about Hypermode as you go.

72. 8:50

    Um, this is the platform we're using to build this, but also the concepts we're gonna go through here really could, could work any way that you're building an application that's going to leverage AI and functions and data and all that stuff here.

73. 9:01

    Um, let's see. We already, we already kinda went through a lot of this basic stuff with hypergories. Um, I will just say for prereqs, um, if you hadn't seen this earlier, um, hopefully you've got some kinda text editor.

74. 9:11

    I like to use VS Code. Um, you'll need Node.js twenty or higher. Um, and that's, that's pretty much it. Um, obviously Git and, and some way to, uh, to, to clone a repo and so forth.

75. 9:21

    I like to use GitHub Desktop, so you might see that flash on and off, but you can use the command line or whatever you're comfortable with. Um, so let's, um, let's start with GitHub issue, uh, triage.

76. 9:33

    We're gonna go to this website we set up called [REDACTED:url]. Um, and this is, um, this is something we just launched, so, um, we'd love to hear any feedback you may have from it.

77. 9:45

    There are a lot of other templates on here, but we're gonna work with this GitHub issue triage template for the sake of this workshop. Um, and if you're following along, so just click on the GitHub issue triage, and we're gonna click...

78. 9:58

    It's gonna tell you a little bit about the t- the template. You can click to go view the template, but we're just gonna de-- we're gonna go right ahead and deploy it.

79. 10:04

    Um, now for most of you, this will be the first time that you've, um, been into Hypermode. So, um, you will get a screen that looks like this, but when you go to connect it to your GitHub account, it's gonna ask you to sign in.

80. 10:16

    Um, and that would be a normal, uh, GitHub login. So if you, if you don't have GitHub, then you would need a GitHub account in order to continue. Question in the front.

81. 10:23

    Um, if I sign in, if I sign in, I need to give it access to all my current and future repositories. If I only select one existing one, it doesn't work.

82. 10:33

    You should only need to give it access to the repository that we're, uh, working with today. If you're seeing something different, we'd love to see what that looks like. [chuckles]

83. 10:43

    Okay.

84. 10:43

    Um, yeah. They, uh... And, and if, if you have any concerns about that and, and don't wanna continue, you could just watch along or we could, um, we could, we could check afterwards.

85. 10:52

    I know that... I can't remember the exact detail, but I know that there's something on GitHub where some of the wording that GitHub gives you on that first link is a little confusing.

86. 11:01

    But, um, yeah-

87. 11:01

    It, it's only requesting repos per-permissions. That screen is a little confusing of the first time if you don't have any existing repos.

88. 11:08

    Yep. Um, so you're gonna basically, uh, this is me here, Matt Johnson Pent. And, uh, if you have other orgs, of course, you can, you could pick them here.

89. 11:17

    Um, and you can name your repo anything you want. Uh, you can make it private or public. Um, but we're gonna create that. And, um, I'm gonna go to the one that I pre-created.

90. 11:28

    It should land you, when all is said and done, let me find it, on something that looks like this.

91. 11:37

    Um, so we'll go through this a little bit more later, but this is basically, um, setting up cloning a template off of GitHub. Um, and if you look on GitHub, you'll see that there is a template, let me zoom in, um, that was cloned from the ship issue triage, um, template repo.

92. 11:57

    And it's pretty bare bones. Um, it's got some, some boilerplate set up, but most everything we're working with here today is gonna be in this functions directory.

93. 12:07

    Uh, let me go back to the doc and just see where we're at for

94. 12:13

    moving things along. This will give them a little time to, to do that.

95. 12:23

    Update Hyper category, set up local environment. Um, okay. So once we, once we have that, we're gonna wanna clone it, right? So you can Git clone it.

96. 12:33

    Um, I have already Git cloned that repo. Um, so if I open it in Visual Studio Code, I will be looking at something that looks like, like this. I'll just pause for just a few minutes to make sure that everybody can clone the starter template.

97. 12:51

    You're gonna see me work with it for a little bit too, so if you're a little behind, that's fine. Um, you can always, um, catch up later or just, uh, cont- continue to watch.

98. 12:59

    Um, I will point out that, um, you may see some things in red even though everything is fine, and I'm gonna just show you how to get rid of that.

99. 13:07

    When you first clone the repo, um, you wanna s-- um, CD into the functions directory, and you're gonna do an npm install. That's gonna go download all the dependencies off of, um, the Node Package Manager.

100. 13:20

     And notice they're still in red. There is a little quirk in VS Code that we haven't quite figured out how to avoid. At this stage, you can, um, you need to open a TypeScript file.

101. 13:29

     But you can, you can do a few different things. You can just restart the TypeScript language server, and it-- the red will go away. Um, this is a, a VS Code issue that, that I've got an open issue on with that team.

102. 13:40

     But, um, it does install correctly.

103. 13:42

     Yeah, two questions here. So when you're on Just Ship AI, should you deploy on this, um, with GitHub? Can you reassure me on that?

104. 13:50

     Sure. The question was if you are on Just Ship AI, should you deploy or just go to GitHub? Um, if you view it on GitHub, you're just gonna see the template.

105. 13:58

     And GitHub will also let you just use this template directly. It's up in the upper right-hand corner. But, um, you can go through that step, but it's a little easier if you just go through the deploy, and we'll automate that for you.

106. 14:08

     'Cause we're not-- we're also installing the Hypermode application. If you go through the, uh, through the green button on GitHub, then you'll have to import your project into Hypermode after the fact.

107. 14:20

     If you click on the deploy button-

108. 14:21

     Yeah, if you click on the deploy, that's creating your org and everything on, on Hypermode.

109. 14:23

     Yeah. [background chatter]

110. 14:28

     Jay, can you take a look at what they're s- [chuckles]

111. 14:32

     We're g- s-- If, if people are having difficulties, we're, we're gonna send you somebody over, and hopefully it's not everybody in the room. [chuckles]

112. 14:41

     But yes, please, we can, we can make this interactive within, uh, within certain time constraints. Um, there's nobody in the room after us, but we know you have other sessions to go to also, so we wanna keep this-

113. 14:50

     Yep

114. 14:50

     ... on track. Shall I continue?

115. 14:56

     Yep.

116. 14:57

     Okay. So we've installed the project. I've installed the dependencies. Let's, let's build it. So we're gonna use, uh, npm here. If you're used to Yarn or, or Bun or any pnpm, those should all work.

117. 15:11

     Um, but I'm gonna do npm run build.

118. 15:13

     And you're gonna get some output that has our, uh, nice little pretty, um, ASCII art that I'm wearing right here. And it's gonna give you some metadata about your library, and it's gonna describe that there, hey, there are two functions in this, um, in this starter template.

119. 15:29

     There's one called Classify Issue, and there's another one called Trend Summary. Um, so together, we're gonna take a look at those two functions, and so we can understand how it works and, and what, what we're actually doing here.

120. 15:39

     And then I'm gonna actually run some of these for you so you can see, like, them in action.

121. 15:44

     Um, so let's just go look at... I'm actually gonna take them backwards order here. They're listed alphabetically. But we're gonna look at the trend summary first. Um,

122. 15:53

     this particular one, if we look at this, um... I'll make one other quick call-out. So, um, you may, if you're a web developer at all, you may see this and say, "Oh, we're in a TS file, and it looks like we're writing TypeScript."

123. 16:06

     We're actually writing something called AssemblyScript, which is a TypeScript-like language that uses a lot of the, um, the same features of TypeScript and a lot of the same ecosystem as TypeScript.

124. 16:15

     Um, the reason we're using AssemblyScript is because all of your Hypermode functions are getting compiled to WebAssembly, and that's how we actually execute all of this code that you're writing in a nice, secure, fast, uh, performant manner, um, as it's running and part of the Hypermode platform.

125. 16:28

     Um, won't dwell on that too much, but that's, uh, that's the, the intent here. Um, Hypermode will get support for other languages in the future. This is what we're using today. [chuckles]

126. 16:38

     Um, so what are we, uh, using from here? We're us- we're gonna use, uh, for this first one, we're gonna use, um, OpenAI, and you'll see that we import an OpenAI chat model and a couple system message and user message objects so we know how to use it.

127. 16:51

     I've got another, uh, little helper function that I wrote here. Let me just close that. Um,

128. 16:58

     that is going to go get some issues from GitHub. And if I drill into that, j- this is not AI at all. This is just a, an HTTP REST call.

129. 17:05

     GitHub has a nice API that you can use to go fetch data, and I'm gonna fetch data, um, pass in some parameters into a template. I can even do a little logging so I can see what's going on.

130. 17:16

     Um, requires some headers. Um, there's a little comment here that, like, we have, we have a little secrets management system, so you don't have to put secrets in your code, and we can still pass your, your GitHub API tokens through securely and so forth.

131. 17:28

     Um, ultimately, you get your data. We come back and say, "Hey, there's some issue data. We're gonna throw it in an issues object so we can start working with it in the rest of your code."

132. 17:36

     And that's what you're getting back here in this issues. Let me hover over. It's an issues object. It's an array of issues, right? 'Cause it said, "Hey, go get some giss-- GitHub issues."

133. 17:46

     So once we have the issues, what are we gonna do next? Um, we're gonna, we're gonna need to build a prompt to send off for analysis. 'Cause what we want here is we're gonna take a whole bunch of information from the issues, and we're gonna say, "Hey, uh, OpenAI," in this case, uh, we're gonna use GPT-4.0, um,

134. 18:02

     "would you summarize these for us?" Uh, what's the overall trend is the, is what we're asking for. Um, so it's not just summarize the issues, but like some-- "Give me a trend."

135. 18:10

     So that's part of my, my instruction off to there. "Provide a, provide a summary of the trends of their repository, repository based on the issues created." And, um, and we're gonna just send OpenAI, like a constructed string that includes like the timestamp, um, the user handle so it can kind of get some analysis, um, and like the

136. 18:29

     issue title and, uh, and we're, we're passing all that through into, um, OpenAI. The other thing I'll, I'll call out for you here is like now we're into some, um, Hypermode specific code that we call our model interface.

137. 18:43

     And, uh, this model that I'm looking at, it is an OpenAI chat model, so it ha- it knows everything about the OpenAI API. You don't have to go over and read the OpenAI docs in order to construct this, this, uh, API call.

138. 18:54

     You just say, "Hey, I'm going to use the OpenAI chat model," and when I create input, it's telling me right here on the IntelliSense and the type ahead, I can just say create some messages.

139. 19:03

     It's gonna take some messages. I've got a bunch of different messages of different types. Um, even goes as far as like optional parameters. Here I show temperature. Um, so a lot of people when they're brand new to AI they're like they never heard this term before.

140. 19:14

     What is, what is temperature? I don't know what that is. And then you see top P and like people are like I don't, I don't get that. Um, it's really intuitive for a developer to just say, you know, um, I can just say input dot and I get like here are all the different options.

141. 19:27

     There's like log probs and N and parallel tool calls and all the different options that GitHub has, uh, or sorry the OpenAI has. And, um, you know, I can read them individually.

142. 19:37

     Uh, if I use them incorrectly it won't compile. Um, all that, that kind of stuff that you would expect from, um, a strongly typed client library and we've, we've baked that into Hypermode.

143. 19:47

     Not just for OpenAI, but for other models as well. We've got some out of the box and we're gonna be expanding on them and we've designed the system so that if we don't support a certain model's interface you can write it yourself.

144. 19:58

     There's nothing that, uh, that prevents you from doing that. You can just inherit from a base class and, and write the implementation.

145. 20:06

     I'll pause there for a second because I feel like that was a lot of words. [laughs]

146. 20:10

     This is a pretty basic RAG use case, right? We're fetching some data, we're passing it to the model to ask for summarization. You could do this a lot of ways, right?

147. 20:17

     At Hypermode we make it really easy to stitch all those pieces together, right? And you can start to iterate through that, right? To Matt's point, you can try new models, right?

148. 20:24

     And you don't have to go learn a new model and understand a different model uses temperature in a different way or has different thresholds on it. Right here when you invoke a new model, do input.temperature, you're gonna see that right there in context.

149. 20:34

     And so we find that it makes it a lot more productive for users as they're starting to stitch these things together, not having to make any of these choices and really kind of hard choices up front knowing that you can always swap them out really easily.

150. 20:45

     One of the things we found when we were designing this is that um, um, a lot of the, the use cases seem to believe that all models were interchangeable and we found just through testing that like it's just not the case.

151. 20:56

     Um, not only are our prompts different and outputs are different, but also like um, the different hyperparameters are different, the different, um, ways you can control the APIs are different.

152. 21:04

     Like uh, everybody's got their own little custom snowflake tweaks and we needed a way to consolidate that, that so that a user can very easily get to all the different options that are available.

153. 21:14

     So this is the API we came up with and I'd love feedback about that for any of our like API design engineers that are in the room. [laughs] Come talk to me after.

154. 21:22

     Um, all right, so git-- uh, OpenAI returns me some complex things. It's there, there is the ability in OpenAI to have multiple choices. We're just gonna take the first one and we're gonna take that message and this may seem like a little thing but like, um, once I get the content of that string I, I could do

155. 21:38

     other things with it. I could-- in this case all I'm gonna do is just trim it because like sometimes it comes back with extra characters. But say I wanted to throw it in a database, or I wanted to log it, or I wanted to go call another model without input, um, as soon as I say that to

156. 21:50

     people they're like, "Oh, like Langchain." Like well kind of it's just, it's just programming. It's just like you're taking in one input and you're, you manipulating it to do the next thing.

157. 21:58

     Um, so let's, let's just run this. So what does that mean when we run it? Um, in this case it's already in Hypermode because I cloned from the template.

158. 22:05

     But say I'd just written this, all I'd have to do and this is the, where the title comes from is just git commit, git push. Uh, and we pick it up, um, automatically and roll that out into your Hypermode console which I'll go back to.

159. 22:18

     This is the one we're looking at. Um, and um, you'll see that I've got trend summary on here. Um, now I've already configured this, um, back end to have an API key off to OpenAI which you would need.

160. 22:30

     Um, but if you were doing it yourself you would go over to host and you would need a GitHub token as well because we're gonna go pull issues off GitHub, right?

161. 22:38

     Um, so let's run this. We got a little query tab here that uses GraphiQL. It's a, it's a built-in thing. We may, we may replace that with something a little more, uh, interactive later but it, it works for, for now.

162. 22:48

     And um, I use Hugging Face transformers but let's-- uh, somebody have, somebody throw out a favorite repository. I need the, uh, the org name and the... Anybody have another one?

163. 22:59

     Anyone? Anyone? I'll pick something at random. Okay, I used to be, for a long time I was a C# developer. Um, don't hold that against me. So I'm gonna go, let's go see what's going on in the .NET runtime and, and I haven't tested this so we'll just see what happens here.

164. 23:15

     It's gonna go fetch the top hundred issues off of .NET runtime repo and within a few seconds here it has summarized it and said, "Hey, there's the trend summary."

165. 23:24

     Um, and you notice, um, we made a GraphQL call, um, but I didn't have to write any GraphQL schema. Um, Hypermode automatically generates the GraphQL schema based on the functions that you export.

166. 23:35

     So all I had to do was make this function trend summary and tell it what its inputs are and it now has a full lit up GraphQL endpoint. Um, I happen to be calling it from the Hypermode console but you could use another tool like Postman, uh, has a really good GraphQL interface or if you use any

167. 23:50

     GraphQL client where you want or you could just use Curl, um, on the command line and write into a script or your favorite HTTP client. You could use anything you want.

168. 23:59

     Um-

169. 23:59

     Matt, we've got a question.

170. 24:00

     Yeah. Question. Um, will it create GraphQL types from inferred from the TypeScript types? It does. The quest-- Yeah, uh, I'll repeat the question. It says, um, does, does Hypermode create the GraphQL types from inferred from the TypeScript types?

171. 24:16

     It does. Um, that is part of the Hypermode experience. Uh, it will even create custom scalars for you if you have those. Uh, there are some limitations but we are, we are actively engineering to try to overcome them.

172. 24:28

     The biggest limitation you'll see right away is at the moment, um, input types are, are not supported. Uh, but you can have structured output types. So input types at the moment have to be all scalars, um, or arrays.

173. 24:38

     But um- Say you have your custom object in GraphQL, we'll let you supply that. But we, we need to do some engineering work to make that work. [laughs]

174. 24:48

     Great.

175. 24:48

     Yeah. Um, it's a really nice way to, like, um, define your code once. Um, and you have a single source of truth, and it can be in a file you import.

176. 24:58

     If you're in a corporation that has lots of libraries, you can, you can make separate libraries for this, and it's just, it's just the MPM package ecosystem. So it just folds in like you would expect.

177. 25:06

     When we support other languages, it'll be whatever the package ecosystem is for those languages. We're not trying to do anything special here.

178. 25:12

     So Matt, here, why don't we show what you get as well when you run that query, right? So if you go to the inferences tab,

179. 25:19

     you'll see actually a history of this model run, right? 'Cause we know that when you start to actually run models, you need more observability into that, right? Your app may be triggering things from the front end, dynamically pr- uh, prompting that model.

180. 25:29

     So we make it really easy to see every time you hit a model, you're gonna get one of these entries to understand how long it took, what was the input, and what was the output, right?

181. 25:37

     And so you can start to debug this with, with additional visibility, and we find that makes it really easy, especially if you're starting to chain multiple models together in a single function.

182. 25:44

     Yeah, like if I wanted to know, well, how many tokens did, um, OpenAI consume for that? It did return that to me, but I didn't use it in my code.

183. 25:51

     I could have. But I... Even though I didn't use it in my code, I do have it in our inference history, so you can go back and, and say, "Oh, that, that prompt, you know, took three thousand tokens."

184. 25:59

     Yep. And then you also have access to, to raw logs of, of the functions as they run, right? So you can log things out to that. You'll see system logs a- a part of that, and they're all kind of, uh, oriented based on each individual function execution, right?

185. 26:14

     So you can have high volumes of overlapping functions. Makes it really easy to debug here to understand what's going on, how models are being invoked, what the specific inputs and outputs of those are-

186. 26:22

     Yeah

187. 26:22

     ... so that you can continue to iterate and understand what's going on.

188. 26:25

     This is the one I just ran. Um, these are some ones I was running earlier, but we'll, we'll show you those in a minute. Um, there's a whole bunch of stuff, other stuff on here.

189. 26:32

     One of the interesting one is like deployments. Um, I've only done one on this repo. This is the initial commit. But as we push, you'll see that, like you have the Git hash, and you can click to link over there, and it makes it real easy to like, see who changed what over time.

190. 26:44

     And, um, okay. Um, let's go on to... So this one, we're not gonna, um, have you guys implement yourself because I don't want anybody in here to, to have to supply an OpenAI key or GitHub token right now.

191. 26:58

     But like I said, they, they are in the starter template and workshop. You can do that on your own time, however you like. We're gonna go on to the next one, which I think everybody can do here together 'cause, uh, we're going to actually run the model on Hypermode rather than, uh, externally.

192. 27:11

     Yep.

193. 27:12

     Is that the right spot in the sequence? I think so.

194. 27:14

     Yep.

195. 27:14

     Okay.

196. 27:15

     Yeah.

197. 27:16

     Bear with us if, uh... We're just going through presentation, uh, coordination here. Um, okay. So the next one we're gonna look at, like I said, there's two functions that are in, in this, um, starter template.

198. 27:27

     Um, the next one is classify issue, and we're gonna, we're gonna use this at first, and then I'm gonna have you guys modify it. We're gonna actually put your code to work here.

199. 27:34

     So if you've never written AssemblyScript or anything before, just pretend it's TypeScript. If you run into a thing where like it says, "Hey, I don't know what type that is," that's, that's where you get into AssemblyScript.

200. 27:41

     It's in there. I can answer those questions, but it's pretty straightforward. Um, uh, so, uh, what does this function do? Well, w- in this case... So this is gonna be a little disconnected from the last one we did, but, uh, because it's not gonna go query GitHub.

201. 27:54

     It's just gonna take as inputs the title and the description and an issue ID. And we're gonna, we're gonna log some information on what we're doing, and we're gonna s- we're gonna just build a really dumb, uh, prompt here of, um, or input anyway, of summarizing the, uh, the title and the description concatenated together.

202. 28:12

     Um, in a real app, you'd wanna play with this a little bit and figure out what the right exact input that matches the model you're using. Uh, oh, let's talk about the model.

203. 28:19

     Um, so this model is, uh, called DistilBERT MNLI GitHub Issues. We found it on Hugging Face. Um, this is a, a, a con- con- community contributed model, I believe.

204. 28:30

     Yep.

205. 28:30

     Um, but it's pre-trained on GitHub Issues, so it's a purpose-built model. We don't need to spend time model training here. We don't need to start with anything big. This is a s- a really small model.

206. 28:38

     Um, it's, it's good for like just deciding whether a GitHub issue is, uh... I believe it's, it tells you whether it's an issue, a bug, a feature request.

207. 28:46

     Or a question.

208. 28:46

     Um, or a question.

209. 28:47

     Yep.

210. 28:47

     Yeah. That's what it... so it's gonna do. Um, the interface for this, uh, you'll see we import classification models from our models.js library. Um, this word experimental, don't be scared off by that.

211. 28:58

     It's just we're trying to make sure that, um, we are careful about when we, uh, finally call it not experimental. Um, but this will match most of the classification models that are on Hugging Face at the moment.

212. 29:09

     Um, and, uh, yeah. So what are we gonna do? We're gonna, we're gonna build a prompt. We're gonna say, "Hey, get that classification model." And now I've got, uh, an input that in this case requires, um, one or more input strings as an array.

213. 29:23

     And the reason it's one or more is 'cause you can classify things in batches. I could say, "Hey, there's are, there are ten GitHub issues I would like you to classify."

214. 29:29

     I only need to make one model call for that. Um, and, and, and that's why I've got prediction zero, the first prediction coming back, 'cause I'm only asking for one.

215. 29:38

     But, um, those would match the, the inputs and outputs.

216. 29:42

     And again, if there were other options here, you could say input dot. In this case, it's the only one. Um, but we would wrap that up for you in the interface.

217. 29:51

     And then we'll just log output. So this, this is really simple. Um, again, I've already got it running, so let's just run it. And, um, the queries for these, by the way, are, are pasted into the docs.

218. 30:00

     If you're following along in the docs, let me just make sure I show you where we are. We are at issue type classification, and there's some description there. But basically, this is the code we're working on.

219. 30:11

     Here's the example query we're gonna run. I'm gonna paste this query. Um, and this, this query's got a lot of, um, what looks like junk in it, but it's because we just took literally the entire text of the query, um, off of GitHub, um, and made sure it was properly escaped and stuff.

220. 30:26

     So if you were using the GitHub API, it would work. Um, if you're literally like with your mouse copying and pasting text, you may find, um, like getting the new lines in there and the quotes escaped might just be a little tricky, uh, on the fly.

221. 30:39

     But if you had input from an app, it would be doing that for you. It's just, it's a string in and a string out, right?

222. 30:44

     Um, but let's go run that guy Where am I? Make sure I'm in the right area. I think I've already got it in here. Yeah, I'll just paste it anyway, just make sure.

223. 30:54

     Is that the one? It didn't paste correctly. There might be a docs issue there. This one I know works. Um,

224. 31:04

     maybe something I need to fix in the docs. I don't know if Jay can fix that. The, uh, the one that's in the docs isn't, uh, escaped properly, so when I copy and paste it, it's, it's got new lines in there.

225. 31:15

     Weird. [laughs] Um, but basically everything I'm putting in should be in the description. If, if you do get, um... If you're trying this out and you get some weird error like, like this doesn't work, um, just reduce it down to s- to some string in here just so you can, you can follow along.

226. 31:31

     Um, we'll make sure we clean up the example later. Ideally, it would be the entire thing of the issue. Um, indeed this is the, the classification category that came out 'cause it's a real issue.

227. 31:41

     Um-

228. 31:42

     So, so what's, what's happening here, just to kind of highlight some of the contrast here, right? So when you deployed this project, this small model, different than kind of OpenAI's models, right?

229. 31:51

     Uh, this small model actually got provisioned automatically in Hypermode. So when you're running this query, you're running against a dedicated instance of this model just for your project, and you can start to customize and, and adjust that, right?

230. 32:00

     And so really wanted to show you these two functions to kind of show you the contrast, but also the similarities, right? The model, uh, interfaces are relatively similar and pretty intuitive to make it easy to kind of continue to iterate through that.

231. 32:11

     Mm-hmm.

232. 32:11

     Why don't we jump ahead slightly just in the interest of time-

233. 32:13

     Sure. How are we on time?

234. 32:15

     And actually start-

235. 32:15

     Okay. Yeah, I see

236. 32:15

     ... start building, um, a vector s- or natural language search.

237. 32:19

     Well, one, one thing I would like to do is just give people a few minutes to do some-

238. 32:24

     We don't have time.

239. 32:24

     We don't have time. [laughs] Okay.

240. 32:25

     Sorry. Um-

241. 32:27

     Uh-

242. 32:27

     Your, your projects will continue to run. If you could reference the docs, you're welcome to continue to iterate with this. We can spend all the time with you after to, if, if you'd like to, but-

243. 32:34

     We won't do it, but the suggestion in the docs, and there's a walkthrough, is like how would I add a threshold to this function and say, "Hey, I, I, I need to know based on its confidence, like how, how accurate is this label?"

244. 32:46

     Um, and that's something you don't need to train a model on. It's just code. You just write it. There's a, there's a, um, instead of just that, there's confidence score.

245. 32:53

     So you can decide what the confidence score you want is and, and do some comparisons and just write that code.

246. 33:00

     But we'll let, we'll let people on their own.

247. 33:02

     But, yep. So why don't we jump ahead to building a natural language search to identify similar issues, right? Oftentimes people report repeated issues, and wouldn't it be great if there could be a bot that responds to every issue saying, "That sounds a lot like these three other issues"?

248. 33:16

     That's what we're gonna build in the next twelve minutes.

249. 33:18

     Okay. [laughs] Sorry for the rush, guys. Um, like I said, uh, there's nobody in the room after. We'll be around, so if you wanna walk through it slower, if you have questions-

250. 33:27

     Yep

251. 33:27

     ... we will help you. Uh, uh, okay, so we're gonna go, we're gonna start writing some new stuff. Let's, uh, let's just take this directly from the docs so that we can walk through it together.

252. 33:37

     The first thing we're gonna do is we're gonna create what we call a Hypermode collection. Um, and basically, it's like where are you gonna store your data? Um, Hypermode isn't a database, but we give you a very simple way to have an in-memory key value store that we can apply an index to.

253. 33:52

     Um, and that way we can, um, basically do your vector search in, in memory, in Hypermode. Um, so first we have to go to our manifest and say, "Hey, there's, um, there is a collection."

254. 34:02

     So far we haven't talked about the manifest, but you'll see in the root there's a file called hypermode.json, and it has models. It has the two models we've been using so far.

255. 34:09

     It has hosts, which were the, uh, both for the, um, the, the first example. Um, that's how we get out to other, other places, and I'm going to add that collection.

256. 34:19

     So I'm just gonna drop it right in here. Um, so we have our collection file. That's all we have to do. Um, and there's some additional, uh, options and stuff that we're not gonna go through them all right now.

257. 34:31

     Um, the next thing we're gonna do is we're gonna add another model. So one of the things with, um, if we're gonna do a vector search, we need to know, well, what model are we creating our embeddings off of, right?

258. 34:41

     Um, and we don't want you to think about that too much, but you do have to, um, supply a model. So we're gonna take another one from Hugging Face, and we're gonna go add it to the model section.

259. 34:53

     Boom. We're gonna use MiniLM, uh, from Sentence Transformers.

260. 34:59

     And, uh, and you'll see that syntax in the, in the documentation, but basically it's just that's the source of the model we're using. It's gonna be hosted on Hypermode, and we got that from Hugging Face.

261. 35:08

     That's it. Save that. And, uh, I'm gonna do a, um,

262. 35:16

     I'm gonna commit that for a minute. So let's do command line, Git add everything, and Git commit,

263. 35:27

     um, um, add collection and model. Okay. And then it's... We've got nice Git signing on, so it's asking me to authenticate. But once that's in, I can Git push, and just from the push, it's gonna start spinning up that model in Hypermode.

264. 35:47

     Now, you didn't have to do that at this step. You could do the code and then do it all at once, but this just gives it time to warm up a little bit since we're, uh, under a time constraint.

265. 35:55

     Um, the next thing I'm gonna do is I'm going to, uh, go grab the embedder function. So we need an embedder function. Um, you know, you know how that...

266. 36:06

     We, we showed you the model interface a couple times. Well, this time we're gonna use an embeddings model. And I'm gonna go create a new file.

267. 36:14

     New file, embedder.ts, and I'm gonna paste that in here.

268. 36:21

     And, uh, I won't walk the whole thing, but basically, um, embedders need a text in and an array of vectors out, and the vectors are float thirty-two arrays. So it looks like a two-dimensional array, but it's an array of array of float thirty-twos.

269. 36:35

     Um, and that's an AssemblyScript thing that we have a concept of a float thirty-two instead of just a number. Um, but that's basically how that works, and we're gonna create it, and we're gonna just return all the predictions.

270. 36:45

     Um, one thing I have to do also is I have to, um... The way that this is, these exports work is, uh, I have to go to my index file and actually just make sure I include it

271. 36:54

     Otherwise, it's not gonna leave the module when I build. Um, at this stage, I should be able to npm... Oh, I have to go to the functions directory, and I can say npm run build, and we should see it.

272. 37:07

     Yeah. The embedder is, is in the output. It's still not searchable though. Like this, if I were to run it at this stage, you'd see the array of all those floats, and it's kind of not very usable from a, a user's perspective.

273. 37:21

     Um, so let's grab the actual function that's gonna do the search, and that's also in the docs. So I'm just gonna copy that in,

274. 37:29

     and I'm gonna do the same sort of thing. I'm gonna come here and say new file, um, search, we'll call it. It can be called anything. I'm gonna paste that in, and be- and before I show it to you, I'm gonna make sure I don't forget to export it.

275. 37:41

     Sometimes I forget to, uh... Not from GitHub. We're gonna from search.

276. 37:47

     Okay. Um, and I know we're tight on time, but just real quick, what does this look like?

277. 37:53

     Um, it's gonna do a few things. Um, it's gonna define, uh... It's gonna import our collections object. It's gonna define a class that's gonna be, um, for our similar issues.

278. 38:05

     This is, you're asking that can we return structured objects? Yes, I'm doing it right here. I return a similar issues array, and we automatically get GraphQL generated in our, our output schema.

279. 38:14

     Um, I'm going to search the collection using the information we have in the manifest, and this time I'm gonna say, "Just give me the top three," and I do want the text and the results.

280. 38:23

     Um, often you don't. Often you just say, "Give me the ID back," 'cause I'm gonna go query a database using that ID. And so for you to return me the text, if it's big, could be an extra step we don't, you don't need to do.

281. 38:33

     Um, that's it. Return them. Uh, return me an object using the, the output of that, that search. Let's, uh, let's do the git push and, um, git add dot, git commit dash and, um, add search functions,

282. 38:54

     and git push. And, um, oh, I didn't do-- I probably should have made sure it builds locally. Um, it's going to build it in, in the... Oh, I'm not in the right directory.

283. 39:05

     Uh, functions, npm run build. Because if I had a compile error or something, it would happen here, and we are working on improving our local dev experience as well, 'cause ideally, we'd like you to be able to run it here too, but we don't have that just yet.

284. 39:17

     So for now, you, you just push it, and we run it there. Um, notice it says, "Hey, there's a custom data type similar issue." That's gonna be in our new GraphQL.

285. 39:26

     We'll go back to Hypermode, and how are we on time?

286. 39:29

     Good.

287. 39:29

     Eight minutes. Okay. Thanks for hanging with me here. Let's look at deployments for a second. Notice they're there already. So those are the two pushes that I did. Git push, it just works.

288. 39:39

     Uh, I can now go to the homepage, and I should see, hey, there's an issues collection, and I've got four functions, and I've got that new model that I added.

289. 39:48

     Right. So every- everything we just did, it's already in Hypermode. Um, one thing I wanna do just, uh, for sake of time is I'm gonna go, um, pre-seed this, um, collection with some existing data, and I've got a link to that in the...

290. 40:02

     Not there. In the doc. Down at the bottom, there is a link. Here, example CSV file. So we're gonna take these issues which we scraped earlier off of, I think, Transformers.

291. 40:14

     Yes.

292. 40:14

     I think that is where we grabbed it from. And I'm just going to download that file from GitHub issues.csv, and I'm gonna go over to Hypermode and, um, and upload them there.

293. 40:25

     Now, you don't have to do it this way. You could write yourself a function that says, um, "Hey, take this other API input and go upload it," or you could call GitHub API and upload it that way.

294. 40:34

     So there's a lot of different ways you can do it, but we're gonna, we're gonna put it in manually right now.

295. 40:40

     There it is.

296. 40:41

     And what's gonna happen when Matt does this is because of the way that we've set up the collection and the embedding function, every time you add something new to the, the collection, whether that's via your code or via, via this upload, it's gonna automatically embed it, right?

297. 40:52

     It's gonna use that function, automatically embed it, and make it available for your search. It's gonna use that same embedding function in the search so that you know that what, what you're searching and what you're searching against have it embedded, embedded the same way, right?

298. 41:04

     So you don't need to bring three different systems together to get this working. All in one box, you get natural language search off of these couple of functions.

299. 41:12

     There's a little bit of a delay as it's working here, but if I just be patient for a minute and refresh, eventually we'll see the, uh, inference history from those embedding functions should pop up.

300. 41:23

     Yep. Those are processing in the background there.

301. 41:25

     Here's to the dem- for the demo gods. [laughs]

302. 41:27

     These are the ones that we ran when we were classifying what we should see here. Let's just pick the model. Why do I not see that model? Shouldn't I see the embedder?

303. 41:36

     Yeah. Hmm. Do the functions runs work? Let me, let me just try a search and see what happens. I think-

304. 41:52

     It's probably still queuing up.

305. 41:53

     All right. I'm gonna search... We'll see if it's still going. I think, I think we're getting no results here, or we can, we can just wait a little more.

306. 41:59

     But, um, we're going to see if, hey, is anybody... We're gonna search a new issue. Like imagine the workflow here. Somebody writes an issue, says, "Hey, uh, I wanna add a Spanish, uh, version of your README file."

307. 42:09

     And so our app that's using these APIs is gonna be like a bot that says, "Hey, uh, we, we may already have that already. You know, maybe other people have asked."

308. 42:17

     Um, and indeed... Okay. So it did finish uploading from the collection. Um, we can go figure out why it's not showing in the logs. It should have showed in our logs.

309. 42:25

     Um, but indeed, we can say, "Hey, um, there was a Turkish README, and there's some other bits about languages and a French version of this, but I don't see anything with the Spanish README, so it's probably okay."

310. 42:35

     Um, but there aren't any similar issues. Uh, these are the most similar, um, that are available.

311. 42:42

     I am curious why that didn't come through. There they are. Just took a second.

312. 42:46

     Yep.

313. 42:46

     Okay. We'll work on that. Um, these are batches, uh, by the way, so they're a little long here, but I think what we did is we split them up to like twenty-five at a time, and so they take like, you know, two seconds for that one and one second for that one.

314. 42:57

     Um, but we can see the, the history- And like there's the stuff you really don't wanna look at, all of the, uh, the float arrays, right? But that's your vector embeddings.

315. 43:06

     And then the search just uses that vector embeddings. That's it.

316. 43:11

     I think that's-- I know we kinda rushed through that. I'm sorry. Um, hopefully some of you got a chance to code some of it, but like I said, um, uh, feel free to stick around or to continue to walk through the, the doc on your own time.

317. 43:24

     And I'm really ex-excited to see what else you guys want to build. Uh, we have a lot of other documentation off to the side here that you can see, like other stuff we can do.

318. 43:31

     You can make HTTP calls. Um, you can, you can, uh... What else can you do? You can work with the collections more, uh, shortly. Not yet, but shortly we'll have ability to go and ran- and connect to any Postgres database that you want to.

319. 43:44

     Uh, that's coming soon. Um, and, uh, and yeah, there's like the sky's the limit, right? Pick your models and write your functions and away you go.

320. 43:55

     Anything else we wanna show?

321. 43:56

     Yeah, no, that's, that's what we had to show you, right? It was like these are a set of building blocks that you can apply in your applications across contexts, across different points of data.

322. 44:03

     Hopefully, you start to understand how these pieces work, right? We see this everywhere. Like when we start to look at around and say, there are so many places where you wanna do that natural language sorting or search or bringing these pieces in, and we just need people like you to help bring that into those applications, right?

323. 44:18

     And so that's what we really get excited about. We don't want you to have to start from scratch. So that's, as Matt said, we launched Jetship AI. Those are templates.

324. 44:25

     You can deploy those right away. You all have access to Hypermode now. You got a fourteen-day trial. Feel free to respond to that email, and you can claim the credits that Kevin mentioned this morning.

325. 44:34

     Happy to get your team set up, um, and run through this in a more contextual way for you as useful. But thank you. [outro music]