Form factors for your new AI coworkers

Talk transcript

1. 0:00

   [upbeat music] I, I think we've all noticed tools like v0, uh, getting pretty good at, at generative UI and creating, um, good-looking things, as well as Claude Code,

2. 0:25

   um, being able to let us run things more complicated locally and build on those things. Um, so I think the, the thing that comes out of this is designers, um, product people, and engineers all building together, and I'm really excited about that 'cause I've never loved the, the divides, um, between these things.

3. 0:43

   Um, so this really lets us get rid of, um, in my mind, get rid of mock-ups, get rid of the click-through prototypes, um, and, uh, all the hand-wringing about whether, uh, the thing that we're building, um, is worth the engineering effort.

4. 0:58

   Um, so as we, as we go into this, it's time for us to jump in, um, and feel the material that we're working with and see what emerges. So I'll give you a super quick overview of, of Flatfile's, um, AI, uh, stack.

5. 1:10

   This is not an official diagram, but it's how I see it. Um, we migrate data, um, big... If, if you needed to move a lot of data between systems, um, frequently, uh, you use our developer platform.

6. 1:22

   And since we're a developer platform, LLMs are good at writing code, makes it the perfect place for a lot of AI. Um, at the bottom here, we have our, our customer's Flatfile applications that they, they deploy to our infrastructure.

7. 1:34

   Um, then there's this, like, real-time context, which is the data and the validation outcomes. So what are the errors and warnings and things that are in that, that data, that dirty data.

8. 1:44

   Um, and then our AI agents, the tools they have, um, and the jobs that they can run. Um, and then what gets shown to users. So I see it as four buckets here.

9. 1:53

   There's more. Um, there's invisible, so it's kinda like the ghost in the machine, almost called, called that ghost. Um, ambient, so i-it's kind of happening in the space, but you're not directly working with it.

10. 2:05

    Um, inline, so it's actually in your work, uh, in your workflow. And then conversational, the ones that we're, um, I guess all arguing about, and I think that's what I learned, um, being here at this conference.

11. 2:16

    Um, here's an example of invisible. So when you start, uh, if you sign up for a Flatfile, we go in the background, we, we take your email address, we find the company you work for, we look it up.

12. 2:26

    Um, and in the background, the AI agents are writing a Flatfile application. So they're writing code, and essentially sending you up a demo that is perfect for your use case.

13. 2:35

    So if you come in, you're from a HR company, you're gonna get an HR, um, demo. Um, and while that's running, you don't need to know that AI is working on it.

14. 2:43

    Um, so that I'd say is like it's working in the background. Here's something working more ambiently. It's a very f- initial, uh, take on this, but you can see there's something-- an agent analyzing the data in the background.

15. 2:54

    Um, this is a tool I actually, um, I lead this team for AI transformation, and, um, you can see the little sparkles pop up on the columns when it finds, um, opportunities to fix it.

16. 3:05

    So that's ambient. Um, this is inline. So you're busy working in the data, um, and the AI is able-- you're able to use the AI, um, directly in line here, um, to fix the data.

17. 3:17

    These agents are, are writing code that then gets run on this data set. So you could have a million rows, um, and 50 columns or whatever you want, and, and that code will run really fast, um, which is pretty cool.

18. 3:29

    And then finally, the conversational ones we're all used to. So this is build mode. It's the no-code, low-code, um, agentic system that writes Flatfile apps now. So before, you would probably have to have had a engineer at the company, um, building these applications.

19. 3:44

    Um, now it can all be built up. So that's pretty cool. Um, and that's, that's kind of the, the general surfaces I think about. Um, I listened to, um, Amanda Askell, um, from Anthropic, talking to Lex Fridman about, um, building, um, Claude's character.

20. 4:02

    And in that moment, I realized I'd been doing something a little silly. I'd been giving engineers feedback on our agents like, "Oh, it shouldn't start saying this," and, "It shouldn't use these words," and, "Why should it do this?"

21. 4:13

    And I realized I was, I was doing it like I would do design copy, right? I was, I was, um, in my, my, my normal instinct. And when I heard her talk, I realized I needed to go from controlling, um, to being a character coach and, and, and actually building out, um, the, the nature that I wanted.

22. 4:32

    So this is a v0. Uh, uh, I hope-- Have you... Most of you used v0 from Vercel before? Um, yeah. Um, so this is a v0 I built, one of my early ones, and it was, um...

23. 4:42

    I called it a chat tuner. It doesn't look like much, but that wasn't the focus. Um, but I could essentially, um, put our orchestrators, so the system prompt for our AI orchestrator for build mode, um, in here, and then I can modify it.

24. 4:55

    I can say, what is it like if I tell Claude to be more friendly versus more balanced versus more concise? What, what does more cautious mean to this model?

25. 5:05

    Um, and the point of me showing this is just to say, like, the design of the final thing is always a tempting thing to design to. Um, but now we can actually go and build tools to help us to design that.

26. 5:18

    Um, and this brings me to, like, uh, I have, like, three themes. Um, the first theme, which is feeling the material. I'm a woodworker, so you'll have to forgive the analogies to physical material.

27. 5:29

    But if you're going to, uh, design something with a physical material, you have to feel it, right? You have to... What are the properties of it? Um, and you need to understand it.

28. 5:39

    And so I feel like before with design, we were kind of looking at everything through, like, layers, right? Mock-ups and prototypes and, and kind of trying to see what was gonna work and what wasn't.

29. 5:50

    What we need to do now is go feel the material, feel, feel how these models work. Um, my new North Star is, like, creating an environment-

30. 6:00

    for these LLMs to shine, right? What's, what's this form factor that can help them nail their assignment, stay aligned, and grow as the models get better, right? That's, that's my new goal.

31. 6:11

    Um, we're... Basically anything we do with an LLM, I feel like we're putting it in a box. Um, and, uh, that's... You also hear people say that LLMs are like interns.

32. 6:21

    Like, "Oh, it's an intern with a PhD." And so I try think now, if you're putting an intern with a PhD in a box, like it better be a good box.

33. 6:29

    Um, and so we need to put effort in.

34. 6:32

    Um, this was a conversation we were having about what tools does this, uh, coworker, this new form factor, this new model, like what tools do we give it when it shows up for work?

35. 6:42

    And, uh, I got fixated on this idea of cursors. I was like, "Oh, what happens if it just had a mouse or a track pad?" I'm a track pad person, um, so, um, that's probably controversial.

36. 6:52

    But, uh, essentially what happens if we gave the AI, um, those tools? And so I, I created this v0 and laz- moved it into Cursor. Um, and I was like, well, I work in design tools a lot, so I don't migrate a lot of data, so this is the best place for me to feel this, right?

37. 7:07

    To feel this material. So I created a canvas, um, and I could give it orders and be like, "Hey..." And honestly, I was, I was very enthusiastic about this, um, for like a few seconds. [chuckles]

38. 7:18

    Um, it felt like I was touching the AGI a little bit. Um, but I also very quickly started feeling like I was putting a Formula One driver in a Prius.

39. 7:26

    It just, it felt like I was constraining it and controlling it. Um, it could only move one thing at a time. Um, but so, so learning from that, um, was, uh, something like this, which was also, um, a, a v0, um, that I used Claude Code on eventually.

40. 7:42

    And this is a new, uh, product that we're working on, which brings like the, all the stuff we've learnt about, um, migrating data to consumers to let them work on their data.

41. 7:53

    Um, but you can see the AI is, is operating in the space. Um, and it's, it's got presence. And so it's, it's able to read multiple files while writing into another one.

42. 8:03

    Um, it's not like me, who can only focus on one thing at a time. Even though I think I can focus on more, um, it's not true. And so this is us moving from determinism to inference and figuring out what this material feels like.

43. 8:18

    And so, um, that's feeling the material, right? Like working with the model, getting it into your space, understanding how it feels, um, to work alongside it, what's it capable of.

44. 8:30

    And then the form factors that we're putting on them. Actually, actually, you can now go build it and play with it, um, a- and feel it. Um, the next material analogy I have, which is finding the grain.

45. 8:40

    Um, once you've got the characteristics of the material, you understand it. Um, usually the piece of material that you're building with and you're creating with might have its own characteristics.

46. 8:49

    And so as we're creating these form factors, uh, finding the grain is about feeling it out. Where is it smooth and rough? Um, where is it weak? Where is it strong?

47. 8:58

    Um, and we'll have to remain humble here because, um, things are gonna change and are changing so quickly that whatever we, we build is gonna most likely need to be rebuilt.

48. 9:10

    Um, this was an example of that build mode agent. I asked it to do one thing, which was enable the Automap plugin. So this just automatically maps data from the source, um, data to the target data.

49. 9:21

    Um, and I get a wall of text. And it's not bad because this went and, and I-- saved me probably a week of work. Um, I didn't have to have a product manager write a PRD, send it to an engineer, get the, the, the, in the roadmap, get the engineer to write it, QA.

50. 9:34

    This was all just done, right? All that code was written. Um, but the noise gets in the way. And so this was a v0 of, of kind of rethinking the tool UX.

51. 9:44

    What could it be like? And so, um, the way I thought about this was if you're designing for a... If you're, if you're going to a coworker and you're gonna do something complicated for them and you wanna communicate, you think, "Okay, I'm gonna choose my words carefully.

52. 9:57

    I'm gonna communicate visually. Um, I'm gonna stop and check, um, whether, whether it's right." And so I wanted this to feel similar. And so you can see here, split personal details.

53. 10:07

    It's visually telling you what it's doing, saying, "Hey, is this right?" Then it's saying, "I'm aligned. I took a snapshot. You can roll back. I'm holding you accountable. You approved this."

54. 10:17

    Um, and then telling you what you can do next. We also want it to, it to be able to express itself, so if something went wrong, kind of shaking its head in a little bit of frustration, which is probably what the user is feeling too, um, when something goes wrong.

55. 10:29

    Um, and then finally it can back off, uh, when it gets something wrong and sort of say, "Okay, I'm handing control back over to you." Um, and that's a lot more intri- That feels a lot better, and it, and it felt like we'd kind of found the grain and, and found the right place to put this, this

56. 10:47

    material, um, with this. And so what's really cool about this one is that as we're implementing it, we've realized that it can, um, it can fit in other places.

57. 10:57

    So, uh, not just in conversational flow, it can fit inline. Um, and this is gonna be in our kind of like inline transform functionality, um, really soon.

58. 11:09

    So I, I think, like, as we, as we find a new technology and work with it, we run the risk of just automating the tedious things. And I was so excited about those previous two talks because there was kind of like some emergence in there, right?

59. 11:21

    Like something interesting that we wouldn't be able to do before. Um, and I'm most excited about those things. Like what, what emerges from, from playing? Um, we stopped playing for a few years, [chuckles] um, when we were kind of about the internet and we were like really excited and CSS3 came out and then like HTML5 and we were

60. 11:39

    playing a lot. Um, now I feel like we're all playing again, and so that's really exciting for me. Um, this is an example of me playing. Um, I created this, uh, v0, and I, I...

61. 11:52

    We've been in search of this characteristic of an agent that is, that feels forward-leaning. And what I mean by that is it's an agent that's curious and it's excitable, um, but it likes getting shit done, um, and it's very focused.

62. 12:05

    Um, so not going crazy, right? Like we've all seen the LLMs kind of go too far when you give it a task, and that doesn't feel good. Um, so here I dropped a JSON file and a CSV file, um, and the agent decided, um- You know what would be good to do is combine those two things because

63. 12:22

    the data looked pretty similar. Um, and so here we can see it's, it's combined the, the file in-- the two files into one. Um, that's a good thing that it did.

64. 12:31

    I didn't have to ask it to do that. Um, it picked up on it. Um, and then after that, it wrote a, a report. So it told us what it was doing.

65. 12:40

    It said, "Hey, I found some duplicates. This is probably what you need to do next." And so it built up context. Um, and I was actually just trying to play with Claude 4 here and feel the material and kinda see how it would be.

66. 12:52

    Um, but I realized, um, I'd kinda come across this nature that we were after. Um, it made some suggestions and generated a slide deck, which I, I asked it for.

67. 13:02

    Um, so within just dropping two files, um, it's, it's done something emergent. Um, and now we're baking this into our, our new product called Obvious, um, which is coming soon.

68. 13:15

    Another one was we had this idea of giving our agents a knowledge base. So all the customer calls we'd had with them were all recorded and transcribed, like most of ours are.

69. 13:24

    Um, and we had documentation from the customer, and so we put it in, um, to knowledge base, and then when we analyzed all of this customer data, we surfaced up, um, suggestions based off that.

70. 13:35

    I was fully expecting better suggestions. I was fully expecting more suggestions. Got those. But then here, um, the a- the agent decided, "I can't fix this, but I know how to fix it, and so I'm gonna tell you how to fix it."

71. 13:48

    And so it suggests here that the user actually goes to HR and gets them to generate the missing employee IDs. And what emerged here was something I wasn't expecting.

72. 13:57

    Maybe you look at this and say, "That makes a lot of obvious sense." But to me, I wasn't expecting it to be able to help the human to go and do the job, um, where it couldn't.

73. 14:06

    Um, so that was really exciting. I don't think I would've been able to get to that without, um, playing and, and being curious. Um, and then the last thing I, I wanna talk a little bit about is eyes on the future.

74. 14:19

    And we all have our eyes on the future because how can you not? There's always something new now, um, with models. Um, so I, I like to think about it as like, what's your pelican on a bicycle?

75. 14:32

    Um, and one of my pelican on a bicycles is autocomplete. I'm super excited about this. Probably a bad idea, um, actually to use an LLM for this, but I'm like, I wanna make an autocomplete that, um, is backed by an LLM.

76. 14:43

    And so this one has 100 suggestions for fixing some data. Um, and it's kind of like a bake-off, um, between these two things. I'm yet to find, um, a model that is both very fast and very good at this problem.

77. 14:56

    Um, but this is a, a, a benchmark or something that I've created just for myself to be able to feel the materials, um, that we're getting. And so I think about that for my design practice now.

78. 15:07

    Like, what are the things I care about, and can I like design into the future and start to think about the form factors I want and then build an application that can actually test that?

79. 15:18

    So yeah, that's, uh, all I have for you today. I, um, I'm very excited to see all the new form factors, um, that we build, um, with our new tools.

80. 15:26

    Thank you. [audience applauding] [upbeat music]