The Bitter Layout or: How I Learned to Love the Model Picker

[on-hold music] Bitter Layout or the alternative name for this talk, How I Learned to Love the Model Picker, and hopefully you will too.

So the idea for this talk started when I was perusing all the AI-first apps I use all the time and just realizing how similar they're all starting to look.

Very consistent layout between them all. And it's not just the chatbots and the answer engines, it's also the creative tools like coding assistants like v0 and even Canva. They're all try-- starting to use this very similar layout.

They've got an input field, they've got this turn-by-turn UX, they've got this dropdown with just way too many models to pick from. And, uh, it all feels like they're kinda retrofitting stuff into this chatbot UX.

Uh, but don't worry, this talk is not about if chat is the future or not. I think we've all heard that enough times, at least I have. And Swyx did a good job of humbling all designers with this tweet where he basically, he basically called, uh, all the design thought leaders out who are saying chat is dead,

and then... Or sorry, chat's not the future, and then they show off their cool demo, and then shortly after we'll just go back to using ChatGPT. So, uh, fair point.

Um, I think that right now we're in this state of a bit of a dualistic future of AI UX. So I've called this first section Schrödinger's Chat, uh, because we all, you know, all designers know how many usability issues that, that chatbots have, but yet we all still use them every day.

So it's kind of like obviously they are the future, but at the same time, obviously they shouldn't be. Uh, so I won't go into my thoughts on if they should be, but I'll do some, some anthropology here of just getting everybody up to speed in case you're not familiar with, uh, the great chatbot debate.

Uh, not sure if people realize how long we've been debating this, but I can track it all the way back to two thousand and twenty-two with this post from Linus Lee, which this is a great post by the way.

Still holds up today. But he essentially says he doesn't think that exposing the raw, uh, text completion is really the right paradigm long term. And so, you know, note the date, uh, May two thousand and twenty-two, uh, because if a, a couple months after that, we have ChatGPT essentially saying, "Hold my beer."

And, uh, you know, if that's not the right UI paradigm, it certainly didn't bother them. And, uh, I think a lot of other, other designers have kinda taken note of the escape velocity of that.

But still, uh, the next year, May two thousand and twenty-three, we saw some other great posts from people like Amelia Wattenberger, and then the next month, um, Maggie Appleton, who are making great arguments about why chat's not the future.

These, I think held up pretty well. Uh, but at the same time, yeah, obviously you have other designers who are arguing how intuitive it is, and then as we progress and ChatGPT hits escape velocity, we're kinda seeing everybody just start to meme the defense of chat, which is like, just look at the chart, you know?

It's like, obviously it's, it's working, right? Uh, and I think there's something interesting about that. If you can kinda come to the debate with a meme, it means there, there's something a bit intuitive about your argument, so, uh, fair enough.

Um, but then still, even in March of this year, we've had, uh, people like Julian Lear writing very good reasons of why chat should not be the future, and he's like showing clock speed here relative to the different interface paradigms.

So it's very convincing. Uh, he pretty much says it's a bottleneck. Uh, but at the same time, you know, we'll all probably still go back to using chat after this, so the Schrödinger's Chat remains.

And so I'll segue into the next, um, the next section here, which is called Models and Modes, and this is on this idea of the model picker, which is this other UI paradigm that's been developing alongside chatbots' popularity.

It's that, you know, I'm sure everybody knows what it is, but to be clear, it's this dropdown where you just have to select from a million different models. And so I put this...

I made this section in memory of Larry Tesler. If you're not familiar with him, he's kind of a big deal, invented like copy and paste and stuff. But another thing he was famous for was, uh, apparently saying, "Don't mode me in."

Uh, I don't actually believe he would say this, but I mean, the, the quote is like attributed to him. But you know, he hated modes. And if you're not familiar with modes, this is a setting in a UI where once you flip it, all of a sudden your inputs are mapping to drastically different outputs.

And so Larry Tesler hated this. He thought it was unintuitive, and he wanted everything to be modeless. And I don't know how many-- Or sorry, to give an example of a mode, just to be clear, uh, caps lock, this is a mode, right?

You hit caps lock and now your keyboard performs differently. Um, and then a, a more recent mode, [clears throat] excuse me, um, I'm not sure how many people would, would agree with me on this, but I think that the model picker...

Sorry. [clears throat] Losing my voice at like the worst time. Um, I think that the model picker is a bit of a mode selector as well. Um, you know, it's... Ob-obviously we're dealing with stochastic output and generative AI, uh, applications, so everything, it, it's kind of not a distinct change in setting, but once you flip a different model, you

have a whole step change of output. So to me, that's kind of like a mode selector and here's a quick video to, to illustrate this point. This is a bit old.

Uh, it's the, an older version of ChatGPT, but you can see I'm trying to use certain modes and, uh, the model is not supporting it. So I have to go through this menu and find which model allows me to use this mode.

And so the argument here is like we're kinda putting modes on top of modes, right? You now have to match models to modes and, uh, it's not super intuitive.

Uh, they've actually done a great job of redesigning this lately, lately, so this is certainly not like throwing shade at OpenAI. I think they have a great design team, but I'm just kinda illustrate a moment in time when this was super frustrating to me.

And like you kinda just want to talk to the model and be like, "Here's my use case. Like what mode and what model should I use?" But I don't know, maybe, maybe the model will pick itself, and so that won't work.

Uh, so this is, uh, really illustrating the point of the u- the flexibility-usability trade-off. This is a design principle where pretty much you're constantly trying to decide like how well do you understand user needs?

And if you can pretty much pinpoint them down, then you can create a very, uh, very usable, optimized UX for them. And but as you, you try to make a more flexible system, the usability tends to, to, uh, decrease because you're just increasing the amount of edge cases and the complexity and the requirements.

And I think that this is a, a trade-off that doesn't get talked enough about in this "is chat the future" debate. Uh, you know, we generally talk about in terms of absolutes, but it's really less of a yes and a no or-- and more of, like, a timeframe and, like, what trade-offs are we talking about?

Um, and so I'd like to, uh, segue into this next section, which is going to push the idea of, like, when we design interfaces, we really need to consider the zeitgeist that we're working in.

So what are the trade-offs of the time? Um, you know, what constraints we're working with? What timeframe could an interface be good for? Uh, the subtitle of this one is called The Context of All in Which We Live and All That Came Before Us.

Uh, Easter egg for anybody [chuckles] who remembers that. Um, and so to get into this section, I'd like to lean into a theory from The Innovator's Solution. This is the follow-up to The Innovator's Dilemma.

And, uh, in this book, they try to, uh, give you some guidelines on theory of how to approach building products with this idea of product architecture. So the architecture is generally this idea that you have a system, and you're trying to figure out how the components in the system are interacting with each other or interfacing with them.

Um, and so when you start to understand how they interact with each other, you can see different, um, different attributes. So they map them out to these two distinct sides of a spectrum.

You've got integrated architectures, and then you've got modular architectures. And, uh, you know, generally integrated, this is more common in early-stage disruption, and you have proprietary technologies. They're very optimized, interdependent.

It allows vertical scaling. And then to contrast that, you have modular, and this is generally when technologies start to commoditize, and they can be more interdependent, and you can allow horizontal scaling.

Uh, but the key point in their theory is that, you know, you're never really on one side of the spectrum or the other. You're instead kind of bouncing between the two.

And the in-- the, uh, industry as well is having different parts of the tech stack commoditize and decommoditize all at once. So as a designer or anybody else building, you pretty much have to figure out where are the strategic points to be integrated versus modular.

And so their, their theory uses, like, IBM as an example, uh, when they started out making mainframe computers very integrated. Then they shifted to personal computers and started to make it more modular, and then, of course, ended up c-- the whole, um, computer itself ended up commoditizing at some point, and they got out of that business.

Uh, so thinking through, uh, today what parts of the A-AI industry are commoditizing and decommoditizing can help us think about how to design interfaces. And so, of course, if-- the main question probably everybody would ask in this when you start to pose this prompt is, uh, are the models themselves commoditizing, right?

'Cause it-- this is kinda, like, the big topic that everybody debates. And, um, it brings us to, uh, The Bitter Lesson, uh, from Rich Sutton. If you haven't read this, I definitely recommend doing so if it's not wise to build an AI today without knowing this lesson.

Uh, but I'll take the TLDR for this talk is that you just-- we shouldn't assume that computation is constant, as long as we're seeing scaling laws in effect, right?

Like, as long as the, the next model is still important, which you can see it still is, like, every time a new model comes out, everybody's like, "Drop everything, and let's check out the new model."

Uh, as long as the scaling laws are still in effect, then we can assume that the, the models themselves are not commoditized. And so the bitter lesson actually leads us to what I'll call the, the bitter design lesson.

Why not? Uh, which is this idea that if the basis of computat-- of, of competition is inference performance, then the UI itself must be primarily focused on conforming to the next model.

So said plainly, if the model's not commoditized, it's actually the interface that's the commodity now. And until that changes, when, like, models overshoot user needs and you don't need a rocket scientist doing whatever use case you're doing on-- in ChatGPT, uh, then we can start to explore different integrations within the interface.

Uh, but until then, the primary job of every interface is really has to be figuring out how to conform to the next model's capabilities. So it's, it's kind of a bitter design lesson because then you get, uh, layouts like this, which, you know, the bitter layout.

Uh, pretty uninspiring, uh, not super usable, uh, just not very cool at all. But the one thing this does really well is it can absorb the next model's capabilities.

So as soon as that next model comes out, jam it into The Bitter Layout, and then, you know, update your model picker, and your app is more intelligent. So, you know, I, I hate this design, but, like, as a designer, like, you can't really hate on this ROI, which is you just add one line item, and now

your app is NX more intelligent. So kinda hard to debate this as a design decision. So that's the, the bitter design lesson. Um, and the takeaway from this, I think, is, you know, I'm not ready to eat my words on saying chat is not the future yet.

Uh, but I think that it's quite clear there's one attribute of chat, which is that it can really conform to the next model very easily, is one of the, the key features we need to keep in mind.

And so the future of AI UX until where we see models stop commoditizing or, or excuse me, until models do commoditize will be that the, the future of U-- AI UX must conform to the next model.

So that's the, the bitter design lesson, but how do we go from bitter to sweet, right? What comes next? And as most things in life, uh, Bret Victor has already given a really good talk on this, so you should just watch this talk.

It's much better than this one. Uh, but it's called The Future of Programming, and he explains all of the kinda mistakes we've made over the past decades with thinking about programming and specifically uses this, uh, or for this context, he uses this, uh, example of how people re-- found it very hard to go from binary to soap,

right? The, the binary programmers could not understand how you could give up control to the machine and use these abstraction layers efficiently, and, uh, they like to hand code everything.

And of course, like, making this mind shift was, was really important, a-as we can see in retrospect. And so his ta-- his, his lesson was you should stop thinking in terms of procedures and start trying to think of programming in terms of goals and constraints and pretty much guiding programs with, with these higher layer, layers of abstraction.

And so, uh, designers, I think, are actually pretty well suited to do this. Um, you know, it'll be a mind sh-- a mindset shift to jump from, uh, thinking procedurally to goals and constraints.

You know, we, we like to be very detailed in how we design user flows and consider all the edge cases and all that, and it's very important. But we are likely gonna have to start to move up a layer of abstraction as we're seeing apps become more and more stochastic and dynamic and just more probabilistic.

So we can't really envision every possible op-- uh, path now, so we have to think a bit more in terms of what constraints can we set and what goals can we set to get people to the-- a-along the happy path.

And so, you know, design systems, um, these are already things we use to guide to, uh, developers and our designers to, to goals and set constraints for them. So what happens when we start using this for generative UI, right?

If we start to collaborate with a model, do we have a design system that keeps them, uh, within the right constraints? Uh, quality assurance, is that like reinforcement learning?

I-I'm not really sure, but like maybe it is when you go to like critique a design that a, a model has created. Will, will this be like a reinforcement learning loop?

Uh, user stories, these are-- You know, we're very good as designers of thinking about how to envision what the user is trying to do via user stories, and these are kinda like system prompts in a way.

Like, can the model become also a, a partner in helping, uh, set the, the goal for a user? And, uh, we, we can translate some of these user, user stories into the system prompts themselves.

Uh, so this is pretty speculative, speculative, but I think this is a nice prompt for the future of UI design in the AI age. And, uh, I'd like to end on this quote from Dario Amodei, who says he feels that generative AI systems are more grown than they are built.

And I like this as a prompt, uh, for inspiration for helping us do this, uh, shift mindsets, and start to think about how the future of the-- of UX might be more one where we're-- it's less like a process of construction and perhaps more like a process of gardening.

So, um, if we can embrace these types of lessons and start to think about design in a new way, a little-- at a higher level, uh, perhaps then we can move beyond the bitter layout.

Uh, thanks a lot. [audience applauding] [upbeat music]