Build the Right Thing: Product Engineering (Part 1) — Kent C. Dodds, EpicProduct.engineer
Read the talk
Build the Right Thing: Product Engineering (Part 1)
Kent C. Dodds turns a queue outside the workshop into a product discovery exercise: find evidence that a problem matters, connect it to technical choices, and keep learning after the first prototype.
From a talk by Kent C. Dodds
At a glance
Ideas worth remembering
Cheaper implementation makes selecting valuable work more important: completed code can still fail to produce customer value.
Product engineering connects customer needs to data models, workflows, constraints, observability and the smallest useful slice.
Ask about specific past incidents, workarounds and costs before asking users to judge a proposed solution.
Match technical investment to the work’s purpose. An experiment should produce evidence early; a business-critical capability may warrant greater attention to detail.
Continue discovery through prototypes: observe whether users reach their goal, persist through gaps or abandon the solution, then use that learning to revise the system.
A bottleneck before the first slide
The workshop begins with people still waiting outside. A line stretches away from an entry bottleneck, while seats inside need filling. Kent C. Dodds jokes that a room full of engineers ought to solve the problem, then asks attendees to wave newcomers into empty seats. Before anyone proposes software, the scene already contains several different concerns: processing entry, finding available seats, and getting people into a session that has started.
That immediate frustration will become the workshop’s example. First comes the reason to practice product judgment at all. Dodds went into full-time teaching in 2019, helping experienced engineers acquire experience with testing, React and full-stack development. Better coding agents unsettle that business: an engineer who already understands software may now direct an agent through an unfamiliar framework instead of taking a course on its implementation details. His admission has a personal cost—he sells those courses.
The educational choice is between teaching this week’s agent techniques and teaching a skill that survives changes in tools. Dodds chooses judgment: deciding what deserves to be built. His thought experiment imagines automation doing almost all engineering work and asks what useful human contribution remains. This is a forecast about the direction of automation, not a claim that present agents can independently produce dependable software; he acknowledges their mistakes and rejects planning around a fully automated future whose consequences nobody knows.
The archer metaphor makes the change concrete. Engineers once differentiated themselves by hitting a difficult target: translating a request into working software despite technical obstacles. Agents make the arrows behave more like homing devices and let an engineer attempt many more targets. The targets still have different values. Easier implementation increases the importance of choosing one worth hitting.
After a round of 12 air squats, the room gets a second kind of participation: attendees can submit and vote on an example app, and use the same interface for questions. The example is deliberately unrehearsed. The frameworks must work on a problem chosen in the room, and attendees are asked to apply them to their own applications too.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Finished implementation can still be failed work
The question changes from “can we build it?” to “is it worth building?” This applies to an internal enterprise feature as much as a new startup. Money remains finite, and a product that attempts every possible feature can overwhelm its users. Selecting useful work has always mattered; cheaper implementation makes the consequences of poor selection arrive faster.
A ticket can be implemented correctly and still produce no customer value. Treating the specification as the end of responsibility leaves the engineer doing a job that increasingly resembles an agent’s: accept instructions, produce code, take the next ticket. Wayne Allen’s formulation puts the dependency in order: “Building the thing right is downstream of building the right thing.” Technical quality cannot rescue an unnecessary capability.
Faster generation introduces several distinct traps:
- Product expansion. Adding features becomes easy enough that someone must deliberately stop the product from accumulating unnecessary behavior.
- Repeated bad practices. Agents can spread existing codebase mistakes faster, increasing the importance of the environment and patterns they work within.
- Convincing prototypes. A polished interface can feel ready to ship while leaving scalability and edge cases unresolved.
- Sunk effort. Rapid progress creates attachment to the implementation, making it harder to stop when the direction proves wrong.
Owning the eventual system changes how a prototype is used. A prototype can establish the desired experience, then be replaced with an implementation shaped by engineering judgment. The prospect of being paged at 2:00 in the morning supplies a practical reason to care about the architecture, constraints and agent working environment. Looking functional is only the beginning of that responsibility.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Customer context changes engineering decisions
Product engineering connects customer needs to technical choices. The engineer still decides the data model, workflow shape, observability, constraints, failure modes and smallest useful slice. Understanding where a request came from helps make those decisions without overbuilding or creating a system that cannot accommodate the real workflow. This responsibility overlaps with a product manager’s understanding of users while retaining a technical center of gravity.
The smallest useful slice can require subtraction. In Dodds’s comparison, AI can hand over something that looks like the whole product, leaving the engineer to whittle it down. He illustrates this with Instagram’s early history: watching users revealed that photo sharing mattered more than the surrounding check-in features, so the product concentrated on photos. The useful operation is to preserve the behavior people value and remove the rest.
Watching work happen also reveals requirements that an office conversation can miss. In a story from Uncle Bob, software for telephone-line technicians looked different after its developer rode along and saw someone use it while hanging from a pole. The environment made improvements obvious. A workflow that is tolerable at a desk may be difficult in the conditions where the customer actually works.
A technically interesting API does not automatically justify switching workflows. A prospective user already has a way to get something done; the new product must improve that experience enough to make changing worthwhile. The same context matters inside the architecture. Even when agents make a database change or service rewrite cheap in tokens, the change can still cost the team coordination and disrupt users. Choosing suitable building blocks early avoids costs that code generation alone does not remove.
WorkOS provides a concrete architectural example. In Dodds’s account, Michael considered distributing the authentication platform as an on-premises licensed product or through an open-source model with a commercial arm. Customer conversations brought an operational requirement into focus: a security fix needed to reach users immediately. Waiting for each customer to upgrade an npm package would leave that timing outside the provider’s control. The resulting choice was a hosted service.
What changes when the provider needs to deliver a fix immediately? The comparison below follows the update path. Customer-managed installation inserts a customer upgrade between the fix and its use; hosting lets the provider deliver the correction through the service. The architectural choice follows from who must control that step. This example supports the decision for the requirement described, rather than establishing that hosted authentication is always the right choice.
A correction must reach customers.
The requirement for immediate fixes favors a delivery path that does not wait for each customer to upgrade a package.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build a working environment, and look beyond your ticket
Directing agents extends an older team-lead responsibility: create a system in which implementation goes well. Good tests, clear architecture and obvious building blocks make it easier to assemble features without unnecessary mistakes. Running more agents increases the importance of that preparation; it does not remove the need for it.
System design also includes the conditions under which people make decisions. Dodds recounts Don Norman’s response to the Three Mile Island incident: instead of stopping at operator mistakes, examine the system that led competent people toward those decisions. Norman’s provocative phrase, “User error does not exist,” is presented as his assertion. Its practical use here is to redirect attention from blaming a user to understanding what the design made difficult or misleading.
The first questions turn these ideas into decisions about scope and hiring:
- MVP scope. Ask for the minimum thing that demonstrates a solved problem for a particular audience, rather than an acceptable number of features. In a crowded market, narrowing that audience may help.
- Hiring. Dodds recommends evaluating how candidates discover the core problem and derive its system implications. Banning agents from a coding exercise poorly reflects work in which agents are available.
- Discovery timing. Talking to customers and building should continue together as a feedback loop.
- Distribution. Getting users to adopt a product is a separate difficulty; this workshop does not offer a developed answer to it.
Close collaboration with product managers supplies context, while collaboration with neighboring engineers prevents duplicated systems. An engineer who sees only one slice of an application may rebuild a capability already available elsewhere. Dodds recommends looking at least one neighbor outward, upward and downward—much as using React effectively benefits from understanding JavaScript and the browser beneath it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Just Get In the Workshop: change the questions before building
The workshop names three intended frameworks: The Mom Test for early validation, jobs theory for framing requests, and the Kano Model for prioritization. This part develops the first. The winning audience proposal returns to the opening scene: an app that gets people into conference workshops. After considering a Lightning Lane name, the room settles on Just Get In the Workshop.
The first questions mix observation with proposed solutions. How long have you waited? Why are you waiting? Are there seats inside? Would pre-registration help? That last question already commits attention to a mechanism before establishing what causes the difficulty. Soon it is easy to imagine a mobile app, a website for one-time users, and integrations with conference providers. The product grows in the conversation before its necessity has been established.
The Mom Test changes the object of the interview. Asking someone to evaluate an idea invites reassurance, speculation or a polite way to end the conversation. Asking for a concrete episode gives the engineer behavior to interpret. The user supplies the experience; the engineer remains responsible for diagnosing the problem and choosing a solution.
The useful questions collect different kinds of evidence:
- The episode: Tell me about the last time this happened. Who was involved, and what made it annoying enough to remember?
- The workaround: What did you do instead? This reveals the existing workflow a new solution must fit into or replace.
- The cost: What did the workaround cost in money, time or effort? How often does the problem occur?
- The consequence: What are you losing by not having a better solution?
For Just Get In the Workshop, the observable change is in the discovery process, rather than a shipped app. The queue begins as an invitation to automate entry. It becomes a question about a paid attendee’s lost experience: Dodds gives missing the first 20 minutes of a talk as a concrete cost to investigate. The interview then asks what the attendee did, how much that delay mattered, and whether it happens often enough to justify a change. Those answers could constrain the solution; the exercise does not yet establish which entry mechanism would solve the bottleneck.
This queue makes discovery unusually direct because people are experiencing the problem now. Elsewhere, an interviewee may struggle to remember a recent incident. That is useful evidence too. Existing expenditure and effort suggest motivation to improve the situation; weak recall and little action suggest a weaker opportunity. These are signals about likely willingness to change, rather than guarantees of future adoption.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Validate demand before engineering for everyone
Wayne Allen’s real-estate product story shows the cost of getting the order wrong. An experimental product inside a larger organization was built with microservices and a capacity ambition: handle every person in Australia logging in simultaneously. In the account Dodds relays, the team spent a year and 1.2 million Australian dollars, then had no users. The system solved an imagined scale problem before establishing demand.
The retrospective alternative was a two-week launch with manual integration. That would have traded automation and capacity for faster market learning. It is a proposed counterfactual, not a demonstrated successful launch, but it identifies the avoidable commitment: a whole team spent a year before discovering whether anyone wanted the product.
The same decision applies to a feature request. If its validation is unclear, ask the product manager why it exists. That answer guides both whether engineering time is justified and how much to invest. A one-off experiment and a capability essential to the business deserve different tradeoffs. Essential work may need more attention to detail while still being delivered in a slice that can produce early evidence.
Enthusiasm from weeks of private hacking can obscure this distinction. Showing the idea to someone who does not understand its appeal may feel like having the fun spoiled. That discomfort is useful: it is a reason to investigate the mismatch, rather than retreat into more implementation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep learning when the prototype reaches users
Problem validation and solution validation answer different questions. The Mom Test helps establish whether a problem deserves work. A prototype lets users reveal whether this particular solution gets them where they need to go. Early adopters who remain excited despite substantial gaps can signal that the core benefit matters; users who abandon it after a few difficulties provide evidence about the solution’s usefulness too.
Dodds’s vivid image is a user willing to run through glass to reach the outcome. The point is the strength of motivation visible in behavior: a valuable result can make people tolerate an unfinished prototype. Their persistence helps identify what to preserve, while their difficulties show where the implementation still interferes with the goal.
How does customer learning continue to influence the system after building starts? The cycle below joins discovery to implementation and observation. Each pass can revise the useful slice and its architecture. Knowledge of existing workflows can also expose transition costs, such as downtime during migrations, that would be easy to miss if the engineer saw only the requested feature.
Ask about episodes, workarounds and costs.
Past behavior helps choose the work; observed prototype use helps revise the solution and its technical shape.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Technical ownership remains as roles change
The closing questions sharpen the distinction between product engineer and product manager. A product engineer understands the system’s available data types, queues, scheduled jobs and other building blocks, including their limitations. Given a user goal, that engineer can decide whether existing infrastructure fits or whether the system needs to expand. Customer understanding makes the technical choice better; familiarity with the implementation makes it possible.
Whether these responsibilities need separate people depends on the organization. Dodds handles multiple roles as a self-employed builder; a small startup may share them among three people, then separate them as it grows. He expects AI to let a larger business operate with a smaller organization, but offers no numerical staffing rule. The responsibilities still exist even when the titles are combined.
The junior-to-senior question receives a modest answer: juniors still learn by trying to emulate more experienced engineers, while formal promotion remains a conversation within their organization. This part ends as the scheduled session rolls into the next hour. The unresolved work is to keep translating user goals into system decisions, rather than treating faster code generation as evidence that those goals have been met.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the complete timestamped transcript
- 0:12
Okay. Hey, everybody. We have, like, the line... [REDACTED], hey. Uh, we have a line that goes, like, all the way over here, and, uh, a bottleneck right here that we're all engineers, we should be able to figure out how to solve this problem. No, just kidding. I d- I don't wanna step on anybody's toes. Um, but if somebody wants to figure out what app he's using and help him get them through, that would be sweet. My name is Kent C. Dodds, and we're just gonna get started 'cause we have two hours. I think, uh, we actually have plenty of time, um, but,
- 0:42
uh, I, I don't want to have you all just waiting around. So as people are filing in, be friendly, and if you've got a seat next to you, like, wave them over and let them in 'cause I, I am not certain that we're gonna be able to fit everybody who is in line in this room. And so we do wanna get every seat filled up so people, uh, are able to make it in here. Um, I am just thrilled, and honestly, I don't wanna say I'm shocked, uh, 'cause that reveals some, uh, level of, um, uncertainty on my part, but, um... So
- 1:12
we won't say shocked, but I'm thrilled, um, that, uh, you all are here, uh, to talk about building the right thing, um, product engineering. Before we get into that, to give people a little bit of time to come in, uh, I want... or by a raise of hands, I want to understand who, uh, I'm talking to. So how many of you would consider yourself at your company, you are, like, individual contributor, you're, um, maybe not writing the code anymore, who's doing that, but you're, like, directing the agents to write code?
- 1:42
All right. I kinda figured most of you would be that. What if you are somebody who leads those people? And you can be both. Okay, awesome. What if you are only somebody who leads those people? Okay. Okay, we got a couple managers. Uh, all right. Uh, what about, like, product owners? Anybody consider themselves a product owner? Okay, sweet. Uh, any CEOs in here? Who's a CEO? I was just gonna be like, "Hey, that's cool for being the chief." Um, uh, all right, sweet. And how many-- who has traveled the farthest? I talked to, uh, a
- 2:12
group of guys over here who traveled from India. Anybody travel farther from India? Where did you go? [REDACTED]. Oh, [REDACTED]. Nice. That is one continent I have yet to set foot in, would like to one day. Um, [REDACTED] is nice. My sister actually lived, uh, in [REDACTED] for a while. She liked it. Big bugs. Uh, anyone else come from really far away? Uh, yeah, over here. [REDACTED]. [REDACTED]. Nice. Uh, I, uh, I was gonna be in [REDACTED] [REDACTED], but I'm in a
- 2:42
play with my kids, and so I had to cancel. Um, Finding Neverland. I'm not a main character. That's for n-next play, maybe. Uh, anybody else from far away? He- yeah. Brazil. Brazil. Oh, whoa. Dude, what do you... Oh, no, the game just ended. Who won? Brazil. Oh, wow. Okay. Good for you guys. That, that was an interesting game, one-one. That was, uh, for a long time. Very cool. How exciting. Congratulations, Brazil. Uh, okay. Let's...
- 3:12
Dang, this stinks that, um, that everybody's not in. But I'm-- uh, it's so exciting that so many people are interested in l-learning what-- uh, how to build the right thing. But let's get right into it. Here is... Um, my clicker not working. Let me plug it in.
- 3:30
Okay.
- 3:33
Um, oh, there's an on switch on this clicker. There we go. All right. Here's my thesis. This is the reason that, um, I am talking about this. Well, actually, to back it up. Here, I'll, I'll just back up give... since we have a little bit more time. To give you a little backstory on myself, um, I have been developing software for over a decade, uh, professionally. I graduated from my, uh, university, BYU, in, uh, 2014. So it's been 12 years sin-since then.
- 4:03
Um, and then I'd been developing software as a, like, you know, part-time before that, uh, a little bit. So yeah, about a half a decade, uh, I've been doing software development. And, um, through all of that, I, uh, very quickly got into teaching, whether that was, uh, teaching on the side, uh, or eventually, in 2019, I went full-time teacher. And that has always been, for me, about teaching experienced software engineers how to... uh, or rather accelerating the acquisition of experience for, uh,
- 4:33
experienced software engineers. So you don't know how to write tests? Well, great. I'm gonna teach you how to write tests. You don't know React? Great. I'll teach you React. You don't know full stack? I'll teach you full stack development. And, uh, something happened this year that, uh, gave some of us a bit of a fright. I know for myself, I had two existential crises this year. How many people can relate to that? All right. Yeah. We all get back from Christmas break or maybe on Christmas break, you're using your agents and you realize, "Oh my
- 5:03
goodness, this, this thing is better at coding than I am." Or m- okay, maybe it's not quite, but it's, uh, gotten a lot better. And so my existential crisis was not only am I a software engineer actually building software, but I'm teaching experienced engineers how to get experience in technology they don't have experience in yet, but now who needs to learn React? Like, that is just not a... Like, if you're an experienced software engineer and you're like, "Oh, I've done some Vue before," or whatever. "Oh, this project uses React," you don't need a
- 5:33
React course for that. And I... me just saying that l- means I lose money because you could go buy my React course. But I'm telling you right now, you don't need that because you're an experienced software engineer and you can direct the agent to go build the right thing. Like, you don't need to know what useState is. Like, who, who cares about any of that anymore. As long as you have the right building blocks, you can build really anything now. So existential crisis for me, like, who needs to learn implementation details anymore? And so I came down to... Yeah, I kind of...
- 6:03
Maybe I should just pull up. I have this other talk while everybody's coming in. So you're gonna get- Uh, a little bit of that one, um, that is, uh, sort of related. So uh, we'll, we'll, we'll just do the first part of this one. So, um, this is, this is the work. It's standing up on these two stands. If you don't get the work all the way across, then it's gonna fall over. Our tools we... Your IDE, your CI, whatever, and then here's the work that you do. And over time, um, our agents have just
- 6:33
been eating into the work that we do, which is wonderful because it means that we have less work that we have to do. We can accomplish more to, uh, get more done. It's awesome. Um, but, uh, yeah, it makes you ask yourself what do you actually do, and here's just total overwhelm of what's prompt engineering and reasoning and, and loop engineering and flow engineering. Like, what i- all of these different things that we're supposed to learn, right? As an educator, I look at that and I think, yes, I could make
- 7:02
bank on just every other week I do another course. You come, and I'll teach you what the... how we do things this week, and then I'll see you in two weeks 'cause it's gonna be different, and I'm gonna make so much money off of this. Uh, as much as I love making money, uh, and I do have [REDACTED] kids I have to take care of, um, I... Yeah, I'm not into that. I would much rather teach you durable skills that you can, uh, learn and, um, a- and use long-term, so you don't have to keep coming back to your, uh,
- 7:33
dealer or whatever. So we don't know what the future's going to look like. It could look like that. It could look like this. We really just have no idea. Um, but, uh, the, the fact is that there is a possible, may... We can argue this, but there is a possible future where the work is actually completely done by AGI, right? Like, that is p- like, if you think that that is impossible, then I'd love to have a conversation with you about predicting the future. Um, but I think
- 8:03
that this is definitely possible, but it's not very useful to plan for this because who knows what the world is gonna look like if we eventually get to that point. Uh, well, like, what is, uh, humanity at that point? So let's say let's take it a step back from that. So i- it's doing almost everything, but not everything, okay? So it's gonna fall over. It can't fill in that last little gap. So there will be a piece that is us filling in that last little slice. So my... As I'm thinking about, okay, what do I do as an educator, my interest is
- 8:33
that, okay? Let's fast-forward time all the way to the future. AGI is here and is doing everything. Take one step back from that. What is that slice that humans are still doing as software engineers that is still valuable? And that is where I came to judgment. Being able to tell, uh... and, and not even just judgment, but product engineering in general. Being able to tell what to build. What's the right thing that we should be working on? Um, and the metaphor that I like for this, and... Wah. Gosh. Okay. The metaphor that I like
- 9:03
for this, and I'll, I'll actually bring this up in the talk, uh, the official, uh, workshop later, uh, is, uh, an archer with arrows. So you've, you've got your arrows, and you've gotten really good at, like... Your product manager tells you, "Hit that target," and you're like, "Okay, great." And you aim. You take account the wind, everything, and you hit the target. You're like this dude who is aiming to hit the target through that ring, which is just... And he does it. I just... Like, wow. Can you imagine being that good? Some of you are that
- 9:33
good as software engineers. But see, here's the problem. The problem is that we have gotten our arrows changed, and now you don't have to be that good to hit the target because the... A- and, like, I realize I'm doing some hand-waving. Yes, we all know that agents make mistakes, yada, yada, but, like, look at the trajectory. They're like homing devices now. And so now not only can you hit any of those targets with ease, but you can actually hit way more targets than you ever possibly could have imagined before.
- 10:03
And the, the trick is the fact that because of, uh... Well, the... Actually, this has always been the case. Tar- all the targets are not created equal. And so knowing which one of those targets is the valuable thing to hit, that's the differentiator. Okay, I think that's as much as I... Oh yeah, this is... If, if you have seen, uh, Ramanujan Man in Tights, you know this reference. If you haven't, then, um, I'm not gonna show it anyway, so don't worry. Uh, all right. We've got enough people in the room. I'm gonna get
- 10:33
officially started. So thank you all for, uh, joining. Um, we're going to talk about that skill. This is what I call the durable skill. It's the last skill that the last software engineer needs. Um, it's the last thing that you need to learn, and if AI learns this skill, then I don't know what we're gonna do. Um, but it's not this. So, um, this is the thesis. When AI agents level the implementation playing field, which they are actively doing, then the differentiator becomes
- 11:02
building the right thing, okay? I'll, I'll, I'll stop. I see some people taking photos. This is the slide to take a photo of. This is my thesis. The entire workshop depends on this slide. Uh, I'll bring it up a couple times, uh, later. All right. I hear some of you chuckling. How many of you know what this slide is? Have you seen this slide before? Really? Oh, how ex-... Okay, got a couple. This is very exciting. Um, I do this in every one of my talks. I'm going to invite you to please stand. If you're physically able to join us, please do. Your
- 11:32
blood needs to flow in your body for your brain to operate at peak efficiency, which you will need for the next two hours. Hour and a half. So I want you to put your arms out in front of you like this. Don't hit anybody, please. Squat down like this, and come back up. This is called exercise. This is an air squat. We'll do 12 of them together, and you need to count out loud with me. If you're not counting loud enough, I will start over. Ready. One. One. Oh, wow. Two. Two. Great. That threat really worked. Three. Three.
- 12:02
Four. Four. Five. You're doing so great. Six. I see smiles. You love this. Seven. Seven. Eight. And you can go really deep if you want. Nine. 10. You know, this is so fun, let's start over. One... No, just kidding. 11 and 12, and then stretch over your head, and then over to one side, and over to the other. All right. Sweet. Go ahead and sit down. Thank you so much. Your body needs blood flow. Yes, that's right.
- 12:29
Uh, if, if you're starting to feel a little sluggish, it is after lunch. Um, so if you're starting to feel a little sluggish halfway through, just stand up, do it. We all know what's going on. It's fine. There are some seats up here in the front, so if you're standing in the back, it's not awkward. I already told them to be friendly to you, so, um, they will be friendly. All right. We are going to... For the workshop, we're gonna be working through an example product, an example app. And so, um, this is a QR code where you can submit your ideas for
- 12:59
what that should be and up vote other people's ideas. Um, I'm running a big risk because it means I wasn't able to practice the specifics of what we're doing, so I'm gonna need a lot of feedback from you on, um, the different ele... how we apply the frameworks we're gonna, uh, be talking about to the example app that we come up with. And then also, this is the same QR code, and, uh, this is how we're gonna do Q&A. Uh, so I will be doing Q&A. You can up vote each other's questions, uh, all there, and I
- 13:29
will put this same QR code at the bottom of every slide, so, uh, if you're, uh, think of a question later, then you can scan it. Um, and I'll come back to the example app, uh, that you come up with. Feel free to, uh, to... I- if you think of any ideas. Or actually, as we're working through this, I want you to be thinking about your own application, taking notes about how these frameworks apply to your own application as well. And I'll be asking you for, um, examples of how you've applied it to your own a-
- 13:59
application. So I'm expecting you to do that. Okay. So the, the premise here is AI changes the scarce resource. Uh, it's no longer scarce to do implementation. That is a, um, a commoditized thing now. It's... Again, I'm being a little hand-wavy. I realize that agents aren't perfect and they make mistakes. You still... But, um, we are in a, a place where agents are getting really, really good, and they're only getting better. So the more valuable thing is deciding what to build, whether it's worth building in the
- 14:29
first place. And that applies not only to entire apps. Like, if you're here and you're like, "Well, Kent, I work at an enterprise, and I'm... It's, like, internal stuff. Uh, I'm not working on new apps every day." That's not what this is about. And in fact, I'm more interested in your use case than the startups. Um, I'm interested in both. But, uh, these, all these principles we'll talk about apply to both of those. So we're moving from can we build it, to is it worth building? What's, what's interesting about that is that has always been important, and the best software engineers,
- 14:59
the most valuable software engineers at any company, were the ones that could answer this question, could help the business get to this answer. We'll talk about that a little bit too. So we already talked about the arrow metaphor. For those of you who missed it, we've gotten really, really good at, uh, hitting targets, uh, with the agents that we have now. There are tons more targets we could possibly hit, and so the real question is, um, whether we... which one of the targets we should actually try to aim for. Because maybe you do have infinite money, but I don't,
- 15:29
and I can't aim for every one of those targets. And even if you tried, you would probably overwhelm your users anyway, and that would be a f- like, a failure mode to use the AI. Like, I never used the words failure mode until AI started saying that to me. But that would be another failure mode. Um, so I'm gonna actually be referencing a podcast that I've been running for the last couple months, uh, quite a lot in this workshop, because they're just really, really smart people, uh, and I really li- like their takes. So this is Ben Alecvadhu. This is an u- upcoming episode. He said,
- 15:59
"It's once development becomes cheap and easy and enables anybody to be able to do it, especially non-technical people, then it's really about the ideas that you have." Now, I wouldn't say that we can take our tools that we have today and give it to a non-technical person and expect them to build, like, real software that can be maintained and, and, like, last for 35 years or whatever. Um, but it's possible we get to that point, and the, the fact is that the implementation playing field is leveling. It's in the process of that. So, uh, ideas have always been, uh, one of the most valuable parts of it, and
- 16:29
then of course there's execution. Um, but ideas are becoming more valuable relative to that execution. Um, Julius, um... I'm not sure how to say Julius's name. Sorry, Julius. It's, uh, Marminge, I think. He works on T3 Code and, and, uh, a bunch of other T3 universe stuff. But he said, uh, that, "The hard question is deciding whether a feature is worth having or what the long-term consequences are." That is a- an important part of ownership that we'll talk about here as well. Um, so it's easy to
- 16:59
build to spec and think that, "Okay, yeah, I'm a software engineer. I build to spec. That's what I do." Uh, hand it off, and then say, "My job is done. Let me go grab the other thing." But guess what you look like when you're just taking a ticket and turning it into an implementation. If that's you, you look a l- awful lot like an agent to me. Uh, oh, we're gonna play some music, I guess. Um, whoop. Turn that off. I think there's a little play button on this thing. I've never seen what that does until now. So,
- 17:29
uh, very replaceable to just turn a ticket into an implementation. Uh, so a product engineer, um, i- is able to recognize that you can still fail after finishing the implementation if it doesn't produce customer value, and that's not just your PM's job. Uh, okay. And then Wayne Allen, brilliant guy. This was an awesome episode. He said that, "The product concern is building the right thing, and the engineering concern seems to be building, uh, the thing right. Um, however, building the ri- uh, right thing is downstream of
- 17:59
building the..." Uh, sorry, I, I said this wrong on the podcast too when he... I repeated it too. "Building the thing right is downstream of building the right thing." So it is very much in your interest as an engineer who wants your company to be successful so that you can continue to receive a paycheck and take your kids out to get milkshakes that night or whatever, you know, with the money that you get. It's in your best interest to make sure that the upstream activities are coming down so that you have, uh, something that's actually going to be valuable to work on.
- 18:30
Okay. Uh, whoop, there we go. Um, Dax Raad, uh, creator of, uh, OpenCode, uh, he... How many people use OpenCode, by the way? Yeah, OpenCode's pretty cool. Um, he says that, uh, the default place for our new coding agent abilities to go, uh, to is work on the wrong things. Products go from good to bad faster than ever. How many people have noticed some of the products you use getting worse? Yeah. I, I relate to that. The products that I build are getting worse. No, just kidding. Hopefully- ... hopefully they're not the products you build, of course.
- 19:00
No, nobody in this room is doing that. Uh, it's just everybody else. No, uh, it's just so easy 'cause the agent's not gonna say, "Hold up, hold up," like, "I think that we're expanding the system too bad," or whatever. They might in the future, but no, like, it, it is our responsibility to slow down and be intentional about what we're adding to our product so it doesn't bloat up to something too ridiculous. And the, the problem is that agents accelerate, um, bad practices throughout the code base and really do need to be wrangled in, uh, as Shaundai is per-
- 19:30
um, uh, talking about here as well. So you can move bad faster if you're not careful. Um, and Rita, uh, Koslov, she w- I just recorded this episode last week with her. Uh, she is a, the, um... Oh, shoot, what? A director of product or VP of product, uh, I think-
- 19:48
She said
- 19:48
... uh, at Cloudflare. Uh, she's a higher-up, and she's wonderful, wonderful person. But, um, she said that it's very easy to, uh, get the look and feel, like, really quickly and just decide, "You know what? Why don't I just ship this?" So she's on the, the management side of, uh, things, and it's really easy to feel like, "Oh, I can just ship this." But then she says, "You can build something that on the surface layer feels functional, but it's not actually scalable, doesn't address edge cases." And, uh, the, the real key, and this is where we come in as product engineers, is to actually take
- 20:18
ownership of that, to look at the prototype and be like, "Yeah, th- I'm gonna throw this away. We'll have Cod or Cursor, whatever, re-implement this in, in two hours," and it'll, uh, you know, with my framing, my systems thinking and everything, um, so that I can take ownership and accountability over it. And the reason that matters is because if you know that you're going to be paged at 2:00 in the morning to deal with issues, then you're gonna take a lot of care in the system, in the playground that you create for your agents, uh, to make sure that they're successful when they're implementing things. And Aaron Francis,
- 20:48
you can do the wrong thing incredibly fast and feel like you're making a ton of progress. How many people relate to that? Yeah, so, so bad because, like, you get so far into it, and you're like, "I can't stop now." Sunk cost fallacy is not a thing. And yeah, so how to do the right thing incredibly fast or at least directionally correct rather than 10,000 lines of code a day on the wrong freaking thing. It's so irritating. So, uh, a question that I get often... We're, we're gonna get to the interactive bits here in just a second. I just wanna establish things
- 21:18
so nobody leaves the room, um, that we are not... I'm not saying, "Now it's time for you to all be product managers." That is not what I'm saying at all. So let's talk about what, where the line is between a product engineer and a product manager. Um, okay. So your job as a product engineer is to connect the customer's, uh, the understanding what the customer needs to the technical choices that are being made so that you don't paint yourself into a corner and you don't overbuild. So, uh, you're gonna decide on the data model and
- 21:48
the, uh, workflow shape, observability, constraints, failure modes, and the smallest slice. All of these things are technical decisions that you make, and you make them better by understanding the upstream, uh, where requests are coming in, where the ideas are coming from. Um, yeah. The, and we'll dive into each one of these a little bit, uh, more here in a second. So, um, this is one of my favorite, the smallest slice thing. Um, I just, I love this. Um, so, uh, how many of you have seen this before? I'm not, I'm not sharing anything new, right? Okay.
- 22:18
So you got your waterfall. Okay, well, by the end, we'll have something useful. In Agile, it's like we got useful, useful, u- but it's different every single time. We gotta rebuild it from scratch almost every time. In AI, it's like, here's the whole thing, and, uh, your job is to whittle it down to, um, the actually useful thing that you're, uh, trying to, uh, to accomplish here. And what's interesting, actually, this image also makes me think of, uh, how Instagram started. Instagram was originally an app called Bourbon, uh, which was basically a Foursquare clone. Uh, so they were having people
- 22:48
check in, uh, to, to things. I think it was, uh, [REDACTED]-related. I don't drink [REDACTED], so I don't know that world very well, but I think that's what it was all... Bourbon, that's out, yeah. I know, I know. I'm just kidding. Um, but, uh, yeah. So, um, they realized after watching ube- user behavior that the only feature anybody really cared about was sharing photos, and so they just gutted everything else and just made it about sharing photos. And they had a pretty nice exit, so good for them. Uh, okay. Uncle, uh, Bob was also on the podcast
- 23:18
just last week, and, uh, he had this experience, uh, as a software engineer. He was working making these little mini computers for, uh, a company that did, like, telephone lines and stuff. And so he'd make these, uh, software for the, uh, guy who would go up the telephone pole to fix stuff. And his boss said, uh, "Have you ever been on the truck? Have you ever, like, seen what it's like?" He's like, "No." "Okay. Well, you're getting on the truck. You're going out," and it... he said it totally changed the way that he thinks about building software for people because that guy's up there trying to use the
- 23:47
software while he's hanging off of this pole. And he's realizing, "Okay, so there's, like, 30 things I can think of to improve this software to make it easier for that guy to use this." Um, so seeing your, uh, users use your software, um, is humbling, uh, to say the least. So in his mind, a product engineer lives half in the technology and half in the customer's house. There's a deeply human side to product engineering. His podcast episode is coming out, I think, tomorrow or the next week. So if you're interested in hearing more from him, or, uh, really any of,
- 24:17
every o- one of these episodes is really good. You should definitely, uh, check it out. Had Grady Booch on a couple weeks ago, and he also feels that engineering judgment comes from technical experience married to human issues. Uh, so being able to attach what the human actually needs to that technical expertise you have, that's what makes a product engineer a product engineer, not a product manager. Um, okay, like I said, we've got a ton of these. I, I'm probably over-indexed on the number of quotes. But
- 24:47
Ronan is awesome, too. It's about launching a product. It's not about the cool API. And I-- in fact, like, there are so many, um, so many instances that I can think of where somebody will come to me and just talk about their solution all the time. Like, "Oh, wow," like, "it does this, and it does that, and it does this, and it does that." And I'm thinking, none of those things matter to me whatsoever. Like it... Uh, okay, that's kind of cool, but I already do this one thing, and it's not, uh, so much better that I want to switch from my current workflow to that thing. And it's just so clear
- 25:17
to me that th-these people are so excited about their solution that they've totally lost the plot of the problem that they're trying to solve for users. So, um, it's really easy to fall into that as an engineer, and s- therefore, a great way for you to stand out as an engineer at your company is to be a product-minded engineer. So you have the technical capabilities, but you also understand what the user is ultimately trying to do and, and what your company is trying to do to solve, uh, their problems. So,
- 25:47
um, a, a lot of the decisions that we make as engineers, um, shape what the product ends up being. And, um, and the reason that is so important is because changing architecture is really expensive. You might think, well, no, Kent, like, it's literally just, like, I don't know, a million tokens, and I can be from, you know, one database to another, or I can be from monorepo to microservices or, or, or, uh, uh, mono- monolith to microservices or whatever. Uh, depending
- 26:17
on the size of your app, maybe that is, uh, the case. But, um, it is really expensive on the way that it impacts the rest of the team and on the way it impacts the user. Um, change is expensive, and and if you say, "Okay, well, it's just a, a couple million tokens," what if I could hire you or I could hire somebody who made the right choice the first time because they understand the product? Yeah, I'm gonna hire the person who can, uh, do it right the first time, uh, and cost me less. So it's all about the primitives. This is Reese. Uh, he works-- Uh,
- 26:47
actually, he just made a startup, um, and really cool dude. But, um, he's, he's saying it's all about having the right primitives that you can build upon. It's really difficult to build a really great solution on top of those, um, wrong primitives. So your engineering expertise matters. Hopefully, that, uh, uh, that... You know what? Dang, I'm... This is going, um... I gotta talk about Michael. Who, who here knows Michael at WorkOS? This guy rocks. Uh, and WorkOS is really cool. But he, uh, WorkOS is an
- 27:17
authentication platform, and he was deciding, okay, so with this new platform, do I make it, like, a on-prem thing? Like, I sell you a license, now you can run this au- uh, authentication thing on-prem. Or do I make it, like, an open source thing with, like, a commercial arm or whatever? And after interviewing and talking with a lot of customers, he realized, oh, no, like, if there is some sort of security problem, then we need to push out a fix immediately, and we can't wait for people to upgrade some NPM package for that. And so therefore, I will make it
- 27:47
a hosted solution. And so your technical expertise and understanding, married to the understanding of the, um, the user and the actual problems, is going to make a significant impact on the direction that you take the, uh, product. Um, oh, gosh. You know what? I'm gonna... It's been way too long since you've actually done anything, so we're gonna skip over this. You need, uh... Hopefully, you get the point. There is a line between product manager and product engineer, and both of them need to really understand the, uh, the actual
- 28:17
problems that the user has. So, uh, one of the... Huh. One of the things that you are doing as a product engineer is, um, building a system for your underlings to be successful in. Your, uh, even, like, years ago, before we had AI agents, you would have a team lead, and their job was r- uh, to make sure that that system that you're working in is very efficient for you. So you've got good testing. You've got good architecture. It's very obvious which Lego blocks to put together to build
- 28:47
out, uh, different features. These primitives are very important. Uh, we are now all team leads over however many agents you're able to run at once, and it's your job to make sure that that playground is a really nice playground and they don't make a ton of mistakes. Okay, I'm gonna skip, uh, ahead. Here we go. Um, and, uh, gosh, I really wanna get you, you all doing stuff here really quick, so I don't wanna miss anything. Oh, hmm. Okay, there are so many things that you miss
- 29:17
if you don't have product understanding. I just, I gotta tell this story. Who knows Don Norman? Ever heard of Don Norman? This guy, legend. He's [REDACTED], he comes on my podcast. Like, that, first of all, is pretty wild. And he has so much... He, th-this is the guy who invented the term user experience when he was at Apple. So, like, yeah, kind of a cool dude, um, and just a delight to work with. Well, uh, how many of you are familiar with the Three Mile Island,
- 29:47
um, incident? Okay, 1979 in Pennsylvania, there's a nuclear reactor, something terrible goes wrong, and, um, and, like, there's a coolant issue and, and whatever. Uh, it was a technical issue, and these operators made some, um, bad decisions based off of the technical problems they were experiencing. So they bring in Don to come and be like, "What happened? Why did these operators make such a terrible decision?" And he looked at the problem totally differently. He said, "You know what? Actually, it was not their
- 30:17
fault. They're smart. Uh, they're competent. The problem was the system. The system was wrong. User error does not exist." That is his assertion. So if you wanna learn more from Don, The Design of Everyday Things is, like, the canonical book. You absolutely should read or listen to that. Uh, and he also wrote Design for a Better World, also a really, um, great initiative there too. So, uh, sorry, I had to jump. Like, this guy is fabulous and really understands users. Okay. So before we get
- 30:47
into, um, talking about the specific app and, and the frameworks that we're gonna get into, I wanted to double-check if, uh, there are any questions. So let me pull those up. And, um, we'll, we'll get to our example, uh, software here in just a second. Uh, we've got one question in here: How many features are too many for an MVP? Um, that depends on, like, w- tons of factors, but I would, I would actually say that that is the wrong question.
- 31:17
The, the question is... A- and we'll get into some of the, the frameworks here in a little bit that will kind of help answer this. But the question really is, uh, what is the minimal thing that you can do to demonstrate that you solve, uh, a problem for, uh, your target audience? Uh, and in, in particular, if you're entering a space that's al- already overcrowded, you want to niche down as well on that target audience. So we'll talk a little bit more about this. Um, but since there are no other questions on here, does anybody have a question that they didn't put on here but they wanna ask?
- 31:47
Go ahead. You can just yell it, and I will repeat it. Oh, we've got a couple. Is there a link to my slides? There will be at the end. Um, right now it's localhost, so I can't, um, but, but at the, at the very end, there is actually a link to, uh, to the slides. So, ha ha, you have to stay, uh, or watch it later. Um, uh, how do you evaluate software engineers in the hiring process for product engineering? Uh, okay. So I... This is a great question. Hiring
- 32:17
is really, really tough, to say the thing that everybody knows already. Um,
- 32:25
having really good technical expertise is, uh, as I explained, is still very important. Uh, I would not be doing coding challenges at all. Uh, that is like ... I certainly wouldn't be doing any coding challenges where I say, "No, you are not allowed to use a, an agent to do this." Like, what are you hiring them to do? Like, how awful is it to work at your company that I can't use an AI agent? Goodness. So no, instead, you're gonna be talking about systems. Um, you're going to be talking about how to, uh... In
- 32:55
fact, I would probably take a couple of the things that we're gonna do here in a second, and I would just ask them, like, here's an example. What are the questions that you're going to ask to figure out what the, the core problem is? And then, and then once they say, "Okay, yeah, you got what the core problem is," now from there, what are the system implications from the, um, that discovery of the core problem? That would be how my, my interview for that would go. That's a great question. Um, I lost it. Uh, okay. When do you stop talking to
- 33:25
customers and start building? Uh, I think that you do both constantly, always. You, you do not stop. Uh, it's, it's a feedback loop that continues to go. We'll get into this a little bit, uh, as we go, um, further on in the, uh, workshop here as well. Uh, not only what to build, but how to get users to use it. Traction is the new moat. Um, okay. So I don't actually talk about this specifically. Distribution is, is definitely a difficult problem. Uh, you just get Theo to talk about it on his YouTube channel. That's, that's the answer. No, just kidding. Um,
- 33:55
yeah, that's a... that actually is a really difficult one. Uh, I have heard that people will just spend an unreasonable amount of money on, uh, marketing and advertisement, and that seems to work for them. So, um, yeah, this is not something we're, uh, really going to, uh, to talk in, uh, talk about, and I'm not prepared to answer. So sorry to not be, uh, all that helpful with that one. Uh, okay. And we... Oh, did I just mark one that was... that I didn't answer? What did that question say? I think it moved on me. There's a bad user experience right there. Somebody who works at
- 34:25
Slido can go fix that. Um, okay. Uh, I'll, I'll answer like three more questions, then we'll move on. Would you recommend product managers stay in product teams and product engineers with it? Oh, goodness. Uh, as far as organization is concerned, um, often applications end up, uh, looking a lot like their org chart. So, um, I would... Huh. Um, I, I definitely feel personally that your product, uh, manager should be very integrated, uh, with engineering. That
- 34:55
communication layer should be very tight. Um, and your engineers should also not be siloed from other engineers in the organization. Because one important, uh, thing for a product engineer to do is not only look at their slice of the application but also step back and see at least their neighbors. Because if you don't, then you'll end up building things that are already solved for you by other primitives that are within the organization. And, uh, and so you're just rebuilding those same things. So you need
- 35:25
to at least go one neighbor out and one neighbor up and down. Uh, and this is actually the same with, um, programming languages. Like, if you're a React developer, then you need to understand how JavaScript and the browser work so that you can use React effectively, even though you're not necessarily using all of the features, um, that are included there because the framework is doing that for you. So it's the same sort of idea. Um, tip: Highlight the questions to bring the focus... How do I highlight? Do I just click on it? All right,
- 35:55
we're doing a product lesson right now. That is a, a useful tip. It seems like a good i-... Oh, is that what that is? All right, there we go. User error. No, it's not. It doesn't exist. Uh, okay. So if I highlight it... Okay, that's cool. Should all engineers be product engineers? Um, well, in my estimation, if you're not a product engineer, uh, if you don't have product sense or design sense, it's... it's gonna be really easy to replace you by a, a, a, as the
- 36:25
agents get, like, continue to get competent. I don't wanna be, like, alarmist or anything, like, you know, whatever. But it does seem like if all you're doing is listening to somebody tell you what to do and then turning that into a code implementation, that part seems like it's pretty replaceable. If you don't understand the product, uh, then the system you build will not solve the user's problem well. Okay, I think we will come back to these questions, uh, as they're voted up a little bit more later. So- After,
- 36:55
um, quite a while into the workshop, we're gonna s- finally talk about what the workshop is gonna, um, be talking about. So, um, what I want you to leave with is, um, frameworks for early idea validation. How many of you have heard of the Mom Test? Okay, sweet. So we'll be talking about that a little bit. Framing user request jobs theory. Anybody? Okay, sweet. And prioritizing software changes. Kano Model. Okay. Yeah. So, uh, one of you raised your hand three times, so, uh, I, I invite you to come up
- 37:25
and you can talk about this. No, just kidding. Uh, okay, so here we go. This is the dangerous thing. What, uh, is the idea that we're going to be framing things around? And this is a risk. I've never given this talk before, so this might go really, really poorly. Um, hopefully not. But, uh, let's look at what we got. The top voted, ooh, with fire, is an app that gets people into conference workshop sessions. Uh, uh, yeah, okay. I, I literally am going to do that. Um-
- 37:55
Okay. What should we call it? What's, what's our short, um... How about Lightning Lane Workshop? You know, call back to, uh, to Disneyland, right? Lightning Lanes? Yeah. Lightning Lane- Just get in. Oh, oh, there. That's so good. Just get in. Oh, I love that. Just get in. I don't know. Like, are, are some of you thinking, like, weird things? The Workshop.
- 38:24
And sick people.
- 38:27
I just heard everybody laughing. I'm like, "Why are they laughing?" Oh, okay. Gosh. Okay, so let's talk about early idea validation for the Just Get In The Workshop app. God. How do we validate this? All right, throw out some questions. What are questions... Uh, you're at a conference. This is the perfect place to talk about the Just Get In The Workshop app. So you're, you, you've got all these people. You're standing in line and you're like, "God, this is so awful." What are the questions that you're asking people to validate that your idea is a good idea? So just throw them out. Raise your hands if you want to.
- 38:57
Yeah.
- 38:59
Why do you need an app for that? Why do we need an app for that? Okay, that... So ask them, is it a problem? Is this a problem? That's... Yeah. How long have you waited? How long have you been waiting? Yeah, okay. That, that's getting really close to a really good question. Other questions. Why are you waiting? Sorry, what? Why are you waiting in line? Why are you waiting? Oh, okay. Yeah, that's a good question. Yeah. Should there be a pre-registration? Oh, could there be a pre-registration? Like- Yeah ... yeah, wouldn't it be better if this was a pre-registration thing? Yeah. When was the last time you were stuck waiting in
- 39:29
line? Oh, you've read the Mom Test. All right, all right. He said, "When was the last time you were stuck?" Yeah, that's, that's... Okay, okay. What, what else do we have? Why are you only- What's the max time you're expected to wait? Uh, sorry, say that again. What's the max time? Oh, what's the max time we're expected to wait? Yeah, okay. Are there any open seats in the room? Are there any open seats? Yeah, okay. Okay, so these are some questions that you might think, okay, if they... W- what you're looking for, uh, or what we naturally are looking for is somebody to say, "Oh,
- 39:59
yeah, this is a huge problem for me. I definitely want... So I would, I would spend, uh, just untold sums of money to be able to get in the workshop really quickly and easily." I, I think you had another question. I got another one, like, uh, how does your- what, what was your experience while waiting? Oh, yeah. How's your experience been while waiting? So you're trying to, like, get a sense for, um, for that overarching experience. Yeah, and that can be a good input into the solution. A- Solved problem. Is this a solved problem? Oh,
- 40:29
yeah. Like, have you heard of an app that could really have sped this process up? I saw somebody holding up the Mom Test book. You literally have it in your... Unbelievable. That's awesome. Uh, I, I also carry my Mom Test book with me everywhere I go. Uh, yeah, so this is the Mom Test. You don't wanna ask users to evaluate your idea or diagnose the problem for you. This actually, this is so common, and it's so hard. Because when you think of an idea, you start thinking, okay, so we
- 40:59
will, we'll make it, uh, a mobile app, obviously. They have to install it on their phone. Oh, wait, no. Uh, maybe, like it's a one-time use thing, so we'll also make a, a website and then we're going to, uh, integrate with all of the different, uh, providers of all these different things and, uh, and so then you start talking with people and you say, "Would it be easier if it was already integrated with the app, and maybe the conference had a link to it, or maybe it was integrated..." And so you're talking about the solution to, your specific solution with this,
- 41:29
uh, potential possible user, uh, before they've even agreed that this is, uh, a problem worth solving. Okay? Um, so here are a couple of the weak questions that may sound a little familiar. Would you use this? Is it a, is this a problem in the first place? What is the problem? Help me define the problem. Some better questions are, tell me about the last time this happened, and what did you do instead, and what did it cost you to do that? Now, for ours, none of us, I don't think, would actually... Uh, I
- 41:59
don't know. You all paid to come here to the conference, so there is some amount of, like, I'm waiting. I'm missing the first 20 minutes of this talk that I actually paid to attend. Don't get mad at [REDACTED]. Uh, these, these things happen. [REDACTED]'s a friend of mine. Uh, but, uh, but yeah, like, the, the questions of what did you do instead, what is your workaround, and how much did it cost you because a solution like this doesn't exist? You do not talk about the solution at this stage. It's so hard not to, but you don't.
- 42:29
Instead, you start with, um, what did you do before? What, what is the past? You do not want to ask them future questions. Nobody can tell the future. You don't wanna say, "Would you pay? How much would you pay?" Whatever. That... Often, what that turns into is somebody's like, "I kinda wanna get out of this conversation, and I'm just gonna say yes to everything that they say." And, uh, in the book, they call this complimenting, so you're just, you're res- you're asking, you're fishing for compliments. Uh, so Don Norman said, "Don't ask somebody what's the problem, 'cause they'll tell you the
- 42:59
symptoms." And often they will actually tell you solutions as well, and their solutions will be just as ill-formed as your own. Uh, and maybe even more so. Uh, so you don't want to just ask generally what is the problem. You wanna ask about specific behavior. So tell me a little about the last time this happened. What happened? Give me specifics. Uh, who was involved? What made it annoying enough to remember? Now, our example is kind of, um, kind of funny because you... If you're in line, you're literally experiencing the problem right now, so it's a little visceral. But
- 43:29
most of the time you're not, um, like, your idea to solve somebody's problem, they're not actively experiencing that problem right at that very moment. So it's not necessarily gonna be so much, "Well, tell me the last time it happened." "Well, it's happening right now." Like, maybe it is, and that's a pretty good signal. Then you're gonna be able to get some really, um, direct answers to that. But often you're gonna have to let them think back about, like, "Oh. Yeah. Why, why was this so bad, and who was involved? What made it annoying enough that I can even remember?" If they can't remember, that's a
- 43:59
signal. And it's not the one that you want it to be, but you want that signal, that's for sure. So, uh, you wanna know what they did, uh, do instead. So what are the workaround? Oh, whoops. Ah. Back up. Uh, how much does it cost? How often is this happening to them? Uh, how much effort are they putting into solving this problem already? And that is gold. That is, um, a, a really strong evidence that this is a problem worth solving. If they're already putting a bunch of effort and paying a bunch of
- 44:28
money to solve this problem with some workaround, um, that's a really good opportunity. Uh, if they are like, "Yeah, I, I don't remember the last time. I know it's happened. But yeah, I don't really remember. Uh, and I just... I kinda gave up." That, that, that's a pretty good signal too. Like, that's a signal that they're probably not gonna pay, uh, and, and change their workflows or whatever, uh, to get a solution. Now, um, the, uh, yeah. Uh, the, the point here is that as software engineers, we've gotten kind of used to just, like, sitting in our
- 44:58
corner. Well, some of us. I'm kind of an extrovert, so I, I like to talk to people. But, uh, some of us are just like, "Yeah, I got into software engineering 'cause I never have to talk to anybody, and I can just, like, take my Jira ticket and turn it into implementation." I, that... I'm sorry. That's not the way that software engineering ever was. The best software engineers were not that way. Um, I mean, you could churn out some pretty amazing code, don't get me wrong. But the software engineer who really talks to people and understands their problems is the one who's gonna build the system that can solve those problems better. Uh, and, uh,
- 45:28
one, one thing that Michael did... Actually, in our conversation, he was talking to so many people, validating his idea before he, uh, built WorkOS. Uh, and he would ask them, what are they losing out by not having a solution to this? What's the cost of not doing that? And that worked out for him. So the output is, uh, i- like, actual evidence rather than somebody just saying, "Oh, yeah, that's a good idea." And this is why it's called the mom test, because your mom, of course she loves you, and she's gonna be like, "Yeah. Of course. This is wonderful. I would absolutely u- use it, and I would..." And so
- 45:58
now you have two users, yourself and your mom, and, uh, or dad or whatever, and, and that's wonderful. Good for you. Um, so, um, here's a, a fun story from Wayne. Uh, he, he lives down in [REDACTED]. He works for a, a real estate, uh, company down there. And they were building a product, uh, it was kind of an experiment in, in this bigger, uh, organization that he was on a team that was building this product. And in the process of that, they ended up building a really high-scale thing,
- 46:28
and they, in fact, decided, "What if every single person in Australia logged in at the exact same time? We need to make sure that we handle that." And, uh, they spent a year working on this. It was, like, microservices, the whole thing. Um, and after it was 1.2 million U- uh, Australian dollars that they spent on this, which is, like, almost 900,000 US dollars, um, they, uh, ended up, like, nobody used it. Not a single person. So, like, not only did they not have to handle every
- 46:58
person in Australia, they, they didn't even have to handle one person in Australia. Nobody ended up using this thing. And so he told me that, uh, something he learned from that is, "We probably could've launched something in two weeks, not a year, and done the integration manually." Integration's always the most painful part anyway. Uh, and we could have tested the market, learned a lot of things very, very quickly, and not invested a whole team to work on this. So project validation, really important, and it's, again, not just the PM's job to do this. It's your job to, uh,
- 47:29
really understand, okay, so this... And, and we're not just talking about startups here either. This is a product feature that you've been asked to do, and if, if that feature request doesn't come with some sort of validation of why this exists, then you as a product engineer need to go back to the PM and say, "I need to understand a little bit more about why this exists." For not only to validate that this is gonna be worth engineering time, um, but also, uh, for some of the other questions that we're gonna talk about here in a second.
- 47:59
But part of that is, um, understanding what, like, where this request is coming from will help you shape the system that you use to build it. Is this just, like, a one-off experiment, or is this something that's gonna be core to our business? I'm going to make different technical trade-offs based on that. Like, how much investment do I need to spend? A- should I spend a year working on this? Is this, like, the business will go out of business if we don't have this? Okay. Well, I'm gonna, uh, spend, uh, a little bit more time and attention to detail, uh, while still, like, balancing
- 48:29
the, um, MVP style, like, let's just get some validation.
- 48:35
Okay. I'm really glad I grabbed that water . Uh, yeah. This is something Michael did as well. Very easy... Yeah, gosh. This is so re- relatable. It's very easy to convince yourself that something is a good idea just sitting in your room by yourself, hacking on it for weeks and weeks and weeks. Uh, so you need to show it to people. Uh, I... How many people relate to that? Just, like, yeah. Uh, it's fun, right? Like, why you gotta rain on my parade? I'm having fun doing this thing. And then you go show it to somebody, and they don't get it, so you don't wanna talk
- 49:05
about it, and you just wanna work on it yourself. That was a signal right there. Like, if they don't, if they don't get what you... why you're so excited about it, then that, yeah, you wanna pay attention to that feeling. Um, and then we need to close the loop with user feedback. This is part of, like, shrinking things down. So, like, the validation of the idea, that the... and the mom test questions we talked about earlier, that is, like, telling you whether it's worth, um, working or solving the problem in the first place, and now we're getting into how do you know
- 49:35
whether your solution, uh, is doing that effectively? And, uh, Ruben says that user feedback is how you do that. So, uh, and also what Michael is doing and, and, and, um, Wayne as well. So you try to get something, some prototype to those early users. Early adopters are, like, totally fine, es- especially if, if the solution that you build has a ton of gaping holes, then yeah, they're gonna be, like, pretty, um, uh... Their reaction to that is gonna tell you a lot. If there are a lot of gaping
- 50:05
holes, but, like, it gets them, like, almost there or, like, just gets them over to what they're trying to do, and they're still excited about it, that is, uh, very significant. If it doesn't quite get them there, and they kinda give up early or something, and they're like, "Ah, I don't know. I could just... You know, I had a couple of these, you know, paper cut issues, and so I stopped," okay, that's a really good signal too. The, the, um, problems that are really worth solving are the ones where people will just run through the glass cutting themselves, and they're like, "I made it." Now that's what you're
- 50:35
trying to, uh, to go for. And, and getting early user feedback, um, through prototypes helps a lot with that. So why product engineers need the mom test, because that helps you translate what the PM is saying, um, into the shape of the system that you build, uh, and select the appropriate architecture. Um, especially understanding what do users do now or what, what do our potential customers do now to solve this problem, um, can help you determine the architectural shape. Okay, so, um, there's, uh, a lot of opportunity here, and, um, there's gonna be a lot of
- 51:05
downtime, so we need to, uh, pay attention to how we do migrations or whatever. Uh, okay. And yeah, it's very easy to end up building the wrong ab- ab- abstraction without that context. So I'm gonna s- pause really quick. We'll see what questions you all have. Um, so far, the most upvoted ones are the ones I'm gonna focus on. Um, and back here. Here we go. So three main things that d- Uh, let's just highlight that. There we go. Three main things that distinguish a product engineer from a product manager. Do we need
- 51:35
both? Ooh, the three main things. Um, so yeah, kinda talked about this a little bit, but I would say that the product engineer is the one that's... I'm not sure if I'm gonna get three, but I'll just say a bunch of things. Um, product engineer is the one that is actually in the code. You may not necessarily be looking at the code anymore, but you are thinking about the system, and you're thinking about the, uh, data types that are available, the primitives that you have available. Oh, we do have, like, a cron job system or queue, but there are some problems with that. And I... And you understand what those constraints are
- 52:05
and, and what the limitations of your current infrastructure are. You understand what the available, uh, primitives are that, uh... And so when a request comes in, and you understand what's trying to be solved here, you can categorize that into whether it fits in the existing architecture or if you need to expand beyond that. And understanding, um, that, um, the ultimate user goal or the job to be done, the foreshadowing, um, that is going to help you decide whether w- we should actually expand the system with a couple more primitives or if you
- 52:35
can reuse existing ones. Um, that's probably the, the number one thing that distinguishes a product engineer from the product manager, is just that technical expertise and understanding of the system. Uh, which a product manager could probably just talk to an AI about over and over and over again to, like, kind of ramp themselves up, but they should actually be doing other stuff. Um, and so yeah. I would say, um, the... to answer the question, do we need both? This is actually interesting. So, um, I am... Oh, sorry. I, I am
- 53:05
self-employed, so I do not have a product manager or a product engineer or a CFO or what... Like, I just work for myself. I am all of the things. And you start a startup, often it's just, like, three of you. Sometimes y- all three of you are technical, or maybe one of you is the, the business person or whatever. And as the company grows, you eventually start, uh, divvying out those responsibilities, and you have multiple jobs. So yeah, um, do we need both? It highly depends on where you're at in the process of, uh, your scaling of your business and, and whether you ever want to get
- 53:35
there. So, um, I think that now, uh, you can actually build a much bigger business without splitting out into a much bigger organization, uh, than you used to. Uh, great question. Okay. How small do you think teams... Oh, huh, literally just said that. Um, pretty small. I, like, I don't know how I can give you a more specific answer to that. Uh, okay. How would people get promoted from junior to senior in the age of AI, AI? Um, well, huh.
- 54:05
I'm not sure how to answer this question. Uh, I, I think, uh, juniors and seniors are both, um... Like, w- we're all ultimately trying to do th- the same thing. Even all the way back before AI, um, uh, the junior was just trying to emulate the sen- senior as much as they could until they su- suddenly or over time become a senior engineer. And I think that nothing about that has changed. Uh, as far as, like, actually getting promoted, that's just gonna be a conversation with your, uh, with your person. Uh, I should actually
- 54:35
stop really quick because I, I'm just noticing people walking out, and, um, my timer went to zero. This session is a two-hour session. It's just back-to-back. And I'm planning on just going straight through. But if there are other sessions you wanted to see, you're welcome to stand up and leave. I will just take a mental image of you and frown at you in the hallway later. Just kidding. I won't do that. You're, you're welcome to take off if, if there was another session you wanted to go to, um, because there are so many good ones.