The Web Browser Is All You Need

Talk transcript

1. 0:00

   [on hold music] Hey, everybody.

2. 0:16

   I'm Paul. I'm the founder of Browserbase, and I am obsessed with browsers, specifically one type of browsers, headless browsers. And I'm here to talk about how the browser is all you need.

3. 0:27

   It's not attention, it's not MCP, it's the browser, or specifically the Browser MCP Server is all you need. And I'm gonna try and keep it light on slides. We only have a hundred to get through in twenty minutes, but you and I, we can do it together.

4. 0:40

   Uh, just stay with me for twenty more minutes, okay? So first of all, a little about Browserbase. This is what it looks like to build your own browser infrastructure.

5. 0:47

   It's messy and it breaks all the time. With Browserbase, we l- can let you run thousands of headless browsers in the cloud for easy to control. And you might be wondering, like, "Well, why do I need a browser?"

6. 0:57

   Well, every AI agent needs a web browser. That's the point of this whole talk. So of course, you know, I can talk to you about how to scale browser infrastructure and all that interesting stuff, but that's too pluggy.

7. 1:06

   We'll save that for later. But I do know quite a bit about, you know, how customers are using browsers as part of their AI agent applications, specifically with MCP.

8. 1:15

   We have the most popular browser automation MCP server out there. And the reason why people choose Browserbase to run their headless browser MCP is that we make it really nice and, and really can help you scale your infrastructure.

9. 1:26

   And when you think about, why do I need a browser? Well, I like to think about two things. You have AI agents and the legacy internet. You know, the DMV is not gonna have an MCP server anytime soon.

10. 1:37

    My barber shop is not gonna open a GraphQL API for me to schedule a haircut. As much as I keep begging John to do it, uh, he's got better things to do.

11. 1:47

    So if we want AI agents to interact with the rest of the legacy internet, they need a bridge. And I really do believe that the browser is that bridge between AI and the rest of the internet.

12. 1:56

    And this is the unsexy internet, I might add. It's, it's the internet that's not gonna get a lot of attention. I've seen countless flight bookers and countless restaurant pickers, and I have not seen anyone do the thing I need, which is a Delaware franchise tax filing agent.

13. 2:10

    Any founders in the room have done that before? Not super fun.

14. 2:14

    And I think people use a lot of acronyms these days [laughs]. You know, you have MCP, you have A2A, OpenAPI. But if those aren't available, you can just do what could be considered the, the dumb thing.

15. 2:23

    You could just use a website. And websites are out there. There are plenty of them. There are billions of websites. And when your user is gonna prompt your agent to do something, you might not always have a first-party integration available.

16. 2:34

    That's where a browser is kind of the integration of last resort, the path that you can take your agent down if you don't have something, a primary integration already built in.

17. 2:43

    And you might be wondering, "Well, okay, Paul, cool, I get it. You're beating this thing dead. Uh, AI agency and the browser, well, how do they control it?" Well, you can think about web agents and browser tools.

18. 2:52

    First of all, you know, what is a web agent? Uh, I wanna keep this technical, as this is a bunch of AI architects, right? You know, web agents, we've, we've heard about them for a long time, take a model and then have it generate some code to control a browser by generally parsing the DOM of the page,

19. 3:06

    the HTML and the CSS. WebVoyager was early here. Adept did a lot of really cool stuff here with their Fuji models. OpenAI had Operator, Proxy by Convergence, now Salesforce, Hcompany.

20. 3:16

    Everyone was kinda doing a lot of this stuff last year. And we really got to see a lot of web agents in production, uh, but they were still early days, you know?

21. 3:23

    And WebVoyager was first, you know. It was taking screenshots of a page. It was using chain-of-thought prompting, and then from that, it was saying, "Click the button at this coordinate."

22. 3:32

    Uh, sometimes they're doing these things, we'll talk about in a second, called, um, labeling on top of the page. But I think it's pretty cool 'cause we haven't changed that much from this.

23. 3:39

    There's really are two different types of web agents. There's vision-driven agents. These are ones that predominantly use screenshots as context for the model. They might do some marking up of the screenshot to indicate what box to click on.

24. 3:50

    Um, or there's text-based web agents who predominantly use HTML as the context of the model. Um, both have different approaches, pros and cons. Text web agents use XPaths and Playwright code.

25. 3:59

    Some may argue it's more repeatable. Vision models can be more accurate on more complex pages. Uh, there's trade-offs here, and they're, they're, you know, it really d- does depend on what website you're trying to automate.

26. 4:08

    Here's an example on a vision agent using set-of-marks prompting. You can see these little boxes here where you're marking up what you should click, and the agent will, or the model will turn and say, "Click the box label twenty-five."

27. 4:20

    That's gonna help you out. And on the DOM-based agent side, there's also ideas of, like, taking HTML, and how can we transform HTML to make it more reliable for web agents?

28. 4:29

    So the accessibility tree is something that's built into every page, uh, in a lot of applications, where you can take a different structure of the same information and condense it down where you get the same layout, but without all the extra div tags and classes.

29. 4:42

    So we have vision-based agents and DOM-based agents. And there's now also computer use models, which are kind of like the next step here, where, well, let's train a whole model on this stuff.

30. 4:52

    Previously, we were just using, like, the stock image models, but now we can train that model on these things called web trajectories. And I won't go too much into this.

31. 5:00

    There's a lot of great papers out here. I'd recommend this paper rec- linked down here about, like, web trajectories and how you can generate them to do RL, to teach models how to not just pick the right button on the page, but how to reason across multiple pages about the right path to take.

32. 5:12

    But all in all, there's just a lot of innovation here happening on teaching AI how to browse the web, and this stuff is getting good. It is working, and you can use it today to help add some sort of extra functionality to your applications if you're making the right choices.

33. 5:28

    And I, I will kind of add, you kind of want to think about if you want a web agent or a browser tool. And you may have never heard of, like, what the difference is here, but there is a difference in my head.

34. 5:36

    Like, a web agent is kind of like one prompt to many actions. I think OpenAI's Operator is a good example of this. You say, "Hey, Operator, go, you know, file my Delaware franchise tax."

35. 5:46

    You give it some, some prompt, it's gonna go take many actions, and if you give it that same prompt twice, it might take two different paths to get the task done.

36. 5:54

    Web agents are good. They're like little cockroaches. You just-- They're just gonna keep trying to find a way to complete your task. Um, but they're a little bit more nondeterministic because the reasoning is in their control.

37. 6:03

    Whereas like browser tools are like one action or one prompt. You say click the sign-in button, that thing's gonna click the sign-in button. You ask it to purchase the Amazon, uh, item that you want, that's a series of multiple steps that really might be more suited for a web agent.

38. 6:16

    So we have a framework called Stagehand that we think is the best browser tool. Um, and it really does depend on what you want. If you know what your workflow's gonna be with some high-level steps, you can actually use a browser tool to take those steps and translate them into reliable web, web actions.

39. 6:30

    If you don't know what you're gonna do, if you, if you don't know what the prompt will be, a web agent more generically might be the right fit. But I do believe your AI agent needs a browser tool.

40. 6:39

    Another thing I'll add is, like, you may wanna think about with MCP, what types of servers are you integrating? So a vertical MCP server with something like Linear, where it's going to give you options to control specific things on a specific task, like create a Linear ticket, create, uh, assign someone to the Linear ticket.

41. 6:55

    Whereas like a horizontal MCP server, you're going to have some sort of primitive that can do many things. You know, for browsing, we view it as a horizontal MCP server.

42. 7:04

    You're exposing primitives like click a button on a page. Now, that page may change. There might be many different pages. But when you have a horizontal MCP server, with one server, you have the opportunity to automate the whole web.

43. 7:16

    And these, like, you know, you will see vertical MCP servers, which are more direct tool calls, as, like, an important part of your agent. I'm not saying we replace those with browsers.

44. 7:24

    If you are interacting with Salesforce, you probably should just use the Salesforce MCP. You don't need a browser there. But if you're interacting with custom bespoke MCP built by large enterprise that doesn't-- Uh, sorry, a custom bespoke CRM built by a large enterprise that doesn't have an MCP server, you don't have to go reverse engineer APIs.

45. 7:42

    You can use a browser tool and a browser MCP server to go out and automate that. Is that making sense for everybody so far?

46. 7:50

    Yeah.

47. 7:50

    Okay. Thank you, guys. I'm hearing some heads nodding. Okay, a few other notes on like MCP and, and all these, like, concerns I have here. Um, I, I do think that MCPs are gonna have to pass compliance, and dynamic tool discovery, this idea that you can plug in your agent to an infinite toolbox of MCP servers, is

48. 8:08

    going to be hard for CISOs to get on board with, right? You're gonna wanna be able to select which MCP servers make the most sense. And with a browser or a horizontal MCP server, this could be browse, it could be email, it could be anything, uh, you really only have to onboard one MCP server as opposed to

49. 8:23

    do an MCP server for each individual integration. Secondly, like, yeah, that's the most important way. Secondly, like evals. Most, most benchmarks are fake news. Just wanna let you know.

50. 8:33

    Especially when the company putting out the benchmark is the one that's also ranking themselves. So I would be very, uh, critical of public benchmarks of any web agent you see out there or any model you see out there.

51. 8:43

    You really need your own evals. My friend Ankur, who runs BrainTrust, he supports our evals. I really like them 'cause then we're able to say, "Hey, actually, for our web agent we're building, which models are the best model for this web task or for this certain website?"

52. 8:57

    And you can get really intelligent and honest about what you actually need to do to automate the web and which model is the right choice for you. Finally, I think you need observability.

53. 9:07

    If your AI agent is controlling a browser, you need to be able to see what happens in the browser. Browserbase bundles this out of the box so you can build your own, you know, taking screenshots, recording history, recording actions, making sure that you know exactly where your agent went to and why is important because let's say your

54. 9:24

    agent is gonna go buy an Xbox and it buys you AirPods, you wanna understand what prompts went into that, what page paths it took, and really break that down.

55. 9:31

    At Browserbase, we include this in every browser. Its sessions are recorded, logs are available. It really makes it quite easy. Okay, so that was, like, a lot on browsers, and I got a ton of time left, but I'll end with this one point.

56. 9:44

    The browser is the default MCP server for the rest of the internet. If you need to integrate with something, whether that's with MCP, an API, and there's not something available, you should really consider including a browser, 'cause a browser is all you need.

57. 9:57

    And since I'm doing so well on time, I'm gonna do some live coding, 'cause I feel like there hasn't been enough live coding in this room today. So I'm breaking it out, and let's pull up Cursor and bring it over here, and it's gonna be so hard to see, but let me try.

58. 10:10

    Oop. Give me this, please. Love. All right. So we have Cursor right here in all of my screenshots. I'm a screenshot hoarder. I apologize. So I pulled up the Cursor MCP server.

59. 10:26

    Uh, sorry, the Cursor controller, and I've written a prompt. It says, "Create a new browser session, navigate to sfpca.org, close out any modals, and find a dog for adoption in San Francisco.

60. 10:36

    Return the URL." We'll click enter, and I may have to jump to a browser here really quickly. So you can see it's calling the browser session tool. It's generated a browser session right here.

61. 10:48

    And you can look how it's actually making these individual tool calls on the page. If I pull up the session at the same time and scoot this tab over there,

62. 10:57

    we can see in parallel the browser is loading. And as our MCP server is making these tool calls, right now it's trying to close out the modal, it's navigating the website.

63. 11:07

    And thinking about how, like, there's a reasoning model here that's deciding, "What should I do? What should I call?" Uh, these tools all are exposed. And now, if we go here, we have been given a dog with this URL, and one of you will be going home with a lucky dog today.

64. 11:25

    All right? Can I get a drum roll, please? [drum roll]

65. 11:28

    All right. That was so half-hearted, but thank you. Uh, give me a dog. Yay. [clapping] There we go. Oh, they really want that 200K match. Uh, this campaign's been running for a while.

66. 11:40

    I feel kind of bad offensive today. Uh, but that's a good example of what will happen when you're building these web automations. Sometimes modals will pop up. Sometimes things you aren't expecting may happen on the webpage.

67. 11:50

    You need to have an AI agent driving the page with primitives so it can react to any sort of weird changes that happen. And hopefully, you can integrate with things that aren't gonna be AI native.

68. 12:00

    To me, the most important problems to solve in AI right now ... are the unimportant boring problems, and they're gonna require, you know, intelligent, you know, engineering to both bring the best models to the unsexy problems.

69. 12:15

    And when we talk to customers at Browserbase, they're not just Perplexity or Clay or, uh, Camure. They're also a [REDACTED:age] dairy trucking company who has never hired an engineer in their 55 [REDACTED:age] until this year, and the first thing they did was use Browserbase to automate a really painful ops workload, workflow.

70. 12:32

    So if I can somehow pull my slides back, I don't know if I'll be able to. I might just have to move this bad boy over here, and we'll just go for it live.

71. 12:39

    Uh, yeah, screw it. Well, let's do this. Okay, so if you do wanna try a browser MCP server, it's available today. You can actually go ahead and scan this QR code and sign up or use this to try adding some sort of automation.

72. 12:52

    It's really easy. If I can use it, you can use it, I promise. I'm gonna pause. I'm seeing some photos being taken. Amazing. Great. And, and finally, you know, if you are looking to, uh, join a company that's growing quickly, Browserbase's been around for a year and a half.

73. 13:05

    We're 30 people. We're backed by some really great investors, and we would love for you to come build the future of automation with us. All right. That's all my time. [clapping]

74. 13:13

    I might have a minute for some questions or two, if that's okay. Yeah, if there's any questions, uh, happy to, to take them. Otherwise, uh, thanks so much, everybody.

75. 13:21

    Any questions?

76. 13:22

    Do you use specific model-

77. 13:24

    Hold on. Maybe we do hands, if that's okay. [laughs] Uh, we'll go, uh, front and then back. Yeah, go ahead.

78. 13:28

    Do, do you use specific model, uh, do you use specific model for navigation, like it's your own model or, uh, like, uh, when the browser is-

79. 13:37

    Mm-hmm

80. 13:37

    ... scrolled and so on and so on-

81. 13:38

    Yeah

82. 13:39

    ... or you utilize different models?

83. 13:40

    We are model agnostic, so Browserbase is just the infrastructure for running headless browsers as well as the frameworks and tools. We have an MCP server. We have a framework called Stagehand, which is like Playwright but better.

84. 13:50

    Um, you bring your own model, and you may wanna choose different models based on your conditions. If you are doing HIPAA and you need like zero data retention on OpenAI, you might wanna use one of their models.

85. 13:59

    So you bring the model, we bring the infra.

86. 14:01

    So with that example you just showed-

87. 14:03

    Hold on. I'm sorry. I'm just gonna... Go ahead. Say it again. You can just... We just skip it.

88. 14:08

    The example you just did.

89. 14:08

    Yeah, the example we just did-

90. 14:09

    Cursor agent versus-

91. 14:09

    I used, uh, yeah, the cursor agent mode. I think it's, uh, 4.0 max. Or sorry, Claw- it's probably 4.0 Sonnet right now is what I have baked in there.

92. 14:18

    Go ahead. [laughs]

93. 14:19

    How do you-

94. 14:19

    Love the questions. Keep 'em coming.

95. 14:21

    How do you guys manage, uh, anything that requires human in the loop? Because we have to deal with lot of legacy infrastructure, but it's financial data, so an advisor wants to review it before it goes into a financial planning tool or something like that.

96. 14:34

    How do you bring in some sort of human in the loop interaction or to a browser, A? And B, if, um, uh, uh, because this is financial data, we also want to, um, give users a clear view of what the agent has done.

97. 14:49

    Is there a way we can send that information even if they couldn't interfere in the process?

98. 14:54

    Yeah. I, I was just gonna hop over the slide. So not only do we have recordings available, and they can be turned on or off depending on data sensitivity, you can embed these recordings into a user-facing application and show your user what happened.

99. 15:04

    We also have this feature called the Live View, where you can embed an iframe and show exactly what's happening in the browser. And better yet, if someone wants to, a human wants to come in and click and type in on the Live View, they can do that as well.

100. 15:16

     So it's not just for browser automation. It can be a browser co-pilot, and it's an iframe, so it's a browser inside a browser, which is kind of fun to see.

101. 15:24

     Yeah. Thank you. I think there's a question over here. Yeah, no, we'll go to you guys next.

102. 15:28

     Um, so my question is kind of two parts. One, have you dealt with CAPTCHAs at all, and is there-- And do you see like future websites kind of using similar strategies to defend against like automation tools, that it's only users can use their website?

103. 15:42

     Yeah, CAPTCHAs. So for the longest time, there's never been like good bots on the internet, and CAPTCHAs were built to stop all bots. But now there are good bots and bad bots.

104. 15:51

     And at Browserbase, we do offer CAPTCHA solving and proxies as something that's needed to browse the web. We have customers that use CAPTCHA solving against their own CAPTCHAs because they can't get their SecOps team to like get through a, a good way to bypass that.

105. 16:04

     However, I think longer term, you know, my friend Michael at WorkOS was just talking about CAPTCHAs. Agent authentication is the path to avoid CAPTCHAs. Most CAPTCHAs we see at Browserbase are when someone's logging in.

106. 16:14

     Once you log in, you know who the agent is. You know who they're acting on behalf of. And I'm really hopeful that solving CAPTCHAs at Browserbase is a short-term solution as we figure out how to do agent authentication on the internet longer term.

107. 16:25

     But down to talk about that afterwards. Come find me.

108. 16:28

     I got, I got one minute 42 seconds. I'm gonna... I'm holding us to it. Yeah. [laughs]

109. 16:32

     Uh, so during web navigation, like I think my question was related to CAPTCHAs, but during web navigation, if, does the browser actually detect it's a robot that's doing the navigation, and does it increase CAPTCHA coming up?

110. 16:45

     Yeah, I, I think that the way that CAPTCHA detection works is often based on your behavior. And what we advise our customers is like, "Listen, you know, even though we provide the best stealth browsing features necessary, in the end, if you're a bad citizen of the internet, you are going to get blocked."

111. 16:58

     It's an inevitability. You can see this on LinkedIn. If, if you have an agent that's using LinkedIn, LinkedIn measures how many actions you take per minute. And if you're violating that, you know, you're going to get stopped.

112. 17:07

     So we advise our customers is like, "You need to be a good citizen of the internet first. You need to try and obey robots.txt. You need to be careful what you're doing.

113. 17:14

     And if not, like you're going to have a really hard time." And, and no matter what we do at Browserbase, we can't stop that. We can help with the simple things, but if you're doing something that's against the law, unethical, we don't really want that on our platform. [upbeat music]