Beyond APIs: How AI Web Agents Are Automating the "Long Tail" of Knowledge Work

Talk transcript

1. 0:01

   Hi, we're Arjun and Bhavani, and we're here to present to you Retriever.ai. Retriever.ai is a universal AI web agent that can do tasks autonomously on the web, extract structured data, and call APIs all for you with just natural language and with you as you browse, uh, like as a side panel with you, uh, with you as you

2. 0:19

   browse. So we believe this, uh, we could-- we'll dive deep into use cases as well as dive deep into how these kind of category of new universal AI web agents will transform people's workflows and be as transformative to the browser as its creation itself when Net-Netfl-Netscape, uh, came out.

3. 0:36

   So your bottle-- your browser is a bottleneck for most of your workflows right now. Even for people with full-time jobs, they're spending hours of their day manually copy and pasting from one website to another, w-uh, we-a website to like Google Sheets or to their CRM, or they even offshore the scraping to third parties that are expensive and

4. 0:55

   also unreliable. And, uh, yeah, and/or they set up RPA bots that just break whenever a website changes. And you also have silos where some data is only available on the website and some only on the APIs, leading to a hassle for users to combine this data across both, and the-- which leads to untapped potentials of what you

5. 1:16

   can do and what you can leverage this data for. So Retriever changes all this by being a, a Chrome extension that can d-- uh, that leverages being a AI web agent.

6. 1:26

   So you can open the side panel and do tasks acr-- uh, give tasks to do, uh, autonomously across pages, as well as extract structured data to Sheets. So let's dive into a use case.

7. 1:38

   So say you're on LinkedIn and you have our, uh, Chrome extension open. You can give a prompt just like, uh, for the Latent, uh, like find, uh, and follow the

8. 1:51

   Latent Space podcast page and, you know, if following, ignore, uh...

9. 2:02

   So just with this natural language prompt and clicking a button, our AI web agent will, uh, go to the task of, uh, interacting the, with the page to do this task.

10. 2:10

    So it can fill out the prompt f-- it can fill out the search field for Latent Space podcast. It can, uh, interact with all elements on the page. So it already would notice we're already following the page.

11. 2:21

    So, um, yeah. And you, uh, yeah, it, uh, even more-- We can do even more complex use cases than doing task autonomy. We can also have this built-in feature to extract data to Sheets.

12. 2:34

    So say you put this prompt, like for every article on the page, extract this data and click this export button. We would write all of this to Google Sheets for you.

13. 2:42

    And, uh, using, uh, our AI browser agent technology, you can also say like, "Do a task on this page and then extract the data." And, uh, like, you can have all of these combinations of tasks and extractions for you.

14. 2:56

    So it was able to extract all this data to Google Sheets, and we can say that ours, uh, our approach to solving this problem is very cost-effective, and it is probably cost less than a penny to do this page extraction.

15. 3:09

    And even more than doing actions on one page, we can do actions across tabs, across like Sheet, like you can give a Google Sheet column of URLs, and we can open those to interact with and extract from.

16. 3:21

    So we can lead to more complex use cases. So say you're on like, uh, this archive, uh, search, and you want for the first five PDFs, extract these, uh, fields.

17. 3:32

    We can break these down as sub-tasks to do and open them as new tabs and, uh, process these tabs simultaneously independent of each other and write this, uh, data to Sheets for you or interact with those pages.

18. 3:45

    Maybe they're, uh, LinkedIn job applications and you want to apply to them. So we are able to extract all of this data to Google Sheets and, um, you can do more complex use cases of you can select tabs to extract from.

19. 3:59

    So say you wanted to compare these Amazon product pages, and you could just leave the prompt empty itself, and our AI web agent will figure out what you-- fields you probably want to extract from.

20. 4:11

    And, um, from all these pages and extract those, uh, uh, pages to Google Sheets for you, extract those fields to Google Sheets, and so you can maybe effectively compare all three of these products without having to open them simultaneously.

21. 4:27

    And, uh, yeah, so we can even extract URLs on the page. So maybe you wanted to, uh, compare these, uh, products and you wanna see the images themselves, so we can extract the source image URLs, and you can compare them.

22. 4:40

    Even more than that, we can also, uh, do actions on the tabs themselves before extraction. So, so maybe you want instead of top reviews, uh, the most recent, so select, uh, by most recent, uh, reviews

23. 4:57

    in the dropdown and extract the details of the

24. 5:06

    most recent review. So these tabs are selected, and then I just do extract, and our AI web agent will do actions on all of these tabs simultaneously in the background and, uh, extract this, uh, most recent review that we wanted.

25. 5:22

    So they're all like on top reviews right now, and in the next action, it should go to most recent review. So yeah, it clicked most recent here and, uh, yeah.

26. 5:38

    So it clicked most recent across all of them, and it will give us the most recent review for us.

27. 5:44

    And even more than doing, uh, uh, actions just as basic extractions, we can also do, uh, um...

28. 5:55

    Wait, so... Yeah, so it was able to, uh, extract these most recent reviews, and Bhavani will dive into more advanced use cases of doing research and, uh, asking across these, uh, tabs you have.

29. 6:10

    Yeah. Um, if these use cases sounds, uh, exciting to you, let's unlock more potential of Retriever in your, like, day-to-day, like, at job tasks, right? Say for example, you have like bunch of design docs opened and you wanna send a quick summary to your, like, colleague at work.

30. 6:28

    All you have to do is select your design docs and then tell Retriever, "Okay, like, extract, uh, key points and summary." Right? Uh, and when you ask that, it goes through the docs that you like gi-- like selected and extracts the data that you like asked for.

31. 6:46

    So all unlocked productivity in like a one click of a button for you. Not just only Google Docs, you have like PDFs, you have Google Sheets. You can give any combination of anything that's on your browser and Retriever goes through them.

32. 6:58

    As you can see, it extracted, like, specifically what the first document talks about, and then what the second document about like design of like thinking with LLMs with memory as such.

33. 7:08

    Now, uh, not only with the existing, uh, documents that you have. Say you wanna do like a market research on a bunch of companies, uh, to like invest in also.

34. 7:19

    Like, here I'm asking it to go through a couple of agentic, uh, companies like Retriever, Bardeen, uh, Browse AI as such, and I'm asking it to extract certain things like strategy of the company, like features and pricing.

35. 7:32

    And what it does is first, based on what you asked, generates the schema, and then as you can see, we started with the homepage of all of these agent startups, and because we asked for like pricing, it navigated to like pricing page.

36. 7:44

    So Retriever is capable of going, uh, like basically like exploring the webpage or going deep into m-multiple pages to find the data that you asked for and extracts it right into the sheets for you.

37. 7:55

    And also to, uh... We are one of the very first in, uh, implementing this deep research feature compared to Operator also. Uh, and as you can see, the best part with our deep research is your ability to select all your, uh, documents that you have access to, not just on the browser.

38. 8:11

    And you can specifically give what all, uh, URLs or websites, uh, the agent can go in terms of like extracting this data for you. Uh, as you can see, successfully extracted all the information you asked for right into your Google Sheets for you.

39. 8:27

    Now, uh, like let's do this on like some steroids right? Now, like how about I want a market research, yes, but I want a market research about couple of stocks.

40. 8:37

    And then I-- you wanna like buy, make a purchase choice as such. So here all I have to do is select the right Google sheet that has the information.

41. 8:45

    So for example, I have the sheet with couple of stocks lined up, and then I'm asking it like complex data. Like I'm asking it to give me like P/E ratio from Yahoo Finance and some revenue in the last two years, and I'm also asking it to create new data fields of computing the revenue growth, uh, based on

42. 9:02

    these numbers. So what it does, it's like first, um, uh, goes on, opens these websites and tries to like extract this data for you. Say, for example, right now I gave the websites as such.

43. 9:13

    Now, even though I gave the wrong URL, instead of giving Yahoo Finance, I just gave company's website, our agent figured out that it was not on the right URL and it went to like Yahoo, uh, and like kind of like tries to extract this data for you right into your sheets as such.

44. 9:30

    Now, so how cool is that? Like, even like there's like an error from the user, uh, Retriever was able to like correct this, uh, correct this for you. And one of the cool features we have is, uh, our function calling features.

45. 9:42

    Like, uh, lot of, uh, companies have like bunch of like third-party integrations with different tools, say Slack, uh, like say, say Discord also. One of the features we are very keen on implementing is like a dynamic function calling.

46. 9:56

    Like, instead of us implementing only like a set of, uh, connectors as such, you can call any API, any third-party tool out there, uh, by literally providing the information about the tool here and easily integrate.

47. 10:10

    Say, for example, I have WhatsApp integration right here to send messages to like WhatsApp numbers. Let's say you are a small business that you have like bunch of customers and like bunch of like customer phone numbers as such.

48. 10:21

    So I have two users here. And let's say I'm asking Retriever, uh, all I have to do is like select the sheets for you, share the sheets that has the customer numbers, and then I ask Retriever to like, uh, send message to customer phone numbers here.

49. 10:38

    So when I do that, uh, it identifies each of the customer number provided and invokes this, uh, WhatsApp tool, uh, essentially a third-party tool to send message to these users.

50. 10:48

    And as you can see, it says all messages sent successfully. Let, let's check it out. Let's try opening, uh, the WhatsApp.

51. 10:56

    So this is my test number, as you can see. Uh, we're demoing it at two thirty-two. So yeah, this is a two thirty-two like successful test message that we received.

52. 11:04

    So yeah, imagine the world of possibilities like, uh, automating your social communications across your Instagram, Facebook, WhatsApp, everything, all in like one click with Retriever right now. Uh, now let's do one more use case, uh, which is the graph generation.

53. 11:21

    So we have, uh, like a graph bot tool. Uh, we-- It's like almost like a mini agent within Retriever Agent Studio. What it does is, uh, let's say you have like a bunch of data as such, like, uh, and then you wanna generate like a data analysis graph on top of it.

54. 11:36

    You can leverage the graph bot in doing so. As you can see, it's like a dynamic graph generated on the fly for you. So we've realized when working with LLMs, they're not only inherently good with like data extraction as such, but are also capable of generating or representing this data in various formats of interest to the users,

55. 11:54

    and we leverage that capability to build such cool, uh, use case and present it to you. Now, like bringing it back home, now let's look at the overall agentic landscape, right?

56. 12:05

    Um, like let's look at, uh, some of like the Ope- OpenAI's Operator, like Anthropic Claude, and like obviously Retriever and so on. So most of the agents that are out there use like vision-based approach, like your Anthropic Claude or OpenAI, even like Google Mariner, who's a hybrid approach Also including vision approach.

57. 12:25

    As such, what do they do is they take screenshots of the pages and extract the data that you're asking for. And what's the inherent problem with this approach, right?

58. 12:33

    The vision-based models are more prone to hallucination compared to what Retriever is doing, which is text-based approach of leveraging the webpage stuff. And vision-based approach is also highly expensive, taking multiple screenshots for just like one single action to do or like for one single page scrolling as such.

59. 12:51

    And most of these companies such as these browser-based or so on, they use browser on the cloud, unlike Retriever, which is an extension right inside your browser. So what are the problems with browser on the cloud?

60. 13:03

    Like number one, uh, non-personalized results, right? So it's like a generic page that's open on the browser. It's not like-- It mi- the content might be totally different from what you are seeing in your browser.

61. 13:14

    And also because, uh, to support browsers on the cloud, uh, these companies should implement a lot of proxies to funnel the n- uh, network or network request through the right IPs as such, which is way more expensive than the whole agentic setup as such.

62. 13:31

    Um, so with this, uh, text-based approach, as well as being like an extension in your browser, Retriever is capable of processing not only the active tabs, but also your background tabs, uh, or like multiple tabs at once.

63. 13:45

    And it can even go beyond your subscript- subscribed content. So with Retriever, you don't have to like share any of your passwords. We do not sh-store any of the passwords.

64. 13:54

    Whatever you are logged in and you are seeing, Retriever sees the same thing. Unlike on the cloud, um, either you have to like store or give the passwords, which are prone to like various security risks, or you cannot get behind paywalls or even like Cloudflare app website protections as such, right?

65. 14:12

    Now, like, A-Arjun will dive deeper into some of these and like, yeah, bring it back home.

66. 14:18

    Yeah. So, uh, just to recap what are the advantages for Retriever is that, uh...

67. 14:26

    Yeah, maybe I can, uh, share this. Uh, actually, it's fine. So, uh, to highlight the re- the advantage of Retriever, since we're using a text-based approach, the, the-- for writing output, there's, is, there's much less hallucination because the text is, uh,

68. 14:43

    the-- because the text is right there in context for the model. And what, uh, and because we are using this text-based approach, we can also take actions on multiple tabs because these background tabs don't get rendered.

69. 14:54

    So you've-- actually can't use a vision-based approach to take a- multiple actions in parallel. So when we can leverage this, uh, back- uh, background processing to do actions, uh, to do multi-tab context-aided actions, as well as just, uh, speed up performance by taking actions in parallel.

70. 15:12

    And in-- compared to our competitors of like trying to do consumer applications of booking flights or booking restaurant reservations, we're all in for productivity and automation use cases because we see that, uh, AI is perfect to automate these manual and repetitive tasks that is, uh, a burden to a lot of, uh, people.

71. 15:30

    And compared to everyone else, we are setting up a client-side Chrome extension that is, uh, not only cheaper infrastructure-wise, but it leverages us to use new sources of, uh, content to-- for the agent to work on.

72. 15:43

    So you can access the local walled sites or login walled sites, and, uh, it is much more secure than s-storing your passwords in a cloud-hosted browser. And compared to other providers of doing one single long horiz-- one t- uh, actions on a single tab with, uh, long horizon, uh, tasks, we distribute subtasks as new tabs to take

73. 16:04

    actions on. So our failure rate is much less than these, uh, uh, competitors. So we don't, don't have to deal with, uh, exponential failure rates. And our approach to third-party integrations is that we, the user can define and, uh, set up function calling that-- function calls that they can share.

74. 16:21

    And this is much more, uh, extensible and scalable than setting up custom third-party integrations that these, uh, third parties-- uh, that these competitors are doing.

75. 16:31

    Yeah, so this is our mission is to basically revolutionize data extraction with a transparent and efficient AI-powered exchange. This is our long-term goal of being a-- allowing people to collaborate across the-- their own local laptops to collaboratively construct datasets and, uh, set up, uh, uh, like very cost-efficient and cheap datasets.

76. 16:54

    So for example, it could be like all the, uh, local government events happening in the SF Bay Area that would involve like extracting data from hundreds, thousands of websites and this is not something that is feasible right now.

77. 17:06

    It's like not cost-effective to do. But if people can use it, leverage our extension to volunteer and collaboratively construct datasets, we believe that's an exciting new future and use case that is, uh, that is just on the horizon.

78. 17:22

    Thank you, and, uh, please feel free to go to rtrvr.ai to download the extension. That's Retriever without any vowels. Or you can scan this QR code and try it out and hopefully you can get a glimpse into what the future of, uh, AI agents in the browser will be.