The Rise of CaaS: Context-as-a-Service for Agentic AI

Talk transcript

1. 0:00

   [upbeat music] So hi, everyone. Uh, thank you so much for taking the time to join this session.

2. 0:16

   I hope I'll, or at least I can guarantee I'll do whatever it takes, uh, to make it worth your time. Uh, my name is Omer. I lead the product marketing team over at Bright Data.

3. 0:26

   Just by maybe a quick show of hands, who here is familiar with Bright Data?

4. 0:31

   Okay, we can do better. I'll pass it on to our brand team. Bright Data is a web data company. Basically, we help more than twenty thousand teams around the world, including more than seventy percent of the world's biggest AI labs, to extract data from the web.

5. 0:45

   Just to put this in perspective of what scale we're talking about, we're talking well over fifty billion pages, HTMLs every day, more than twenty petabytes of video, audio, and other media data.

6. 0:56

   So that's just a perspective of the type of work that we do at Bright Data.

7. 1:01

   But enough about us. Personally, I joined Bright Data, uh, about three years ago, which essentially gave me front-row seats at everything around AI and the web and how they started to actually connect.

8. 1:14

   It sounds very old, but if you think about it, only maybe less than two years ago, right, we were able to start using... access the web and search the web through Claude or through, uh, ChatGPT.

9. 1:28

   That option didn't even exist in the earlier versions, right? So that connection, that way in which both AI and the web are starting to converge is something that is still evolving and evolving rapidly, and that's part of what I want to try and shed some light on today and talk about a new emerging breed of companies that's

10. 1:45

    coming out of this connection. I think we can basically agree that the web is by far the world's greatest source of data. At least historically, when it comes to Bright Data, that's all we cared about, right?

11. 1:57

    It's helping our customers extract data from the web. But with the emergence of AI and the emergence more recently of, of AI agents that need to do knowledge work, right?

12. 2:06

    The web is no longer just a source of data. We can actually start looking at it as a source of context. Context in the sense that if I do knowledge work and I have knowledge agents that support my work, I want to go out to the web, find the information that I need, use it as context, but

13. 2:19

    kep-keep on working. So the data itself is only a step in the process for something bigger, for the actions I need to take, for the conclusions I need to draw, and for every downstream application that follows.

14. 2:31

    Uh, the first ones to figure this one out... Oh, sorry, even before that. But one thing that I want all of us to bear in mind, because this is gonna follow us through the rest of this conversation.

15. 2:41

    The web is messy, it's unstructured, and most importantly, it changes all the time. This is a chart that shows data decay, right? The-- It's analysis done by our team that shows data decay, basically, how long after a new page, a new piece of content goes live, it is no longer relevant, right?

16. 2:58

    So social media, it's easy for us to understand. It's far less than a day, but also news, finance, retail, thirty days later, data that was collected is mostly no longer relevant.

17. 3:09

    And when we acknowledge that, this simple notion, we understand that extracting context from the web or relying on the web is not a snapshot. It's not a one-time effort.

18. 3:17

    It's not even a monthly effort. It's something that we need to keep on doing. It's something we need to look at as an ongoing process and something that we need to be mindful of.

19. 3:30

    The first ones to figure it out were, of course, search companies, right? Only what? Three years ago, we were on the far left, right? This is, this is in our lifetime, right?

20. 3:41

    Three years ago, we were on the far left. Everything was Google. There was no-- There was complete and total dominance up until three years ago for the past twenty or so years.

21. 3:50

    Right? That's what we're talking about. Purely h-for humans, search something, go, collect the information you need, and carry on.

22. 3:58

    Then fast-forward, maybe one half years ago, two years ago, search began to appear within the LLMs, within the chatbots, right? Uh, which already started blurring the line between humans and, and agents because now the same bots also have the same web search available.

23. 4:13

    So the same, um, LLMs have the same access through API for the bot. So we started seeing that convergence happening, right? And so for the first time, we're seeing more and more traffic flowing down, search traffic, search intent flowing down these channels and not only to Google.

24. 4:29

    And last but not least, right, we now see a whole breed of companies, the AI search companies. I'm sure you're familiar with them. I caught a talk yesterday by, uh, by Will, the CEO of Exa, in parallel in new.com and Tavily and a bunch of others.

25. 4:41

    They are purely built and indexing the web, especially for agents. They're not even looking at the humans involved anymore.

26. 4:48

    So Google's dominance when it comes to search, if Google was synonymous of web search, that is very much shaken. When there's blood in the water, the sharks come. Just last week, Amazon.

27. 5:02

    Amazon announced, I don't know how many of you saw it, that they developed their own index and have started allowing the ability to retrieve data from the web, to retrieve context for agents on AgentCore.

28. 5:13

    Amazon developed their own search engine. Two weeks before that, it was Microsoft. Microsoft always had skin in the game. I don't know, one or two percent of the world search traffic went to Amaz- to Microsoft, but they are now repackaged it and launched it again as part of WebIQ, as part of their suite for a-agentic development and

29. 5:32

    orchestration. So we're seeing more and more this space becoming crowded.

30. 5:40

    But when we're talking about context and we're looking at this through the lens of search, I believe it only tells us part of the story. Right? I can search for, I don't know, what's the cost of, um, a certain pair of sneakers this morning, right, in, on a certain website.

31. 5:57

    I cannot really search for how has that price changed over the last six months, what discounts, um, uh, it had.

32. 6:03

    Right? I can search for what, uh, open job positions we have at Bright Data. We do. I urge you to go have a look. But I can't see how m- how the, that was a chart and how that changed over time, and how the headcount of the company changed over time.

33. 6:17

    And all of that information existed on the web simply back then. So when we actually start to think about it, we understand that there's much more context in the web that, than what web search allows us to extract.

34. 6:31

    And this is what we started seeing in the recent years, a whole new breed of companies rising. We like to call them internally CaaS, Context-as-a-Service, because that's what they do.

35. 6:40

    They allow agents to tap into them, MCP, CLI, uh, just pure good old API, and actually start extracting data to, to retrieve data so they can reason over for whatever knowledge work they are responsible for.

36. 6:53

    We see this happening in e-commerce, we see this happening in travel, we see this happening in, in finance, in market research, in, uh, in HR, in real estate, in a bunch of other domains.

37. 7:03

    I'll show a few examples in a second, right? What all of these have in common is that they don't only just discover the web, you know, in terms of think crawling, think searching, think all of that, accessing, extracting the data and indexing it, they take it a step further.

38. 7:17

    They actually develop knowledge graphs to start structuring all of the entities and to dedup them, and they start enriching them with a lot of different sources. So they actually start merging all of that data.

39. 7:27

    If you think about it, they kinda behave like vertical search engines, right? They are very, very, very good search engine for something very specific.

40. 7:37

    And, and it's, it's already in full motion, right? So as I said, we see this in finance and in market research, in retail, in e-commerce, in GTM and sales intelligence.

41. 7:45

    What all of these companies, by the way, have in common, they're all part of Bright Data's startup program. If you are a builder, and this is a hot space to go in because I think we're only tapping the surface, I invite you to scan this and apply up to twenty thousand dollars in credits and all sorts of

42. 8:01

    co-marketing, but that's enough self-promotion. So CaaS as, as an industry is already in full bloom, and as always with these situations, right, also the traditional players aren't left too much behind.

43. 8:18

    They're at the bottom. You see good old Data-as-a-Service, you see ZoomInfo, right? By researching for this, uh, presentation today, I also saw that they launched that thing at the top.

44. 8:28

    It's called GTM.ai. You can only imagine how much they paid for that domain. But they launched it as a secondary brand for ZoomInfo that is catering specifically for the need of agents.

45. 8:39

    Look at, look at the wording, right? It-- They talk about GTM work, right, that knowledge work, that research that you do when you need to prospect, when you need to do headhunting, when, whatever it is you need to do that involves people mostly, straight from Claude Code, straight from Codex or any other agent.

46. 8:52

    They understand the gap, right? So yeah, so it's fun to, to think of CaaS as an evolution of DaaS, and it is in a way, but it's catering for a very specific need as much as the AI search engines are different than Google.

47. 9:05

    When agents need them, it's different than people.

48. 9:10

    When we let this one sink, that at the very least we have two different types of paths to complete knowledge work as an agent, we can start thinking about this in terms of web context engineering.

49. 9:21

    We can start thinking about this in terms of, how do I optimize for the specific task? More importantly, when things come, uh, as it, a- as it is, how do I optimize this for breeds of tasks?

50. 9:33

    How do I, um, I do this for various parts of the organization that I'm building for? If I'm an AI engineer, I need to serve different teams. They may have different needs.

51. 9:40

    It's very tempting to throw AI search at all of them, but maybe that's not optimal. Maybe I need a combination of both. Maybe I can start seeing all sorts of cost efficiencies emerge from that.

52. 9:52

    So for the second half of this presentation, we actually went ahead and created a test. This is not a benchmark. You won't see any, any something concrete that I can say with great confidence other than the actual research that we did, 'cause we wanted to start unraveling the different considerations and how do these two stack up against

53. 10:09

    each other. So we designed a test. We went for something basic. We said, "Okay, let's take a company, an entity, and try and enrich it across twenty-five different fields."

54. 10:18

    Some of them are very easy, you know, the company domain, the name, uh, the headquarters, but some are more challenging, right? Things about hiring and people and something. And we build a, a simple, uh, agent, a loop in a loop that knows-- that uses, uh, uh, Opus four point eight as the harness, and it starts to go

55. 10:35

    over field by field, go out, search for it, or retrieve it from the CaaS. Do it again and again and again until it completes and brings back, set some guardrails, you know, like budget and stuff, just to keep it, uh, fair.

56. 10:47

    And I wanna share with you the results. So the first thing that we would care about, right, being knowledge work would be... Uh, sorry. We ran it one hundred times on all of the sponsors of today's event.

57. 11:00

    So the first thing that we saw in terms of coverage is that there's pretty good convergence. They all did fairly well, right? Um, I'll get to the two at the bottom in, in a second.

58. 11:09

    So search were consistent performance. One of the major CaaS providers were also very well. The third one, by the way, you can see Unlocker and SERP. SERP is good old data.

59. 11:19

    Uh, good old Google. We basically did the same thing just with Google, and it performed pretty well in extracting that information. Native is Claude's own, uh, search, and you see that they converge really well.

60. 11:32

    I was originally surprised about the two CaaS solutions at the bottom. It was counterintuitive. I expected CaaS to dominate this thing 'cause that's... Y- you know, you had one job, right, to map all these companies.

61. 11:44

    But, uh, but a- after diving into, into it a bit more, you, you understand that, well, they are limited in the sense that they know what they have about an entity.

62. 11:52

    If I ask you the question that is beyond that, they will never have that data, right? Not like a search that can go out and continue searching and exploring it.

63. 12:00

    If they didn't collect data about the recent job hiring, it will never be there. Right? So it makes sense that they are a bit behind, but I'm sure at the same time that they have a lot of other advantages that we simply didn't ask for, a lot of other fields that they didn't have that aren't represented.

64. 12:14

    So again, it, it creates some complexities when what... how do we measure coverage when it relates to the specific job that we need to do rather than, um, in general.

65. 12:25

    When we-- The second thing we looked at was cost, of course. Here it's-- we started seeing it spread out a bit. So you can see that massive bulk in the center, most of the search and the, and the CaaS, and even using, uh, Google, right?

66. 12:37

    Uh, uh, converged to pretty much the same cost. J- Only different, right? The CaaS was just about the service itself, what you pay the vendor, right? All of the other search solutions, you also needed a lot of token burn it- burned to actually structure that data so you can actually act on it and, and, and use it

67. 12:54

    as something retrievable, right? So it's the same output. Native, obscenely expensive. And the, the CaaS on the right, I'm sure you're all familiar with, they're by far the most expensive in the industry.

68. 13:04

    I will not name and shame them. Uh, interesting, you see that small CaaS there at the, the left, that, uh, CaaS number two, uh, that were very cheap. They're also the ones that are here at the bottom, which is funny because what I believe is happening there is that we're seeing, even within this industry, niche players that

69. 13:23

    have, you know, lower quality data, but much cheaper. They're already carving that niche of the long tail, right, of small shops or small usage that, uh, don't want to pay as much and don't need as much data, and we're all seeing it, them branch out there.

70. 13:38

    Most of you here, I presume, are engineers, so there's a very evident question that we did not ask here, which is, what is the one thing that an engineer would care about?

71. 13:51

    Scale. Thank you. Let's talk about scale. This is the cost, not for the whole hundred. This is the cost per one, for one record.

72. 14:05

    What happens if we hit a million? Now, yes, a million records will not fit in the context window, obviously. We're not talking about a single run that needs a million.

73. 14:13

    You can think about a million in terms of the frequency, right? If I am a market researcher, I do due diligence for private equity, I revisit these companies all the time.

74. 14:22

    I ask more questions about them as the time goes by. Was there any new news about them? Was there anything that changed? Did somebody join? Did somebody leave? Do they have new hires?

75. 14:30

    I keep on asking the same thing. So when I'm talking about this, the multiply by a million, it's not just about the number of companies. It's the frequency in which I'm asking it.

76. 14:38

    Frequency is the cost killer when we talk about these, and we need to acknowledge that, right? We're, we're thinking about this in terms of web context engineering. We're starting to look at it differently.

77. 14:48

    Every repeated query costs the same as the first, even if it brought back the exact same answers. Nothing changed. Pay up. Right? No, it's false positives. For sure, go in.

78. 15:01

    Token costs, right? We saw the model that the, the very high token. We know that doesn't, that doesn't shrink well over time. It-- there's always some volume element in terms of the cost, but it's not the same as, you know, flatlining, right?

79. 15:14

    And if we bring this back to knowledge work, this is where we see teams that are starting to

80. 15:19

    cut corners. "So I won't research this company every day. I'll look at it once a week or once a month. I won't ask that question now. I will-- I don't want all the results.

81. 15:28

    I'll only take ten results, twenty results, something." We start... So we already have the setup. We have what we need to do the knowledge work, but at the same time, we're not extracting all of the value because we're starting to be conscious about cost, right?

82. 15:40

    Basically, we're renting context. We're not owning the context that we use. That is a very important distinction.

83. 15:48

    Again, if we're good engineers and we ask ourselves, what about scale? The second most obvious thing that will come to mind now, so how about we build it? What if we take all of that web data ourselves and stick it in some vector database and try and see what comes out of it?

84. 16:06

    So I asked my AI engineer to do exactly that.

85. 16:10

    Again, this is a test. This is not a benchmark or a full-blown operation. This is a, a day's work at best just to illustrate the concept and to show something about the cost efficiencies that you can generate potentially by doing it yourself, potentially, in specific scenarios.

86. 16:27

    The test, simple. Take the company name, nothing but, run it through Google, find the relevant entries, the relevant URLs of that company in various websites that have all of that information, right?

87. 16:37

    You use search when you don't know the source. But when we're talking about company enrichment, we all know these sources. We all know where that data comes from. The CaaS also bring it from them.

88. 16:45

    ZoomInfo bring it from them. It's the same thing over and over again. Why not just go straight to the source? Why are we doing that middleman thing?

89. 16:53

    LinkedIn companies, LinkedIn jobs, Crunchbase. Bright Data, we have scrapers for those. You just tap in and you start paying as a pay-as-you-go. We built two dedicated scrapers. We have a new AI tool called Scraper Studio.

90. 17:04

    It basically lets you build the scraper for any website in less than five minutes, all powered by AI. And then it also have a self-healing function, right? So if the website changes, uh, it fixes itself and keeps on going.

91. 17:16

    Merge it all into one entities. Basic heuristics. If there's conflict, choose that over that. And eventually, we have a dataset of these, uh, one hundred companies, zero AI cost involved.

92. 17:31

    There's no tokens. Coverage, fairly well. Not amazing, not the best that we saw here, but stacking up pretty well. And again, this is just a day's experiment. Probably not even as much.

93. 17:46

    Okay? So at the very... again and again, very specific tasks, very limited context, very limited situation. You know, tread, uh, tread lightly and then proceed with caution when it comes to conclusions.

94. 17:58

    The real story is not this. The real story is this.

95. 18:03

    That's what it costs to just go and fetch that data that is out there.

96. 18:08

    We think about knowledge graphs, we think about entities, but if you think about, for example, LinkedIn, the data is already structured in form of entities. There's an entity for a company, there's an entity for a person, there's an entity for a job, and they're connected between them.

97. 18:21

    Sometimes the ontology is already there. Again, this is not the most complicated of scenarios, but this is pretty damn good. Now,

98. 18:32

    yes, it took time to set up. So it's not really fair to compare apples to apples when it comes to the cost because these are out of the box.

99. 18:39

    You can just tap into the API. That one that I just showed you required some setup. Let's say it's a week. Let's price it at five thousand dollars just to give us some perspective.

100. 18:48

     We can actually start thinking about this in terms of a tipping point. We can actually start thinking about what is that tipping point in which it makes more sense for me to build it myself, right, than keep on renting it.

101. 19:02

     Now, again, everything to the left of that dot, in this case, it was just over fifteen thousand entities or queries, right, when we think about it. So it made sense to do it at this point.

102. 19:11

     Maybe it's not fifteen, maybe it's thirty, maybe it's a hundred thousand, maybe it's ten thousand. It really depends on the use case. But there is a tipping point in which it actually makes sense to do it yourself.

103. 19:22

     Uh, which leads us to the fact that both, uh, AI search and CaaS and all these solutions, they're very g- good in the sense that you can just plug and play.

104. 19:31

     But if your knowledge work needs, right, are persistent and consistent, and to a certain degree may even continue escalating and growing, then this is perhaps a direction to start considering.

105. 19:43

     Maybe I can just go ahead and build my own. 'Cause the nice thing about it is that all of the things that we see on the left up until this third part is

106. 19:53

     upfront investment, and the most important thing that whatever retrieval happens later on from the an- from the agents, right, is free. Not really free, but you get what I mean, right?

107. 20:04

     There's no added cost. I can just ask that question over and over again. I did not like the first answer, I'll ask it again. I'll ask it a hundred times until I get what I need.

108. 20:12

     I, I have no more fear, no more cutting corners, which is maybe the most important thing. And I'm leaving aside the fact that this is also custom business logic.

109. 20:18

     I can connect it with my own data. There's all sorts of other advantages of owning it. You know, I'll-- we'll keep it, uh, to the imagination. Remember, we asked about a million, not about fifteen thousand.

110. 20:32

     This compounds. This compounds greatly, right? We need that horizon. Remember, the web keeps changing. We saw the staleness of the data and how the data decays. So we need to be thinking about this in the long run and how this will evolve when we keep on asking the questions about the entities that we care about.

111. 20:50

     Just to wrap it up. So AI search, CaaS, they can get you very far when what you need is ad hoc or when what you need is always changing, when sometimes you look at different things.

112. 21:00

     Th- even the mix and match of them, for certain tasks use this, for certain tasks use that. You can... I'm sure, again, I just showed a test. There's a lot of ways to optimize it, just like any other context engineering, and make-- and use, uh, lighter models and use other stuff.

113. 21:12

     There's a lot of great stuff to be done. But eventually, the frequency will come and bite you in the ass when it comes to cost, and that's something to be mindful of.

114. 21:21

     And there's a fair chance that that tipping point is much lower than you think, and that's something that when-- as we design these systems, when we think about web context engineering, we need to be mindful of that.

115. 21:32

     And last but not least, the last slide we showed. Owned context compounds while rented decays, right? It's not a one-time task. Again, if it's a one-time question, use AI search, it will be amazing.

116. 21:45

     When you need to do it over and over again, there's a fair chance that it will not, uh... that you're missing out on potential compounding effect, and you are losing out.

117. 21:57

     Thank you very much. [audience applauding] [upbeat music]