Agents' next frontier: agent-to-agent and network effects — Jean-Denis Greze, Town

Read the talk

Agents’ next frontier: agent-to-agent and network effects

Jean-Denis Greze reframes agent collaboration as a search problem: how can the right information reach the next model call when it belongs to someone else? Five approaches move the privacy decision to different points in that process.

From a talk by Jean-Denis Greze

At a glance

Ideas worth remembering

  • Evaluate agent collaboration by whether it brings the right information into the consequential model call. Privacy limits that search even when context capacity is unlimited.

  • Custom tools can search broadly while returning limited results, such as relationship scores. Their privacy tradeoffs still require deliberate design and user acceptance.

  • A sweeper agent turns private work into proposed shared knowledge using retention policies and destination descriptions. Human approval and automatic policy enforcement offer different levels of disclosure control.

  • Searching before requesting consent can spare irrelevant owners an approval prompt, but it requires trusting broad internal access and controlling the final disclosure.

  • Begin automatic sharing with an accepted low-sensitivity zone. Persistent errors, prompt injection, disclosure mistakes, and audit needs remain part of the design as that zone grows.

The useful unit is the call with the right context

People get work done with other people. Assistants should benefit from the same connections: another person’s agent may hold the information needed to complete your task. Jean-Denis Greze, introducing himself as Town’s CTO after seven years as CTO at Plaid, starts from that opportunity and immediately questions its framing. “Agent to agent” tells us who communicates. It does less to explain what makes the result useful.

The alternative framing is search. Immediately before a model returns an answer or calls a tool, its context window needs the information required for that decision. A more capable model can do more with that information, but the surrounding system still has to find it and bring it into context. Collaboration matters when it improves that process. 1:13

Greze sketches a progression from humans manually filling the context window, to retrieval-augmented generation using a search tool, to agentic search using multiple tools to explore available content. Each changes how information arrives. The engineering target remains the consequential model call: does it have the content needed to choose the right action?

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:12 · section reference included

The impossible agent that can see everything

Imagine a single agent with access to everyone’s email, every company’s records, and every government’s information. In Greze’s thought experiment, it can assemble all the relevant context before answering. This gives multi-agent systems a comparison point: how closely can they approach the information available to that agent? The optimal outcome is an idealization within this model of the problem, rather than a demonstrated guarantee that complete access makes an LLM correct.

Even an infinite context window would not make this agent acceptable. People do not grant strangers continuous access to their inboxes. Privacy and security restrict what can enter the search process before context length becomes relevant. Greze invokes the Coase theorem as an analogy: removing information barriers and transaction costs can improve negotiation outcomes, while real-world constraints prevent the frictionless ideal. Here, the constraint is permission to use information.

The five approaches that follow place that permission decision in different places. Some restrict the searchable data in advance. Others permit broad internal search and control what leaves it. Their usefulness depends both on the information they recover and on how much human work they require as models improve.

2:132:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:13 · section reference included

Restrict access, or restrict what a tool returns

The first strategy gives an agent broad access inside an existing trust boundary. An HR team agent might read the HR systems available to an employee, perhaps using the access level of the team’s most junior member. Everyone on the team can ask it questions. This fits a familiar SaaS security model, which helps explain its appeal to IT and security teams.

Its limitation is structural. Humans still choose the accessible data, and information outside that selection remains outside it. Better models may answer better within the permitted material, but they do not automatically dissolve the divisions between teams. Greze’s two tests are whether the system naturally needs fewer humans over time and whether model improvements expand what the approach can accomplish. A manually defined team silo fares poorly on both.

The second strategy builds tools that trade some information for a useful capability. Consider the question that returns throughout the talk: does anyone at the company know someone on Acme Corp’s finance team? Unrestricted search would read everyone’s Gmail, identify exchanges with Acme employees, inspect profiles, and find a connection to the CFO. That would also expose far more correspondence than the requester needs.

A narrower tool can accept a company domain and a target role, search the mail internally, and return ranked relationship-strength scores. The requesting agent receives a useful result without receiving the underlying emails. It then messages Bob on Slack about his connection to Jane, Acme’s CFO. Bob confirms the relationship and agrees to an introduction. The observable change is from a broad question about the company’s network to a specific, approved introduction; the tool supplies the discovery step, while Bob supplies consent for the next action.

  • Relationship lookup: Broad mail access stays inside a purpose-built tool; the agent receives a limited relationship result and contacts the relevant colleague.
  • Draft placement: Colleagues can put draft emails in someone’s inbox, reducing the clicks needed when sales staff would otherwise ask that person to write an introduction.

Town uses this kind of tool for some recurring user activities, with an opt-out. More participants make more connections available, creating the proposed network effect. But privacy is still a product decision: people must understand and accept the information a tool reveals. Humans design each tool and explain its behavior, so the approach remains largely manual rather than expanding automatically with model capability.

4:144:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:14 · section reference included

Move shareable knowledge out of private silos

The third strategy creates a shared place where useful information accumulates. Private records can stay private while material suitable for the company or a subgroup moves into a wiki, database, or other common store. All relevant agents can search that store. The benefit comes from making already-shareable knowledge available beyond the conversation or inbox where it originated.

  • Shared skills: An engineer saves a better database-profiling procedure in the repository. When another engineer encounters a slow query, their agent can use that procedure instead of rediscovering it.
  • Shared records: Agents contribute information to a wiki or Airtable over time. This requires both tools for reading and writing the store and instructions or incentives that make those actions part of ordinary work.

A sweeper AI makes contribution a separate, recurring job. An agent inside each private silo knows what must remain there and knows the available shared destinations. At the end of the day, it examines new information and moves suitable contributions into those spaces. “Public” here means available within the company, rather than published on the internet. 10:13

Who decides which contributions may leave? Two versions put the decision in different hands:

  • Human approval: The model prepares a list of proposed contributions and asks the user to approve publication to the shared space. It removes much of the writing work while retaining a review step.
  • Policy enforcement: The model applies a policy itself and publishes the information it judges permissible. That removes routine approvals but makes disclosure quality depend on the model’s application of the policy.

What must the sweeper know to move information safely? The diagram separates the new private material from the rules and destination descriptions used to evaluate it. Both approval paths lead to the same shared store; they differ in who authorizes the contribution. This is where the shared-silo approach can begin to benefit from better judgment by the model.

Greze forecasts adoption of automatic policy enforcement within the next six months, especially in high-trust companies with ten or fifty employees and clear exclusions such as finance and HR data. He expects large enterprises to move more slowly. This is his adoption forecast, not evidence that the disclosure problem has already been solved.

How a sweeper contributes private knowledge

New private informationprivate — Material accumulated inside a private silo.
Policy and destinationsprivate — What must stay private, and descriptions of the shared spaces.
Daily sweeperprivate — Examines new information and selects contributions.
Human approvalapproval — The user reviews proposed contributions.
Automatic policy decisionapproval — The LLM decides what the policy permits.
Company shared spacesshared — Contributions become available to other agents.
A sweeper evaluates new material using retention rules and shared-space descriptions. A human or the model’s policy judgment authorizes what leaves the silo.
Read the diagram as text
  • New private information. private — Material accumulated inside a private silo.
  • Policy and destinations. private — What must stay private, and descriptions of the shared spaces.
  • Daily sweeper. private — Examines new information and selects contributions.
  • Human approval. approval — The user reviews proposed contributions.
  • Automatic policy decision. approval — The LLM decides what the policy permits.
  • Company shared spaces. shared — Contributions become available to other agents.
  • New private information → Daily sweeper: Examine new material.
  • Policy and destinations → Daily sweeper: Guide selection.
  • Daily sweeper → Human approval: Propose contributions.
  • Daily sweeper → Automatic policy decision: Apply policy.
  • Human approval → Company shared spaces: Publish approved material.
  • Automatic policy decision → Company shared spaces: Publish permitted material.
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:43 · section reference included

Shared memory can preserve the wrong thing

Greze’s immediate-return bet is the automatically maintained information base, whether implemented as a wiki or database. Its value is straightforward: information becomes available for later work without someone repeatedly copying it between private conversations and shared records. But the same persistence that makes it useful can keep errors alive.

His personal wiki still calls his agent Apex, although he renamed it Ivy a month earlier. The old name survives somewhere in the memory setup and keeps resurfacing. A wrong agent name is funny; a wrong business fact carried forward into future decisions is harder to dismiss. The example illustrates stale information persisting, without identifying the particular memory mechanism responsible.

  • Prompt injection: Material planted in a more open silo can enter context through agentic search and influence subsequent behavior.
  • Persistent factual errors: A model mistake can enter the shared wiki and continue supplying wrong information to later work.
  • Wrong disclosures: Automatic decisions can release information inappropriately, with consequences ranging from negligible harm to firing or a customer lawsuit.

The black box also creates an operational tension. Someone will eventually need to audit what happened. Greze asks who approves what, what gets logged, and what can be reversed. A system whose traces are hidden from requesters still needs an accountable way to inspect its data use, potentially through someone in the CISO organization. Secrecy from ordinary users does not eliminate the need for internal oversight.

15:1415:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:14 · section reference included

Start with a low-sensitivity zone

The proposed frontier is automatic judgment about when permission is necessary. Greze compares it to coding workflows that moved from approving every action, through “YOLO” execution, toward an auto mode that intervenes selectively. For information sharing, the corresponding design separates low-sensitivity material that may move automatically from material that needs human review or always requires approval.

Supplier meeting notes make the decision concrete. A user asks for notes from a weekly recurring call with a supplier and its finance team. In the proposed system, the model examines the notes, considers the requester’s role, and decides whether disclosure is acceptable without asking everyone involved. The presence of finance participants alone does not settle the decision; content and role inform the risk judgment.

This design gives model improvements somewhere useful to go. Better judgment, clearer policies, and carefully designed tools could expand the set of sharing decisions handled automatically, including decisions about when to involve a human. Greze’s recommendation is to define a low-sensitivity zone where the organization already accepts model discretion, then develop from there. Expansion remains conditional on those improvements; a more capable model does not itself grant permission to disclose more.

The desired outcome is less human coordination work as the system improves. Greze gives that ambition a pleasantly concrete destination: a beach in Hawaii with his kids. The architectural question underneath it is whether better models can take over more of the information-routing work, rather than leave humans maintaining the same silos and approving the same requests.

17:1517:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:15 · section reference included

The harder network effect crosses company lines

Within a company, shared governance offers a starting point for agreement about access and disclosure. The larger opportunity is getting several companies to accept a common agent working across their private information. That requires a use case valuable enough for each participant to permit some cooperation while retaining restrictions on what may be accessed or shared.

Greze describes an unnamed finance company exploring this direction with investment banks. The potential benefit is sharing private information about private companies for activities such as lending, with agents helping decide what can be accessed. The example is exploratory and unnamed, so it supports a possible beachhead rather than a proven deployment. Its appeal is that another institution’s participation could make useful information available that no single participant holds alone.

The ending leaves a real question open: how comfortable should people become with agents deciding privacy? Greze expects more decisions to move in that direction, yet does not claim to trust a future in which agents make all of them. The opportunity and the hesitation belong together. Work across silos can become more useful as judgment improves, while every new connection still needs an acceptable answer to who may learn what.

19:1519:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

19:15 · section reference included

Resources

Transcript & slides

Loading transcript and slides…

Transcript download