Tethered: Our Agents Are Us — Shu Fang, Two Sigma

[music]

Great. Thanks everyone for coming. Uh

this talk is called Tethered. Our agents

are us. I'm Shu Fang from Two Sigma and

let's get started.

So just a quick explanation. Two Sigma

is a little quant fund. I will also take

the opportunity to explain that the name

ostensibly is not because we have two

co-founders who are very online but

because the two sigas are about the

volatility sigma the small sigma and the

large sigma sum. So by summing together

these individual volatilities we can uh

hedge the risk achieve differentiated

alpha.

Now because we are a hedge fund I have

to give you all this important legal

disclaimer. You don't have to read it.

It just has to be in this. And the TLDDR

is that I'm not trying to sell you on

anything. The views are mine and not

necessarily the companies. Any logos,

any other companies I mentioned here are

not me endorsing them or telling you to

buy their stocks or anything. It is

purely maybe coincidental.

But that also is meant to segue into the

fact that we are an old company. We're

25 years old and clearly we're a very

regulated industry, but we've managed to

run an ecosystem where everyone at the

company has a cloud agent. And not only

that, but these agents run as their own

identity. So, we're going to explain how

we got here and why we're actually okay

with this.

So, we're first going to do a little

horror movie review. If any of you have

seen us, you don't have to pay attention

to this. defend if you haven't the TLDDR

of the movie is that everyone has these

doubles and these doubles are called

tethered when the doubles decide to run

loose and cause chaos and run around

with these golden scissors they're

called untetered and this is going to

somehow relate into my talk

so back in June 2025 you know cloud code

GA and all that stuff people started

using agents through the local computer

your local machine and it's very

powerful but one it was CLI constrained

and two it was localized right we wanted

to achieve a world where people could

use these agents from wherever they were

whether it be mobile through slack

through browsers but still have the

ability to run them remote and this is

important not just because of the

capability but many many people

technical or not are not comfortable

fully operating within a CLI so the

question became Okay, how do we actually

run these in terms of what identity they

run as?

The conventional wisdom is that you run

these as some machine identity that is

attached to your user in some way. You

have a shoe and you have a shoe agent.

But this quickly collapses and we found

this collapsed because of all the

reasons that you can imagine, right?

It's very hard to keep permissions in

sync. Anytime you're dealing with

software licensing, now you have to deal

with two licenses.

uh there's certain systems that do not

support multiple identities interacting

with the same underlying data. You know,

stuff like Google Workspace, your

emails, etc. And then how some systems

are going to block as a first step. So

you have to you're just going over the

barrier of entry. And then you also have

to figure out how you actually manage

the public and private boundaries.

So obviously it's like why don't we just

run these as the user, right? How do how

do we run these remotely as the exact

same user identity? And as a result, all

the capabilities, all the access, all

those previous constraints are no longer

valid.

And we already had the info for this and

I imagine a lot of you do too. If you

don't, I would uh encourage investing in

it, which is that you know you could

have a Kubernetes cluster. You have all

all of your clusters, your regions, etc.

And you have name spaces for

individuals, right? And the reason we

had this is because we often already

needed this capability not for the

agentic purposes, but for all the

automated operations that we need to do

that did not suit confinement to

someone's local machine. So we'd run

automated jobs, you know, code

containers usually operate on this

principle, research notebooks, etc. And

every single user already had these

namespaces existing in every single

region and everything in it runs as the

user.

A very simplistic way of how this works.

Some trigger is going in to your

controller and it's saying, "Hey, I need

to spin up some compute resources. You

have a separate identity service that a

sidecar in the pod pulls down from to

allow your actual containers

to run and mount that identity and it

runs as you.

So of course there are big dangers with

this right and you know the first danger

you may imagine is an internal danger.

How do you actually differentiate who or

what took action right? You have you and

your U agent are now the exact same

identity. That's why I grew this

mustache so you could tell the

difference between us for now. But uh

you really want to know that

differentiation because certain actions

that can be taken. You want to audit,

you possibly want to block and you want

to just have the trace, right? You want

to have the attribution to determine,

hey, was it someone operating as the

human operating purely human actions or

was it the agent identity doing these

things?

Another danger and perhaps a bigger one

is we all know that for all of these

capabilities and LOMs in general, it's

essential you have access to the

external web. These are point in time

mathematical functions that cannot

actually update based on current data.

So it's like open internet access.

That's why it's a core capability, web

search, web fetch tools, right? The

problem is once you have that

capability, you leave yourself open to

huge vulnerability vectors. One of which

is exfiltration risk. This is one we are

deeply concerned with in terms of

possibly losing IP, you know, just

exposing our sensitive information. but

also certainly the possibility of

untrusted content flowing back in and

you know prompt injection just malware

and vulnerabilities are all big risks

there and then something we separately

deal with is just the ability to make

sure we don't

u use licensed content without the right

copyrights or actual licensing right you

can map this to the golden scissors that

they use and us so this is kind of our

biggest fear to be honest

So we are a finance firm and in finance

there's a concept of obviously risk and

return. So when we think about what is

the positioning on the risk and return

graph there's huge value in allowing

agents to run you but also there's very

high risk. What we generally want to do

is make sure we capture as much of the

value as possible but reduce the risk.

We're optimizing that ratio of return

over risk. Some of you may know the

sharp ratio. We're looking at that from

the perspective of how do we let agents

run as users and optimize that return.

And the ways we need to do this, right,

are to solve those two critical

problems. One, differentiating access

attributed to the human versus the

agent. And two, somehow getting safe web

access in place.

So the first thing we did is this

attribution step, right?

And how we did this is we use a header

and we make sure that every single agent

continues to append to that header. And

this is something we've all hopefully

done in some way, right? Trace IDs.

You've you've all done this in

deterministic code. Making sure that

your observability stack propagates

through a trace ID through disparate

systems. How we did it is very similar

to how you would do it for trace ID

except we are dealing with a certain

difference in the control vector which

is the agent itself right and you can

force you know using certain HP clients

using MCPS using skills to make sure

that that header initially gets

populated and everywhere else along the

way continues to be populated right you

have a lot more deterministic control

over agents and the harnesses and the

frameworks than you may ink and you can

enforce it with some of the already

existing primitives.

Now this gets very interesting because

this is not only giving us the proper

identification of who did something

right it actually goes beyond that and

no longer are we confined by just

knowing the act identity but we also

actually get the full provenence through

the system right as we deal with

multiple steps in the system we are able

to replay the entire chain of actions

that actually led to some end result. So

the comparison here is if we had used

that shoe agent identity we wouldn't

have this and we would just know that at

some point shoe agent triggered this

initial flow into the span but we don't

actually know hey those subsequent

actions how do we properly trace back to

that origination point with this header

this trace ID we get that full

propagation and the actor is still me

right it's still my identity

And the second step that we needed to

fix is this web access. Right? A lot of

web access these days uses indexes for

search. Right? I think cloud code's

native one is Brave web browser and it

uses a Brave index. Well, we were like,

hey, why don't we see what Google has,

right? Google is at its core hopefully

still a search company and they do this

index generation already. And it turns

out they actually do offer something

specifically for regulated industries

like ours that allows you to use their

web index but within your existing VPC

your you know network controls right and

it's called web grounding for

enterprise. It basically works like this

where it's still within the exact same

network boundary where you're probably

running your cloud agents and stuff like

that and offers two core capabilities

search and fetch. Right? So the exact

capabilities we want to mirror, we

leverage that. We have all these

guarantees. There's one tiny downside,

which is the data is obviously not going

to be completely fresh, right? And the

constraints around this last I checked,

it's fresh within 24 hours. And for more

regularly updated websites, it's fresh

within 6 hours. But for most use cases

that you may have for agents, that's

probably more than sufficient and

completely removes this external egress

vulnerability vector.

Now the second question is how do we

actually ensure the agents use web

grounding and again this is very

simple with the existing primitives

right you just need to make sure that

they don't get confused and you

certainly block the access itself but

just for user experience and stuff like

that you need to make sure those tools

themselves that are already existing and

primitive and native to these agent

harnesses and frameworks and such are

actually blocked right again here's

cloud code example. I think every other

hotness has the same thing. Web search,

web fetch. We just deny those tools.

It's like, hey, you can't even use

these. These are not even in your suite

of tools available to you. Instead, we

use the redirection going through, you

know, MCP CLI and actual client code

using the supported paths, uh, skills,

whatever to make sure that whenever

someone does need the capabilities of

web access, it goes through that web

grounding cache index.

So takeaways from this talk basically

make sure you ted your agents right

letting them run around untetered you

know is very dangerous we want to tar

them and it's much safer to do so and in

fact if we go back to that initial slide

of how we consider this relative to the

risk and expected return because of some

of the things we found while doing this

we actually believe we didn't lose

expected value while huge hugely

reducing the risk, right? So, the index

certainly lags, but we get a ton more

observability by just using that tagging

primitive versus the actual uh just pure

identity verification.

And I think this is [sighs and gasps]

one thing people should really consider,

especially people working at companies,

enterprises, which is that there are a

ton of things happening in the Genai

landscape that are probably scary to us

that make your security teams really

afraid that feel like, hey, they are too

far on the frontier, right? You you're

like, I wouldn't run this locally. I

wouldn't run a open cloud agent on my

local machine with full permissions,

right? There's all these horror stories

and you know various anecdotes about why

this is bad. But in an enterprise again

you can figure out how to leverage your

enterprise resources to actually reduce

those risk factors and get the real

value out of the capabilities and this

is where you should be investing that

time.

So what we ultimately shipped is this

entire framework, right? We have the

ability to run cloud agents as user

identities uh because of all of those

guard rails and vectors we put in place

and using different kind of interface

vectors to actually operate with them so

that people who are not comfortable with

CLIs can leverage them but certainly for

other cases as well. And as part of

that, we made sure to ship out just a

managed fleet of cloud, you know,

whatever agents for every single user in

this remote fashion that they can

already interact with so that we can

continue to deploy and improve what is

actually available to individual users.

But also the core capability itself of

being able for every anyone at the

company to deploy an agent that runs in

the cloud remotely with their full

identity is there and is something we

are comfortable with.

So to finish up, uh, everything I talked

about actually happened last year. So if

you are interested at all in wanting to

build and see what we're working on now,

or even better, if you're like, that was

horrible, we could do so much better. We

are hiring and we encourage you to

apply. If you have any experience in any

of these domains,

you can check that QR code, check that

link. Yeah, that's it. Any questions?

[applause]

>> What do you think about local connected

to agents for enterprises?

>> Uh, not the views of my company, but

personally I I think that is

>> question.

>> Yeah, sorry. His question was how do how

do I view local LLMs for enterprise

usage? And I think local in the sense

that we manage ourselves is probably

where we eventually want to go for a lot

of our token use and inference because

of cost because of deprecations because

of you know every time Frontier Lab

drops a new model you see some

degradation. It's just there's too much

volatility in that that we don't need to

risk as the openweight models kind of

become more advanced and sophisticated.

request.

>> Yeah. So as you can see the header is

not purely differentiating in itself.

Someone someone could certainly populate

that, but the actor the identity itself

will not be me, right? So some someone

could I guess write in that they're

using some agent, but the core previous

identity itself is not mimickable, not

like actually interceptable, right? So

we still we have both we have both the

originating identity and that you know

all of your identity ecosystems and

chains to ensure that but also the

header. part of the header.

>> Yeah, that that that is separate. The

header is just XSLm agent and then you

still have some way you you need some

way to actually determine the identity

of who's coming.

Yeah, I you know gen generally all of

our RPC in some way has an initial entry

point that is like

populatable with that header and then

once it actually goes downstream you

know you make sure that same with trace

ids is part of the span internally

within the So

>> can you clarify how the use of Google

index addresses the prompt injection

issue?

>> Yeah. So the core things about this

index is not only is it a cached index,

it is it has a lot of other controls and

safety guarantees around it. It is

specifically made for these curated

financial like um highly regulated

industries. So they themselves are doing

some of their own curation on top of it.

Now certainly I think that creation

could fail. It's probably done using Gen

AI but the prompt injection risk is much

far reduced because everything still

remains internal.

>> Yep.

>> Do you mean how they're being used?

>> Yeah. I I think it's critical um in the

sense that that behavioral data is

something we can further configure based

on right like that we we do try to

ensure that not everyone at the firm can

see what your agents are doing right

like there's stuff certainly work-wise

but also more sensitive information that

might be privileged to you. So your

session data is kind of localized. Now

that behavioral data in the session data

is very powerful because it can define

additional configuration that can be

applied to these agents for the purposes

of making the user experience better. So

we we try to leverage that to figure out

what to configure further not just based

on someone's like hierarchical persona

but actually based on their usage to

make sure that their experience

continues to improve actually based on

what they're doing.

Uh we're almost out of time. me and my

co sorry I'll take the last question

you're saying you have a process for

letting individuals create their own

>> yep

I mean how do you go about like anybody

there

is a process for like agents across the

company

>> yeah uh for building our own agents you

know we we use some of the existing

frameworks for agent building. Obviously

all the genai harnesses are very good at

using those frameworks to build agents.

So you have a lot of agents

proliferating based on that. In terms of

provision, everything is already

provision. All this is every single user

at the firm has all the necessary

infrastructure in place. So that's not

really a worry. They can build an agent,

deploy it as necessary into their

namespace running as their identity. the

the aspect of how do agents then become

like a universal companywide or larger

beyond a individual users uh agent you

know goes through your standard kind of

mechanisms right like hey is there going

to be proper production support is there

the right security it's like any

application you might develop

uh yeah so me and my colleagues will

stick around here if anyone wants to

talk further you know I guess if you're

sticking around I can also take more

questions but thanks for coming to this

>> [applause]