AI Engineer World's Fair 2025
The Unofficial Guide to Apple’s Private Cloud Compute
Read the talk
How Apple’s Private Cloud Compute Makes Remote Inference Verifiable
Private Cloud Compute combines anonymous authorization, hardware-backed attestation and public software records to constrain what a remote AI service can do with your data.
From a talk by Jonathan Mortensen
Before you start: Familiarity with HTTP, public-key encryption and software hashes will help; the article explains attestation and transparency logs as they arise.
What does private actually prevent?
What makes an AI service private: a promise about how it handles your data, or technical limits on what it can do? Jonathan Mortensen approaches that question from both AI and security. His background includes a PhD in biomedical informatics and data science, two sold companies spanning AI/data and cybersecurity/infrastructure, and work at South Park Commons building Confident Security. This is his unofficial, public-source explanation of Apple’s system, not an account on Apple’s behalf. Privacy and security overlap, but protecting a service from attackers does not by itself prevent its operator from retaining your requests.
The first example is DeepSeek’s exposed database. Wiz’s original disclosure reported over one million exposed log lines, including chat history and other sensitive information. That is more precise than treating the count as distinct conversations or confirmed stolen records. Mortensen then polls the room: Who cares about privacy? Who uses ChatGPT? Who considers it private, or uses its private mode? The tension is familiar: people can care about confidentiality while relying on a service whose internal handling of their data they cannot inspect.
The next example is a court order to preserve ChatGPT logs. Mortensen describes the obligation as covering everything, including private interactions, and calls API Zero Data Retention a default. Those details need a historical distinction: OpenAI’s account excluded qualifying Zero Data Retention API arrangements and Enterprise/Edu services; ZDR was a distinct arrangement, not the ordinary API default. Its later update says the prospective retention obligation ended September 26, 2025. The architectural question survives that correction: a privacy setting does not necessarily remove the provider’s ability to retain data.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Remote compute without a black box
Apple’s problem begins with a mismatch: some AI tasks require more compute than a phone can provide, while privacy is a major reason people buy the phone. Sending data elsewhere introduces another party into its handling. Giving every phone its own dedicated cloud H100 would be an extravagant answer; the practical requirement is remote compute that remains both private and affordable.
Imagine an iPhone facing an untrusted server. The server is a black box: once the phone submits data, it cannot see what happens inside. Private Cloud Compute, or PCC, attempts to give the client evidence about the system receiving that data, plus enforceable restrictions on its behavior. Mortensen organizes the design around five requirements and six technical components, then examines remote attestation and transparency logs in depth.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Five requirements that change the server
| Requirement | What it constrains |
|---|---|
| Stateless computation | Use request data only to satisfy that request. |
| Enforceable guarantees | Remove prohibited capabilities rather than merely forbid their use. |
| Non-targetability | Make it difficult to direct an attack at one selected user. |
| No privileged runtime access | Prevent production administrators from bypassing the restrictions. |
| Verifiable transparency | Let outsiders check the software supporting those guarantees. |
The distinction between policy and capability is concrete. “Administrators should not SSH into this instance” leaves an access path available. Removing SSH removes that path. Mortensen similarly uses removing the disk as a conceptual illustration of preventing persistence. The objective is that user data cannot be logged or reused for unrelated purposes, and that the client can verify the system supporting that objective.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate identity, then verify before disclosure
Start with an ordinary service: an authentication layer fronts an AI engine, an SRE can access the engine, and the engine can write to disk. Adding an anonymizer hides the request’s network origin, but it does not solve the whole identity problem. If the phone still presents identifying authentication credentials downstream, those credentials can reconnect the request to its user.
Blind signatures separate obtaining authorization from spending it. Mortensen’s analogy is an arcade: first exchange money for coins, then spend those coins at machines. A machine accepts a valid coin without learning which purchase of coins produced it. Likewise, the inference service needs evidence that a request is authorized without receiving a credential that identifies the person who obtained it. Combined with the anonymizer—loosely compared to Tor—this reduces the information available to someone trying to locate a particular user’s requests.
Next, change the order of disclosure. Before sending the sensitive input, the phone asks, “What are you running?” It accepts a particular software configuration and only then submits encrypted data. Mortensen describes the intended binding as allowing decryption only by the trusted configuration. Removing sshd and preventing persistent retention then limit what that configuration can do with the plaintext. Verification and capability restrictions work together: knowing the software matters because the accepted software is supposed to enforce the data-handling rules.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The six components behind the design
The conceptual changes map onto six technical components. The first two separate identity from inference:
- Oblivious HTTP: an independent relay forwards the request so the destination sees the relay’s IP address rather than the user’s. Mortensen identifies Cloudflare as the relay for PCC requests; the essential architectural property is the separation between the requester’s network identity and the inference destination.
- Blind signatures: the service can validate authorization without linking redemption to the user who obtained it. This supplies the anonymous arcade coin that an anonymizer alone cannot provide.
Mortensen points to specifications and existing libraries for these mechanisms rather than suggesting that developers invent the cryptography themselves.
The next components establish and constrain the machine’s identity:
- Secure Enclave: separate hardware protects private keys from export. If another machine could copy the identity key, it could impersonate the genuine engine. Mortensen introduces TPMs as a non-Apple analogue for this role.
- Secure boot and a hardened operating system: a restricted, signed software environment reduces opportunities to modify the server. Mortensen describes it as a limited iOS-derived system; Apple’s architecture draws on both iOS and macOS foundations.
The remaining components expose evidence to the client: remote attestation describes the running system, while a transparency log records published software so the client can compare that description with a public record.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Bind the encryption key to the accepted machine
Remote attestation can be understood independently of the iPhone:
- The client requests evidence about the server.
- The server returns signed claims and a public key.
- The client checks the claims against its acceptance policy.
- If the evidence is acceptable, the client encrypts its request for the corresponding key.
Claims might identify genuine hardware, a GPU, the bootloader, the operating-system version and the application software. An acceptance policy might require a trusted signer or reject an older Linux kernel. The server provides evidence; the client decides whether that evidence describes an acceptable environment.
The key must be bound to the attested environment. Otherwise, a server could present reassuring claims but ask the client to encrypt for an unrelated recipient. Mortensen explains the binding as decryption remaining possible only while the accepted claims hold. At this conceptual level, that describes the security objective, not a continuously polled software check: the details depend on protected keys, cryptography, certificate chains and vendor trust. The black box becomes more translucent because the client receives authenticated evidence about the destination before disclosing its input.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Connect a live attestation to reviewed software
An attestation can identify software without explaining whether that software deserves trust. A transparency log supplies the public record needed for review. Mortensen’s example is a record saying that Bob signed a compiled binary, identified by its hash, on November 1, 2024. Records can describe whole releases or individual components. Reviewers obtain the published binaries and inspect their behavior offline; later, a client can recognize the same binary hash in an attestation.
The matching rule is small enough to illustrate directly. In this TypeScript example, reviewedHashes represents a local policy decision made after review. Both inputs to the function are already authenticated evidence; the function performs only the final membership and acceptance check.
typescript
type PublishedBinary = {
signer: string;
publishedOn: string;
sha256: string;
};
function acceptPublishedBinary(
attestedHash: string,
authenticatedEntries: readonly PublishedBinary[],
reviewedHashes: ReadonlySet<string>,
): boolean {
const published = authenticatedEntries.some(
entry => entry.sha256 === attestedHash,
);
return published && reviewedHashes.has(attestedHash);
}
A published binary is not automatically a trusted binary. The log establishes that it is publicly recorded; review and client policy determine whether it is acceptable.
An attestation for software absent from the log should fail that gate. Mortensen treats such a mismatch as a sign of compromise, potentially including a hijacked connection. The log itself has restricted writers and an append-only Merkle-tree structure: cryptographic proofs make unauthorized changes to its history detectable. That is stronger than trusting an ordinary mutable database to keep an honest release history.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put the request path together
The phone first requests an attestation package through the anonymizer. It evaluates the package before sending sensitive input. If it accepts the environment, it encrypts for the key bound to that environment; the intended result is that a different, unaccepted environment cannot decrypt the request. The transparency check connects the attested software to Apple’s public release records, whose binaries can be examined offline.
Around that exchange, Oblivious HTTP hides network origin, blind signatures provide anonymous authorization, and the Secure Enclave protects keys in hardware separate from the AI engine. Secure boot and operating-system hardening constrain the runtime. These are complementary boundaries: the relay does not establish software integrity, and attestation does not by itself make an identifying credential anonymous. The assembled architecture needs both.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What the guarantees still cost
Attestation does not eliminate trust in Apple. Apple controls the supply chain and checks nodes when they arrive at its data centers. Mortensen also describes provisioning identities called DCIKs, while tentatively expanding the acronym. The deeper dependency is key provenance: shared identity material, insecure key generation or a private key set to the same trivial value everywhere would undermine the evidence. These are hypothetical failures illustrating where trust remains, not allegations about Apple’s implementation. At the time of the talk, Mortensen describes PCC as confined to Apple devices and official consumer applications.
The performance path starts locally: a device uses local models when they are adequate and calls a data center when they are not. Network distance then matters, especially for real-time voice. Mortensen describes roughly six layers of encryption before the inference node, adding compute work. He provides no measured latency or cost comparison. The offering he describes also excludes custom models and fine-tuning.
Stateless inference does not mean a stateless client. The client must coordinate authorization, attestation, transparency checks and encryption rather than make a single ordinary HTTP request. If the phone fails after authenticating and loses its authentication keys, the client must recover from that lost state. The server’s refusal to retain request data makes the client protocol more consequential, not less.
Operations and accounting become harder too:
- Debugging: removing SSH and preventing request-data logging removes familiar diagnostic tools. Apple’s design still permits restricted operational metrics; the restriction concerns exposing or retaining user data, not eliminating all observability.
- Usage accounting: Mortensen uses a 2,000-token allowance as an example of the tension between customer-specific metering and unlinkable requests. Charging a named customer for particular inference work requires a linkage the anonymous request path is designed to avoid.
- Abuse controls: he acknowledges coarse fraud and abuse tracking even while fine-grained user linkage is restricted.
This is a design constraint for anyone building a similar service and passing costs to customers. Mortensen closes the trade-off discussion by reiterating that PCC was not available as a general third-party developer service in the offering he was describing.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build with the pieces available outside Apple
Developers outside Apple’s ecosystem can assemble related mechanisms, though the hardware and trust boundaries differ.
| PCC component | Building block outside Apple |
|---|---|
| Oblivious HTTP | Relay and client libraries |
| Blind signatures | Anonymous authorization libraries |
| Secure Enclave key protection | TPMs and cloud virtual TPMs |
| Secure boot and hardened OS | Restricted, measured software environments |
| Remote attestation | Platform-specific attestation evidence |
| Transparency log | Sigsum or Sigstore’s Rekor |
Mortensen presents TPMs as widely available on contemporary Intel and AMD hardware, and virtual TPMs as a cloud option for the same key-management role. That functional analogy does not make every virtual TPM’s trust boundary identical to physical hardware. He describes attestation integration as less standardized and straightforward than the other pieces. For public transparency systems, he names Sigsum and Sigstore, whose relevant log component is Rekor.
Confidential VMs extend the discussion from protected keys to protected computation. For AI, that protection also needs to cover the GPU path. Mortensen identifies H100 and H200 as the confidentiality-capable GPUs in his account and explains the goal through a physical-memory-inspection example. The exact memory protections and supported hardware depend on the platform; that example should not be read as a blanket guarantee of encrypted GPU memory or immunity to physical inspection.
Open source and reproducible builds add another useful connection: reviewers can examine source code and establish that it corresponds to the published binary, rather than relying only on black-box binary testing. Mortensen contrasts this with Apple’s approach. Apple had also released selected PCC security-critical source under a limited-use license; that is different from making the entire production stack open source and reproducibly buildable. The engineering opportunity is to connect source review, binary identity, public release records and live attestation into one chain of evidence.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Private inference beyond one ecosystem
The closing discussion moves from components to adoption. Apple announced PCC in June 2024; Mortensen places its release in October 2024. He then points to Azure AI—explicitly correcting himself from Azure OpenAI—as pursuing private inferencing, with a September start and private-preview status in his account. Meta’s Private Processing announcement followed on April 29, 2025, describing an early architecture ahead of rollout. Mortensen characterizes its approach as essentially identical to PCC; the useful comparison is the shared privacy principles, not identical hardware or implementation. For WhatsApp, as for Apple, protecting private interactions is also a reputational concern.
Mortensen’s own commercial answer is Confident Security, which he pitches as a similar service for people outside Apple and WhatsApp. He closes by inviting questions afterward and offering privacy-themed merchandise, including a shirt he says confuses facial recognition in photographs. The shirt is a playful extension of the same concern that opened the talk: what happens to personal information once another system receives it?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Details the publicly accessible database that exposed over one million log lines, including chat history.
The public project site for one of the transparency systems named in the talk.
Documentation for the transparency-log component of Sigstore.
Further reading
Apple’s explanation of PCC’s five requirements, hardware foundation and verification model.
Introduces Apple’s Virtual Research Environment, selected source releases and security research program.
Explains affected services and exceptions, with a later update on when the prospective obligation ended.
Updates since the talk
A later framework associated with Confident Security, including clients and local exercises for private inference.
Read the complete timestamped transcript
- 0:00
[upbeat music] So we're gonna talk about Apple's Private Cloud Compute.
- 0:16
This is an unofficial guide. Uh, I don't work at Apple. We'll talk about it, um, in a sec. But so, um, this is my background. My PhD in bio-- uh, in data science, biomedical informatics.
- 0:28
I've sold two companies, one in AI and, uh, data, one in cybersecurity and infrastructure. I'm at South Park Commons. I'm building a company called Confident Security, which we'll get to at the end.
- 0:37
Um, but again, disclaimer, put this-- I'm not an Apple employee. I'm not speaking on their behalf. Everything I've gleaned is from public sources. And hopefully, what we'll learn today is some tools that we can use ourselves.
- 0:47
There's really six key components are-- and some approaches to ensure privacy, and privacy and security are very related, not perfectly overlapping, but related. So, um, before I get there, I know that we're in the security track, but I wanna motivate why you might care about privacy.
- 1:02
Not everyone believes that you should have privacy. Uh, so let me just give some, give some examples. This year, DeepSeek leaked a hundred mil-- uh, a million sensitive records of chat logs, and you might say to yourself, like, "Well, that's DeepSeek.
- 1:17
Everyone knew that was gonna happen." Um, but, uh, before, before I show you the next piece, I wanna poll the audience. How many here care about privacy?
- 1:26
All right. How many of us use ChatGPT?
- 1:31
All right. How many consider ChatGPT to be private? Okay, good. Um- [laughs] How many use the ChatGPT private mode?
- 1:40
Uh, and then how many use the API but with zero data retention, kind of like the default setting? Okay, great. Well, as of yesterday, uh, OpenAI has to retain everything, uh, anyway, whether or not you flagged it as private or not.
- 1:54
Uh, and I don't wanna get into the comments of why they have to do that, but the point is, is that, uh, they have the capability of retaining your private chats that you can flag on the UI as not private, other things.
- 2:06
And, uh, obviously, being forced to do that's not great, but this is why we should all care about privacy.
- 2:13
So Apple doesn't want to have these headlines, uh, 'cause their-- one of their major value props is privacy. So let's talk a little bit about the problems that Apple solved and then how we might use it.
- 2:25
So fundamentally, AI requires more compute than a phone, uh, but every-- obviously, they wanna bundle AI into their phones. Privacy is a major selling point. Any time you give up private data to something remote, you're inherently reducing your privacy, right?
- 2:38
Any time I give you my data, uh, it's not as private as it was the second before. So the question i-- that Apple is trying to answer in, in their PCC system, which is now available on all of our iPhones and used for inference, is:
- 2:49
How do you get remote compute while remaining private? And the simple way to do that would be to buy all of the iPhones, H100s, pair them in the cloud.
- 2:56
You get your own H100, boom. But obviously, AI is even more expensive, so, uh, that's not gonna work. So they actually need some approach, which is: How do you get rem-remote compute while remaining private and cheap?
- 3:09
Uh, otherwise it doesn't work. So I'm gonna kind of frame the problem this way. You've got an iPhone. You've got an untrusted remote server. You can't see inside of it.
- 3:17
It's a black box. Once you give them your data, you have no idea what happens inside. And what Apple does is tries to make it not a black box, so that the iPhone has some control of what happens to the data inside Apple's remote servers.
- 3:31
Uh, and then hopefully that this trusted remote service is also hard to hack. So for the remainder of the talk, and, uh, we're not gonna be able to get into all of it in sixteen minutes, uh, but we're gonna talk about Apple's PCC requirements that they set up, and I'll review a conceptual architecture about how they meet
- 3:47
those requirements. Then we'll go into two specific components of the six, because I don't have time to go through all six, and you'll be bored by that point. Uh, and then talk about some, some pros and cons of Apple's thing and how we might use some of those components ourselves.
- 4:02
So there are five key requirements to Apple's Private Cloud Compute that they're trying to meet when they design the system. The first one is stateless computation. This is essentially the guarantee that when Apple receives your data, it's only used to satisfy the request and cannot be used.
- 4:16
It's impossible to use it for anything else. You can't log it, anything like that. Um, the second thing is enforceable guarantees. That this-- notion that the code-- everything's enforced with code, not by some sort of policy.
- 4:28
Not, I shouldn't SSH to the instance, but I can SSH to the instances. No, there's no SSH on the instance. You can't SSH to it. Um, you don't wanna save things?
- 4:37
Well, don't have a disk, right? So these are what they call enforceable guarantees, not just policies. The third principle or requirement is non-targetability. That means that if you wanted to hack my data on PCC, you'd have to target everyone and sift through all of it, rather than having some easy way to find just me.
- 4:57
No privileged runtime access. I just briefly touched on it earlier, but essentially there's no way to bypass these restrictions in production.
- 5:06
Um, and then the final one and the most important one is verifiable transparency. Verifiable transparency essentially says we can prove that all of the above items are true. Great.
- 5:19
So let's talk about, again, this is a little more bigger representation of the black box. Um, in a classic, you know, kind of remote system, you have some sort of auth service, and then we have an AI engine.
- 5:30
And in this AI engine, you have some SRE who can access it and some disk that you can write to it. But again, the iPhone doesn't know, uh, what it's sending its data to.
- 5:38
But this is fundamentally. So let's see how we can change this to, uh, get to some of these at a conceptual level. So the first thing that Apple does is it adds an anonymizer.
- 5:51
And, uh, this anonymizer is the first part of two parts of non-targetability. But ideally, right, Apple can't tell who the data's coming from, so it would be harder for an attacker to come and, like, fish out my sp-- particular set of data.
- 6:07
Um- If, but if you're a student looking at this, you're still auth behind the anonymizer, and so the iPhone provides some sort of auth credentials, and those auth credentials are obviously tied to the user.
- 6:16
So the second thing that Apple does is separates auth. Um, and conceptually, you think of this as if you're going to the arcade, uh, and you wanna go spend your money on arcade machines.
- 6:25
You first put your money into the coin machine, you get some coins out. These coins are anonymous. Now you can go to the machine and no one knows, uh, what machines you spent your money on.
- 6:34
That's essentially what happens here. It's called blind signatures. We're not gonna have time to get into it today, but that's what happens. So now the iPhone is making an anonymous request, going through an anonymizer that's taking everyone's...
- 6:46
It's kind of like Tor. It's like laundering everyone's data, uh, so that if someone were to access the system internally, they wouldn't know who it's coming from. So that gets us non-targetability.
- 6:58
The second thing that Apple does is it changes the set of requests that are happening. The first thing it does is before it sends its data, it says, "What are you running?"
- 7:06
Uh, and if the AI engine replies with, "I'm running this and only this," the iPhone might say, "Okay, I trust that. Uh, and if that remains true, then you can run this, this AI on the data that I'm submitting."
- 7:19
Um, this is where-- how they achieve verifiable transparency. There's a little more subtlety to that, which we are going to get into. Um, but it's essentially the iPhone says, "I trust the code that you're running.
- 7:28
You can only decrypt my data if you're still running that code." So the iPhone can verify what they're doing. The next thing, no privileged runtime access. That was easy.
- 7:36
Just get rid of the SSHD, make it no way of accessing those machines. Uh, enforceable guarantees, get rid of the disk. We talked about that. And then stateless computation, again, with no disk, no access.
- 7:48
There's nothing to do with the data other than respond to the iPhone. Um, and since the iPhone verified the code that was running on this machine, it knows it's not being logged anywhere before it gives them the data.
- 7:59
Okay. So they achieve those five guarantees that I talked about here using six technical components, and again, we're gonna go into two of them. Um, but I'll describe them all very briefly.
- 8:13
Oblivious HTTP is a spec developed by Cloudflare and Apple and others that allows you to essentially make anonymous requests using a third party to, uh, use-- launder your request through this third party.
- 8:26
So all of their requests that goes to Apple's Private Cloud Compute first goes through Cloudflare. So when Apple receives it, it only knows that it came from Cloudflare, not from an individual user's IP address.
- 8:36
The second thing that they use is blind signatures. Blind signatures is that arcade analogy that I gave you, but it essentially is a way to auth separately and then verify that you're bearing true authentication, but you can't link it to your identity.
- 8:50
And again, we don't have time to go into that, but if you want to look it up, it's a formal spec as well. There's lots of packages and open source libraries that let you to use that.
- 8:58
Third component is the secure enclave. Um, this is an equivalent we have in, in our world that if we're not programming on Apple is TPMs, if you've heard of that.
- 9:07
But they're essentially a place, a separate piece of hardware where the private keys are kept, and that makes a guarantee that those keys can never be removed from the hardware.
- 9:17
That's really important, um, because you don't want the keys to be given away that does all of these. All of the interactions that this is doing is with keys that they prove who they are.
- 9:28
If they could move it and have some third party hold it, then it wouldn't be trusted, right? You could essentially fake everyone out that you are an official AI engine, but actually you're somewhere else.
- 9:37
So the secure enclave helps with that. Again, won't be getting into those. We're gonna get into these two. Uh, the last one is secure boot and hardened operating system.
- 9:45
This is like a standard technique, um, but it's essentially they run a very limited version of iOS, um, that makes it very difficult to hack or modify. Um, everything has to be signed, just like every...
- 9:55
If you've done an iOS now, app now, you have to do signatures, but theirs is like even crazier. Um, okay. So the ones we're gonna talk about are remote attestation.
- 10:05
That was this flow I talked about here. Um, great. And then, uh, the other one is the transparency log. It's, the transparency log is a record of all of the software that Apple's deploying on their private nodes so that you can go and verify what's on the record is actually what's being sent to you during the attestation.
- 10:23
Okay, so let's talk about remote attestation very briefly.
- 10:28
Uh, and I'm gonna talk about it abstractly, not with iPhones. So you have some client, and the client says, "What are you running?" Uh, and the server replies with two things, a set of signed claims and then a public key.
- 10:40
And the signed claims essentially say, "I'm on genuine hardware. Uh, I'm running a genuine GPU. Uh, I am running this set of software. I use this bootloader. I use this version of Linux."
- 10:53
And then the client gets to look at those claims and decide whether it trusts that version. Uh, right? It might be like, "Oh, I only trust this version of the Linux kernel and above."
- 11:03
Um, or, "I only trust that it's been signed by Apple." Uh, and if so, it can use this public key that comes across to encrypt data that is later sent to the server.
- 11:16
Uh, and this is really important, which is this public key and these claims are tied together. So during later interactions with the server, the client will encrypt using the public key and the signed claims, and the server will only be able to decrypt if it is still matching those signed claims.
- 11:32
There's a whole bunch of cryptography that makes this possible, and a bunch of certificate chains, and a bunch of like trust and vendors, but that's the fundamental idea. And this is what is letting you change that black box to something that's a little more translucent, right?
- 11:45
Not just throwing it over the wall. You can kind of see what's going on inside.
- 11:50
Okay. The second thing is the transparency log. Transparency log is actually very simple conceptually. It's just a database with records for each software release, any-- or each component in a software release, signed by a particular person.
- 12:04
Um, so for example, in this record, Bob added- This binary or piece of, like, compiled source code, uh, and this is the hash of that binary on November 1st of '24.
- 12:16
And then, uh, that's it. It's just a declaration that this binary was signed by Bob. Uh, why does that matter? Why would you care about this? Well, first of all, um, reviewers can go through and offline look at these binaries that are made publicly available and verify their behavior.
- 12:36
And so then when you get a remote attestation and it- the remote attestation says this hash of this binary is there, you can be like, "Oh yeah, I've already checked this binary.
- 12:43
I believe that it's doing the right thing." Um, the second thing-- So that's what I said the second point, which is you can check the remote attestations match what's in the log.
- 12:52
And then finally, if you see an attestation that's not on the log, you know the whole system's been compromised because if, if it's not on the log, definitely someone is, like, doing some sort of shenanigans, right?
- 13:02
They might have, like, hijacked your connection, whatever. Um, and it's just because, like, a limited set of people can write to this log and there's no way to modify the log, right?
- 13:11
It's append only. It uses, like, a Merkle tree so that you can't change the contents. Um, great.
- 13:19
So that is the transparency log. Um, so let me tell you how this all comes together, right? So remote attestation is this flow. Again, uh, the iPhone first through the anonymizer requests a remote attestation package, uh, and then says, "Well, if I believe that remote attestation package, I trust the contents that is running on the server, I
- 13:40
can then send my data," and I phrase this as try to decrypt the data and run the AI. Again, if the attestation changed, the AI engine would not be able to decrypt the data, right?
- 13:50
So that's the most important part, right? It says, "I'm running this thing. Trust me." And it says, "I trust you." Okay, great. Encrypt it. And I can only decrypt it as long as it's still running the exact thing I said I trusted.
- 14:01
And the second item we talked about is the transparency log, which is check if the attested claims match the transparency log, and this transparency log we talked about. So on here, Apple is writing a lot, a lot of software onto the log and then essentially saying, uh, "Trust what's on the log.
- 14:17
You can verify it offline. And then when the attestation claims come in, just double-check that they do indeed match."
- 14:25
Um, okay. And then I, I don't have time to get into all of these, um, but, uh, here are some of the other items that we talked about. The blind signatures, the oblivious HTTP is the anonymizer.
- 14:38
Blind signatures are the way they do the auth. Um, and then of course, uh, over here we have the secure enclave. I kind of put that outside of the AI engine.
- 14:47
They're, they're separate pieces of hardware. And then the hardening is just this little lock, but, you know, we don't have time to get into it. Um, and, uh, that's kind of at a very conceptual level.
- 14:59
Like, you could essentially do a PhD on each of these, um, how Apple's PCC works.
- 15:06
So what are the gaps? What are the downsides? Well, first you have to put all of your trust in Apple still, right? On the bright side, like, Apple runs their whole supply chain.
- 15:15
They verify the nodes when they get them at their data center. They actually reassign them, uh, what's called, what's called Data Center Identity Keys or something like that, DCIKs.
- 15:25
Um, but there's no guarantee that Apple doesn't share the certs with anyone, uh, or insecurely generate them or set the private key to one everywhere. Now, I, I think they are trying to do their best effort, but you, you still have to trust-- You've shifted the trust now into, like, Apple's behavior rather than the hardware.
- 15:43
Um, but anyhow. And then they're only available on Apple devices for consumer use, uh, on official apps. Maybe at some point they'll make PCC available to everyone else, uh, but not yet.
- 15:55
So what trade-offs does Apple PCC make? Um, they're limited by latencies to Apple data centers. So, um, they do have local models first that they try and use, um, but if those local models aren't adequate, they'll send them to data centers.
- 16:09
Um, as we start to do, like, real-time voice and other things, uh, this is a little more latencies, uh, like adds a lot more latency to the system. Um, the compute costs are higher.
- 16:20
There's doing a lot more encryption. There's like... I didn't-- I mean, you're not seeing it, but there's like six layers of encryption before it even gets to that node that actually, that actually makes it happen.
- 16:28
So you're spending a little more compute there. Like I, I told you, no custom models, no fine-tuning. The client libraries are very complicated. Um, the client having to orchestrate all of these requests, this transparency log, this auth, that's way more complicated than a simple HTTP request, which kind of sucks.
- 16:47
Um, and what if your iPhone goes down after it's authenticated and then it loses all the authentication keys? Like, you've essentially, like, lost all of your state, right? So it's a lot more stateful.
- 16:59
Um, operationally complex. You can't SSH into the machine and there's no logging. So that's difficult. Uh, not everyone would sign up for that. Um, you can't do any usage tracking.
- 17:10
If you could do usage tracking, then you'd be identified, right? And so Apple can't, like, parcel out, you know, you get two thousand tokens. They do do some fraud and abuse tracking at a very gross level, but, um, if you wanted to use this and maybe pass on your costs-- a similar architecture and pass on your costs
- 17:27
to the customer, you wouldn't be able to know which customer was doing what, right? Um, and then not open to third-party developers. Okay. What can I learn from this?
- 17:37
I gave you the list of six that Apple uses, and here's what's available in our world if you're not developing on Apple silicon and an Apple hardware. You still have oblivious HTTP and blind signatures.
- 17:47
There are libraries to do that. So we don't have secure enclaves, but we have TPMs. Uh, almost all Intel and AMD hardware now has TPMs. And then in the cloud environment, they have virtual TPMs, uh, that provide the same behavior as a TPM.
- 18:00
And again, that, that's where you put a bunch of your private keys that are tied to that public key that I talked about. These are available for us, secure boot and hardened operating system.
- 18:10
Um- remote attestation is kind of available. It's kind of tied to the TPM. There aren't great standards yet, um, but there is a little bit of work there. Transparency log, there are two open ones.
- 18:23
One's called sigsum, the other one is called sigstore, if you've heard of them. If not, um... [lips smack]
- 18:30
And then confidential VMs are just becoming available on cloud providers with GPUs. So confidential computing has been around for a while, uh, but now you also have to have confidential H100s.
- 18:41
And only H100s support, and H200s support confidentiality. What that means is that their memory is encrypted, so if you were to physically go up to the H100 and, like, try to look at its RAM, you wouldn't be able to see what's going on there or figure out what's going on there.
- 18:55
And then finally, what we have that Apple doesn't have is we have open source and we have reproducible builds. We have the ability to link the source code to the binaries.
- 19:02
Uh, and so we can have s- security research look at the source code as well as, you know, black box test the binaries and develop confidence in what the server might be running. [lips smack]
- 19:13
All right, what's next? Okay. So Apple has set the standard for private AI and the market is definitely following. Um, it-- That was in June of 2024. It wasn't actually released until October of 2024.
- 19:25
Azure OpenAI-- or sorry, Azure AI, not Azure OpenAI, is doing private inferencing starting as of September. They're still in private preview. And then about a month ago, Meta of all companies, I guess I'm recorded, Meta of all these great companies, uh, also added, uh, private processing, which i- if you read their blog post, it's like they copy
- 19:44
and pasted this. Maybe they used Llama to rewrite it, um, into their language, but it's essentially identical, which is great for a- all of us thinking about privacy. Um, and sure, I'm sure, uh, WhatsApp also doesn't want those, like, press releases like I showed earlier.
- 19:58
Um, so I'll just close by saying we're building the same thing, uh, but for everyone else. If you're not on Apple or you're not in WhatsApp, we have it.
- 20:05
Uh, it's called, uh, Confident Security. Um, and if you'd like to talk more, let me know. By the way, this is an anti-AI shirt, uh, which means that if you take pictures of me, it will confuse all the facial recognition stuff.
- 20:18
We have others. If you have some cool questions and wanna talk afterward, if he deems it worthy, I will give you an anti-AI shirt. We also have some other privacy-based swag in the back, so come hit me up.
- 20:29
Thanks, everyone. [upbeat music]