Privacy-Preserving Intelligence — Steve Korshakov, Bee (acq. Amazon)

Talk transcript

1. 0:00

   [upbeat music] Hello, everyone. I hope this talk will be shorter.

2. 0:16

   Uh, uh, I'm from Amazon. Um, our company was acquired about eight months ago, and we built the, uh, AI wearable, which is on my hand, uh, which is essentially a microphone that records everything and builds your personal agent, personal AI.

3. 0:35

   And, uh, on top of that, you can extract all the data that you record and plug it to your systems or agents, uh, and do whatever you want.

4. 0:47

   Um, just to be-- to get in perspective how confidential, how, like, private data we're capturing, uh, a single person usually, like, captures about ten million tokens per year. So this, um-- And even within, like, a first week of recording, uh, uh, people usually tell, uh, extremely sensitive stuff to their friends, to their family.

5. 1:12

   Uh, you can learn virtually everything about the person within the just, like, one week of wearing the Bee device, which is extremely sensitive. I think we're one of the most sensitive, uh, capture device on the market now.

6. 1:25

   So, and, uh, because of this, we had to encrypt everything, and our mission was to not have access to any of this data and not being able to look at it, uh, anyone at Amazon.

7. 1:39

   And it became a little bit challenging for us at Amazon because, uh, Amazon itself provides strong security and privacy guarantees. But if you Amazon and you using Amazon stuff, there is, like, much more, uh, serious security stuff you need to do.

8. 1:59

   Uh, first of all, we define, like, few core principles what we needed to do for our specific agent. First of all, we believe the agent should be, like, working all the time, nonstop for, for your good.

9. 2:12

   Um, it should be doing stuff on your behalf, uh, and, um, we should not consume, uh, customer resources such as batteries and stuff. So this way, uh, we, uh...

10. 2:27

    So this leads us to one specific design, uh, of the, um, of the our system. Current system usually builds the, uh, on state, uh, on request response, uh, system where you send, uh, req-request to a lamb from your, say, iPhone, uh, calculates something and the backend it gives you back.

11. 2:52

    Unfortunately, we already see that this is not enough,

12. 2:56

    uh, that we need to, uh, that we need to run stuff continuously and sometimes for days. We can see, uh, we can see this, like, as a, like, glimpse into the future how cloud code works.

13. 3:11

    So, like, just few months ago, it was, like, more like request response stuff, like change this, change that, and now it can works for, like, hours for us. We think the same will happen to all your, like, personal agents anyway.

14. 3:24

    So, uh, because of this, we built, uh, a stateful runtime with persistent memory, um, that we still don't have access to. Uh, it can connect to different tools. It can connect to any, like, third-party services if you, if you program it to.

15. 3:41

    Um, and we don't require the user device to be online, so it's fully autonomous, but at the same time, it's fully controlled by the user. Um,

16. 3:53

    so encryption system is built, um, uh, on few, on four, like, core ideas that we need to follow. First of all, the key lives and managed only on customer device, so it's user's iPhone or Android device itself.

17. 4:08

    We don't have the key ourselves. We don't persist it anywhere. So keys is, is, um, is stored only on the customer phone.

18. 4:20

    Everything is encrypted. We don't have any opt out. There's no way to disable it. There's no way to bypass it. Uh, at the same time, we, uh, to protect ourselves from, um, like, internal threats, uh, we do fully transparent and, and audit, uh, of our-- all our workloads.

19. 4:42

    And, and we, on top of that, we try to minimize the dependencies on the, uh,

20. 4:49

    on what we can trust really. Um, so, uh, any security system, if you do end-to-end encryption or any kind of encryption, there is a huge problem, is key management.

21. 5:02

    So the first step is, like, I want to tell you how we manage the key. So we start with the, uh, the key, as I mentioned before, starts on the phone and at least persisted on the phone.

22. 5:14

    Then the phone connects to our back end and runs very sophisticated attestation, uh, pipeline, uh, that verifies both integrity and that the specific workload is inside of public, uh, transparency log.

23. 5:30

    We use Sigstore for our transparency log, and anyone can go there and try to look and verify that this workload is genuine. Uh, the, the method is too complicated to include in this, uh, talk, but we will publish details, uh, at some point.

24. 5:46

    Uh, once the, uh, once, uh, attestation was finished, we, um, the, the client shares with the, our main front-end back-end, and back-end then replicates this key with, uh, the similar nodes, uh, that runs within our confidential compute.

25. 6:03

    Because we can't leave the unencrypted data out of our perimeter, uh, all our... We run our own inference too, so this puts us a little bit, uh, more complicated task, uh, than typical AI company.

26. 6:19

    We run, like, all kind of models, all kind of inference, uh, uh, software. And, uh, so,

27. 6:26

    um, yeah. And we don't replicate code to this inference node. We, like, replicate on a, on specific ones, limiting the scope of what we can do. And on top of that, we introduced everywhere where we have the keys in the memory, uh, uh, the forced expiration of seven days.

28. 6:45

    Uh, we picked the seven days because we think it's, like, how much realistically the time horiz- horizon for the, like, something useful can be done for the user. Uh, 24 hours will be too low because you can, like, not open your phone for, like, 24 hours and something will be missed.

29. 7:02

    And, like, so we pick, like, about seven days.

30. 7:07

    Um, then we need, you know, we need to ship some- something to the production, and then the, the biggest question, like, how we can, uh, well, not ship something that will compromise anything.

31. 7:19

    Our goal was to, uh, build a system that no one outside of, uh, Amazon will be able to ship anything unnoticed. Obviously, the software has bugs, uh, have problems, but we shouldn't be, well, being able to ship anything.

32. 7:36

    So the, we solve this by two, uh, two-tier system essentially. So there is a dedicated team inside of organization, inside of, uh, Amazon, and maybe, maybe not even one, I would say, uh, that, um, manages the privacy part of this, the transparency log.

33. 7:54

    Our team, when we ship the software, we can't influence them, we can't control them, and we hard code their encr- uh, their signing keys inside of our client apps and our backends.

34. 8:05

    So we can't really, uh, sh- um, uh, we as the team, we can't do this, and, like, it's very, very hard and very high level, uh, um, employees need to sign off to any kind of change.

35. 8:21

    So it's, like, at a big company like Amazon, it's virtually impossible really. Um, and, uh, and we do this in two parts because this, this process too slow, so we split it in two parts.

36. 8:34

    So, like, the first one is to build the base image, which we put some kind of base software for our, that is needed for our own team, like the tools that measure the build, measure the manifest, measure workloads and data that we need to, uh, put to the node.

37. 8:50

    Um, and, um, and then when we want to deploy, we do the process the same, similar time. Uh, we got the base image, and then we deploy the, uh, to transparency log specific manifest that any dev...

38. 9:05

    And, uh, this setup helps us to be able to security audit companies and inside, outside to anyone to... Well, we're not doing this public, but, like, we to, like, very, um, high-profile audit companies, we work with them all the time.

39. 9:22

    Um, we can provide any image, any, any data that was deployed ever, so we can, like, trace any possible, uh, weak spots, uh, if we, like, deploy something wrong,

40. 9:34

    uh, which we do not. Um, then after ver- uh, self-verification of VM, it's, uh, issues a certificate that embeds all encryption pro- uh, all transparency proofs, attestation documents into certificate itself.

41. 9:48

    We are using private CA because you can't do this in public certificates because it will populate the public, uh, transparency log, so we had to use the private one.

42. 9:58

    Um, we probably will introduce the, uh, extra proxy that will do a normal TLS with attestation with, like, lighter mode, uh, but we don't have this yet.

43. 10:10

    Um, yeah, that's essentially what we built. Thank you.

44. 10:16

    Any questions? [audience applauding]

45. 10:21

    Any questions? I think you should shout it.

46. 10:25

    I'll just shout it. Uh, so-

47. 10:27

    Yeah

48. 10:27

    ... I'm interested in, uh, what the process was like, like, having your own sort of, like, values and approach to security going, to going inside Amazon? Were, were there things that you had to change that had, hadn't been changed prior to, like, going in there?

49. 10:45

    Just as far as, like, uh, pro- processes and sort of values around security, because I know they've, they've got very hard-coded, entrenched ways of doing, doing things. Yeah.

50. 10:57

    Can you repeat? [laughs]

51. 10:59

    Yeah, sorry. There's a lot, a lot of words. What was it like, uh, from a, um, a process standpoint, what had... Were there things that had to change when you went from being, like, your, your own small startup into existing within Amazon?

52. 11:15

    Oh, what changed when, uh, with, like, when we joined Amazon?

53. 11:18

    Yeah.

54. 11:21

    Well, the, the, the big change that we, uh, before, like, you run on Amazon, and Amazon gives you per- like, guarantees as a customer that they can see your data, but once you're inside, this changes a lot because you, Amazon, like...

55. 11:37

    So that's why you need, like, to provide more, uh, protection on top of this. So we need to protect from our internal threats too. Uh, so that was a big change.

56. 11:46

    So, um, before that, it was, um, just kind of easier, I would say, to configure everything. Um, I'm not sure I can tell much, honestly. [laughs]

57. 11:58

    Did, did you guys do any, like, reprovisioning your stack? Like, or... 'Cause I'm assuming you built on AWS side.

58. 12:05

    Yes. It's just normal EC2 instances. Yeah Yeah, yeah. We almost-- We not using like-- Yeah, almost everything we built from scratch. But well,

59. 12:16

    um, we try to use like the existing stuff like, but it's more like common, like popular software. And we built, uh, try, I-- We tried to minimize amount of code that we produce, so it's-- I, I calculated before this talk, it's just like about twenty K lines on memory safe language.

60. 12:35

    So it was very small scope that we were able to audit and verify that all this kind of stuff. And most of this code is just verifying attestation really, and then everything else can be like reused and like very, we like, you know, it's very trustworthy, I would say, software.

61. 12:49

    So we didn't try to-- don't invent, uh, um... Yeah, we don't try to invent. Like when, when I was at Telegram, like we reintroduced, like built our own crypto, and that was like questionable way of doing stuff, so I try not to do the same at Amazon, obviously.

62. 13:08

    Yeah, that's what we do.

63. 13:11

    Any other questions for Steve? Up in the back.

64. 13:18

    Yeah, quick question. So I guess we're the tail end of the, uh, diehard security enthusiasts. Um, just thinking about the AI side of things here, first person I've heard really talk about things like encryption layer.

65. 13:33

    I don't have a lot of experience with that. So we know AI agents can go rogue or have some kind of a mistake in, in the databases or even, you know, the rm flag, rm forward slash.

66. 13:47

    It's, it's everything. But what, uh, what can we do to prevent the machine from saying, "Let me erase our hard drive or tap into it"?

67. 13:56

    I've not heard of that happening, but I don't see that being very far off, and there could be an AI-based ransomware attack on an individual and being encryption-

68. 14:05

    Well-

69. 14:06

    What-

70. 14:06

    I, I just prefer them not to put to the computers, uh, to personal one. Uh, so we, we did several experiments how to tame them not to do bad things.

71. 14:18

    Honestly, I think nothing works except like sandboxing and just not giving them a way to hurt, hurt themselves. It's like, you know, our brains, they can stop the heart at will, right?

72. 14:30

    So otherwise, you know, they will be-- we have much more problems. So I think the same, we shouldn't give them a way to do any harm. That's the only way, honestly.

73. 14:39

    And, um, yeah, and put something between if they want to change something. Unfortunately, I think that's the only way, yeah.

74. 14:48

    Open claw, like Open Claw is already taking action on behalf of agents. It could happen today.

75. 14:52

    I surprised they are not represented on this, uh, right? Like they're screaming so much about security, but they didn't came to this one. Um,

76. 15:01

    uh, I'm not-- I tried Open Claw. It's like it was... Once they started to try to tighten this down, it became much less use-useful. Um, so I think their approach is not really that good at all.

77. 15:15

    So I would love to have wild agent. That's, that's our goal too. But we try to just deploy the sandbox for specific agent, and it will just-- they just can't do much of the stuff.

78. 15:27

    Everything else fails, unfortunately.

79. 15:31

    Cool. All right. Well, thank you so much, Steve, for the presentation. That was amazing. [outro jingle]