The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:03:25 · Harry Stebbings, Satya Nadella
[00:03:25] S1: a seminal piece, ten days ago or so, and you said that we should assume a model is compromised. And I just wanted to expand on that. Does that mean that we can't or shouldn't rely on the disclosures of the model provider companies? [00:03:39] S0: Yeah. I think the challenge, the way I see it right now, is the models are becoming more and more capable. Right? Scaling laws are working. Except one of the assumptions I had was that you'll have alignment just fall out of scaling, and it doesn't seem to be. And there's more science challenges there that need to be tackled, and mechanistic understanding is also elusive. And so given that, I think the best posture to have, if you want to continue to use these sophisticated models, is to treat these models in a trust architecture. I mean, after all, zero trust is not something new. Right? We've had zero trust architectural approaches. So the claim, at least I'm making, is let's create an engineered zero trust environment that can contain any malfunctioning agent before it does any harm. Right? So that's the point. Because if you really wanna have agents that are more capable and yet less aligned, then you need to have a trust system that you can rely on. [00:04:35] S1: Can I ask, what does that trust system look like? Who's the architecture of it? Who's the arbiter of it?
00:04:35 · Harry Stebbings, Satya Nadella
[00:04:35] S1: Can I ask, what does that trust system look like? Who's the architecture of it? Who's the arbiter of it? [00:04:40] S0: It's a great point. And so that's why I kind of look at it and say, You leave AI safety outside of this, right? Because that alignment, for example, at least my take, is that it really mostly has to show up in training, right? In fact, maybe even in pretraining. I don't think you can even RL your way to alignment. You have to truly get alignment to be something that truly gets embedded in the weights right at the very beginning of it. And so that's the model itself. Hopefully, at some point, we'll get aligned. If you leave AI safety, then the trust architecture is going to be just, you know, what a company like Microsoft does or all the cyber folks do. So what we have done, for example, is we built these two things, right, agent three sixty five and perception, right? What does it do? It gives every agent an identity. It then logs all of its activity, all its tool calls, in fact, every model call and the COT unit. In fact, that's where I'm also asking that all the COT better be in English and not in new release. And then you can check using multiple models. It better be tamper proof. You can't have the agent go modify its own logs. And then on that, you set policy and then you behaviorally monitor the activity of the agents so they're confirmed to the policy. So that's the architecture. [00:05:53] S1: We don't have that architecture today. Is that just symbolic of where we are in the production?
00:05:53 · Harry Stebbings, Satya Nadella
[00:05:53] S1: We don't have that architecture today. Is that just symbolic of where we are in the production? [00:05:58] S0: But our architecture exists. So there are two... I think people conflate what I'll call an engineered security, zero trust architectural system for containment and AI safety. Because I think people say, AI safety is hard and elusive and not yet achieved, so therefore there must be nothing we can do on security. And I'm not saying that these two things are all got to come together at the same time. In fact, I would just say, Hey, we have to deal with the situation as it exists, so let's build a trust system outside, in fact, of the model, and then just be super aggressive in setting policy and monitoring for policy deviation. On the monitoring side, if we think [00:06:37] S1: about chain of thought, how reliable is that?
00:06:37 · Harry Stebbings, Satya Nadella
[00:06:37] S1: about chain of thought, how reliable is that? [00:06:40] S0: It's not. Right. We know that the chain of thought itself unfaithful, and in fact, it can be made worse when the model company says, Well, it's unfaithful, and I'm going to hide it. And so it becomes even more problematic. And that's where I think a little bit of the push for standardization and the COT, at least for trusted partners, being available as is so that it's inspectable, it's auditable, and I can check using multiple models and what have you. So that's going to be one of the in fact, that's one of the things that may even drive more open weights, quite frankly. Because if you can't... It's funny. Close rate models are more powerful. But if they kind of don't recognize the need to be more transparent, especially as their trusted partners use it, then we may have a problem. Can I ask [00:07:27] S1: you when we think about that rise of open source? I completely agree with you in terms of closed being more powerful. Open source stays predominantly Chinese models. Is that a concern then when we think about safety being baked into pretraining, when we think about disclosures as we said there, and then juxtaposed with the majority of open source being Chinese?
00:07:27 · Harry Stebbings, Satya Nadella
[00:07:27] S1: you when we think about that rise of open source? I completely agree with you in terms of closed being more powerful. Open source stays predominantly Chinese models. Is that a concern then when we think about safety being baked into pretraining, when we think about disclosures as we said there, and then juxtaposed with the majority of open source being Chinese? [00:07:45] S0: Yeah. I mean, I think there are two sets of things. I do believe if you sort of step back somehow AI safety not being there is not a good thing for China, is not a good thing for The United States, it's not a good thing for The UK. So I think that there's a part of everyone... Because this is not something that somehow an unaligned, open weight Chinese model will probably attack even China. So it's not like you can sell the rest, think that this will only be used as an attack vector. So setting that aside, I'm pretty pleased with some of the progress even the open weight models in the West are making. Obviously, Jensen's working on it, the other folks are working on it. We will have even the MAI models that will be not open weight. Think of it more as licensed weights. Think of it that way. I want to turn this into just a simple, straightforward software business as opposed to a network effects intelligence business. So I think there will be different business models. And I ultimately trust in your ability to use those models and control those models is what's gonna determine what gets used.
00:07:45 · Satya Nadella, Harry Stebbings
[00:07:45] S0: Yeah. I mean, I think there are two sets of things. I do believe if you sort of step back somehow AI safety not being there is not a good thing for China, is not a good thing for The United States, it's not a good thing for The UK. So I think that there's a part of everyone... Because this is not something that somehow an unaligned, open weight Chinese model will probably attack even China. So it's not like you can sell the rest, think that this will only be used as an attack vector. So setting that aside, I'm pretty pleased with some of the progress even the open weight models in the West are making. Obviously, Jensen's working on it, the other folks are working on it. We will have even the MAI models that will be not open weight. Think of it more as licensed weights. Think of it that way. I want to turn this into just a simple, straightforward software business as opposed to a network effects intelligence business. So I think there will be different business models. And I ultimately trust in your ability to use those models and control those models is what's gonna determine what gets used. [00:08:47] S1: As a venture investor, I have to predict where traffic will go today between closed, open. When you think about the two different dimensions, which is, like, traffic and dollars, if we look forward, say, twenty four months, what percent of traffic goes through open versus closed do