Scaling Europe
00:16:12 · Jack Miller
[00:16:12] S2: So as I I think, you know, anybody that reads, the the report on on the OpenAI incident, is... I don't think you can read that report and not update your views on the world. It's just such an an insane thing to have happened. And and, you know, there's there's obviously the report, then WeShed internally was mandated listening for for everybody at the company, the the three Dwarkesh episodes about about this. And so, obviously, Ryan Greenblatt one was not after the report, but but covered the same topics. So that was mandated listening for everybody. Even even the salespeople, they've gotta they've gotta have a listen to it. And I think what it shows is is a couple of I think number one, it shows that we do not have a good handle on what these models can do. It is remarkable to me that they were able to, you know, do the sorts of things they were doing with this kind of secret message board with the coordination. It it is just incredible that that occurred. I mean, using the models day to day, they still make quite silly mistakes. And then seeing that this occurred internally is just... Is actually just a remarkable fact about the world that you have this kind of spikiness and capabilities that's that's hard to wrap your head around. I think the second thing is I think the cybersecurity side is really interesting because in some sense, the cybersecurity capabilities are pretty pretty good. They're they're pretty remarkable that they were able to do this, but I think it's actually more of a reflection on the incentives that exist in the industry today. And those incentives are towards, getting the best model out that you can and not necessarily observing what's going on within, your organization. So, you know, if you read the report, you can see that actually they use AI models to discover what has happened. Right? So, in in a post post hoc analysis, they use AI models to basically go through the logs, look at the traces, determine what's happened. Now they could have easily been doing that during the time. Right? If you had a monitor looking at the way that these models were doing the evaluation, you could have seen that that this was occurring. Right? And so that tells me that it's not necessarily a technology problem. It's... It is a technology problem in some sense, but it's also a, incentive problem.
00:18:35 · Jack Miller
And so one of the big pushes that we have at HelmGuard is to change the incentive landscape for security and compliance in general, to move to a world where if you are verifying something about a company, you don't expect that to be a sample. You expect that to be on the population level. You expect everything to be monitored because it's so cheap to do so. And some of our, work with Callosum actually is bringing down the cost of this sort of agent monitoring by, like, orders of magnitude. And so to me, really, it's an incentive problem, and we've kind of built HelmGuard to change the incentives in this security and compliance industry to move towards a world where companies need to constantly monitor what's going on internally, and that that will be the de facto standard moving into the future. And I think, you know, I I think that that really is is a it is that incentive piece which is which is remarkable to me that that you can spend this much on compute. You can be running this eval, and nobody is checking what's going on. That to me is the central fact that we should be focusing on, and that we need to change the way in which the industry works from a security and compliance perspective. Because, look, this is gonna be happening in OpenAI. Obviously, they trained a persistent model. It's pretty clear that, like, there were some pretty, like, specific circumstances there. But I think what you're gonna see in, like, a year's time, these sorts of incidents, I wouldn't be surprised, occurring in the enterprise, especially with the use of open source models, especially with continual learning becoming more and more of a thing. I don't think it's unlikely that we'll see a case of this outside of the labs in the next year. And the question is, like, if you're a 10 person IT team, what... How are you gonna how are you gonna deal with that? That... It just seems like a like a very difficult problem to solve, and and I think that goes back to our thesis of bringing these frontier capabilities to the everyday enterprise. Right? Because I think they will be struggling with the same governance concerns that labs do, in in a year, two years' time.
00:20:48 · Sebastian Johnson
[00:20:48] S0: I mean, it's crazy. I mean, it's a... It was absolutely absurd what happened. How did it change? How how... Did did it change your... Both of your perceptions about the business that you're building or the scale of the problem that you're facing?
00:21:03 · John Daley
[00:21:03] S1: I I think it made concrete some things that we've held for a while, but I think have been hard for our customers to necessarily understand, and that's a lot of what we're doing in our sales process. But I think there is that, like, if you look at the traditional set of security and GRC work, one thing we're already doing there and we're doing for the the better part of a year now is saying things that were not sort of impossible, but were not economical to do are now economical because the agents can execute that work at expert level at very low marginal cost. And so that changes where you should be spending time and effort as an organization, the types of decisions that you can form all with that work. That's the kind of internal risk and security and the third party prongs of our platform. On the agent assurance and governance side, it's sort of the same idea, but different that you can only do this work if you're operating on... In, like, AI native terms. Right? You can't be keeping up with what agents are doing in your enterprise if you're not operating in an agentic way, right, on those terms at that speed with that same type of dynamic reasoning and and assessment capability. And so that is, like, concretely what it means to be bringing these capabilities to bear in a defensive way. And I think if we do that correctly, it's actually quite a... An optimistic sort because unlike a threat actors who are operating from the other side, you should be able to know everything that's going on within your organization and have that defensive edge. Right? But it is dependent upon actually bringing to bear and effectively deploying systems like ours, we hope it and others that can help you do that as, you know, what agents are running, what are they doing, and on a dynamic basis because they're acting dynamically. Is it consistent with what we're expecting and looking at at the reasoning and the traces and the tool calls. And and so both in a static configuration and at runtime. And that's really how we're extending our platform, and that's a big set of product investments that we're making for this round.
00:23:06 · Sebastian Johnson, Jack Miller, John Daley
[00:23:06] S0: Yeah. I mean, it's it's so interesting that people have just been so focused on kind of, like, value and Yeah. You know, just trying to get as much deployed and loads of people have been given these budgets to spend on AI tools, they're, like, laser focused on on trying to see something happen and value that they're skipping all this vital infrastructure in the middle to actually make sure that the thing is safe and secure. Well, look, very grateful to talk to you both. We're out of time. Massive congratulations to what you both have done so far and and the round that you've raised. It's gonna be really exciting to see the next stage of your journey and keep me informed as as as things grow. [00:23:38] S2: Of course. I I can't wait to be doing this at the Series A. [00:23:41] S0: I look forward to it. [00:23:42] S1: Yeah. Thank you so much, Seth.