00:24:47 · Hanah-Marie Darley, James Wise
You should have oversight, you should have measurement, you should have accountability, you should have um accessibility, you know, the CIA triad, all of those things are really critical. But with agents, the way that you get them isn't the same way that you get them for human users. And that is the important point, I think, that puts us squarely in the middle. [00:25:06] S1: Yeah. And I look, I personally I find some of the discussion around sort of AI doomerism and capabilities overly anthropomorphized, uh, often exaggerated. But as you say, it's undeniable that there are new risk factors at play here. And that's even for organizations at the frontier of best practices. How do you think about sort of some of the broader challenges that, you know, given your previous work, large enterprises, but also even governments face and ultimately individuals face now through this, you know, new technology that we have access to? And you know, are are can you point to good outcomes or good best practices that governments are taking on now to sort of address the threat today, which is cybersecurity, versus some of the sort of more philosophical challenges people might worry about?
00:25:52 · Hanah-Marie Darley
[00:25:52] S0: Yeah, so it's interesting you say that I was with the EU yesterday, I was with the uh European Parliament and European Commission, and we were talking about AI data, all of this. Uh we were talking also about agents, of course. In terms of how people are best placed to address this, I always go back to choose the right solution for the problem that you're trying to solve. It's true when you're bringing on consultants on board to a project, it's true when you're picking an AI discipline to apply to a project, it's true for securing agents. If you are trying to secure agents, you have to understand how they operate. You have to be able to provide oversight in terms of auditable logs, but beyond the logs, they're not self-evident. And a lot of security teams feel a lot of comfort from logs that are a little bit overturned in the way that agent traces work. They are not always clear in terms of what is valuable, what's meaningful, and what's actionable. And so beyond just understanding where you have agents and what they're doing from an auditability perspective, you also need to have a real clear understanding of agent behavior. And I think in terms of where both enterprises and governments are going to need to invest in the future, it's a really deep understanding of agentic behavior. Um, being able to measure specific changes, being able to account for anomalies, being able to see trends emerging. Um, you know, in in one study they found that agents took tea breaks. Um and the reason the reason some people will say haters will say they were trained on European models, other people will say um it's because they're trained on human data. So agents reflect humanity because they're trained on our data. Um so when there's deception, unfortunately, there's also deception in humanity. That's not a unique concept. Uh and so the best practices really are having the right specific systems designed around agents.
00:27:31 · Hanah-Marie Darley
A couple of key principles there that are really critical that I talk to the EU about, but I would also talk to enterprises about, is not just having a data classification system, whether you're talking about GDPR, personal or business data, or whether you're talking about, you know, tier one, two, and three of your enterprise data, but having an understanding of what makes the data within those schemas actually that classification or that purpose. And then being able to adapt in a system where it's not a clean separation. So there's some types of data, I would say almost every AI system now is a little bit personal and a little bit business use. And on the enterprise side, I've yet to meet an enterprise that comes to me and says, we are not worried at all about data security. We are confident about our classifications, we know exactly where our data is, and we can verify that everything that's supposed to be tier one is actually tier one. Um I've I've yet to meet that enterprise. I would love to talk to them if they exist, because I'd love to write down what they're doing. Because of that, you then have a really near-term challenge of understanding your data. You have a longer-term challenge of that data effectively governing how future decisions are made, but also changing avenues and changing access points as the agent continues to adapt through context. And so what we're talking about next is the context layer. What we're talking about next is the context challenge. And from a government perspective, that's owning your own context from a sovereignty perspective. So can you guarantee that the skilled knowledge that your workers acquire, that your government adapts, uh, can maintain itself within your borders? Or is that starting to a little bit diffusely spread out? But from an enterprise side, can you guarantee that your operations and systems, as they're being transformed by AI, hold true to your values and principles as to how you want those systems to operate? And again, that comes from practical controls that are at the agent level. I do think the other bigger picture challenge for both of these groups is thinking about again, the human piece. Um so I come back to human in the loop a lot because I think it's one of the most misunderstood concepts in AI.
00:29:30 · Hanah-Marie Darley
And so quite frequently, especially on the enterprise side, we'll talk about, oh, well, we have human in the loop governance in place. And okay, great. So what does that look like? We guarantee that there'll be a human user at all of these checkpoints. Okay, so you irreversibly slow down the system that is designed to speed you up. The second challenge is you don't specify who that human user needs to be. It has to be an expert. The person has to have context around what decisions are being made. So for example, you're gonna make a deal decision. You and maybe a few other people have all of the context around the market research that you've done, whether or not you think that the TAM is reasonable, whether or not you think the business model is reasonable, et cetera. What if you had someone who makes coffees down the road make that decision? Could they make a decision? Yeah, but they lack all the context. And so it's the same principle with human in the loop. Very few policies and very few operational mechanisms specify who that person should be, how much context they need to have, what credentials, if any, are required to make those decisions, and what oversight or recourse there is for wrong decisions made. Um, it's just human decision equals better than AI decision. And in a lot of research documentation, that's not necessarily true. Sometimes you gotta let it cook. The AI is actually better left to its own devices from just a statistical output than human decisions are, because the humans who are making the decisions lack context, have a high cognitive load, that they're tired, that we make worse decisions at 5 p.m. than we do at 9 a.m., etc., etc. So when it comes to both regulatory best practices, human in the loop was one of the first places and first principles that pretty much every regulating body went to to say, put people in it. It sounds great, but the practical implementation is a little bit more difficult than that. On the enterprise side, it was a stopgap and kind of a band-aid for we don't really have a specific plan. So what we're gonna do is try to gate this with people until we can operationalize the plan that we really think is gonna be functional.
00:31:24 · Hanah-Marie Darley
And so I think that has to it's already fading from an industry perspective as a good solution, but I think that has to make its way to both operational on the ground, how enterprises are implementing solutions, and also government in terms of how they're planning to regulate.
00:31:39 · James Wise
[00:31:39] S1: I think from a personal perspective at Balderton, you know, we're a small entity by by uh the standards of companies you normally work with. But the really interesting thing about adopting Geordie wasn't just as you know, it's a very powerful platform. It forced us to have some of those conversations. Which tools are we happy with, agents accessing, what kind of decisions need a human in that process, and what kind of decisions are we happy with agents doing certain amounts of the work from? Frankly, because of the sort of plain English way you can set security policies now, it's a lot more understandable to have that conversation with our head of security, with the head of finance, with myself as an investor, in a way where you know configuring the settings of some of the older cybersecurity tools, you know, having a debate about sophisticated pen testing wasn't really a everyman discussion. It's less accessible. It was far less accessible. Whereas I think Geordie, you know, by necessity has to bridge those conversations because security has now become the flip side of the coin to every single um ability you want the AIs to have, right? You can't take on these new abilities and use them unless you also consider security implications. And that means both parties have to be in the room, as it were. And I think Geordie does not just force that conversation but enable it going forward.