No Priors: Artificial Intelligence | Technology | Startups
00:07:11 · Sarah Guo, Maxim Bar Kogan
[00:07:11] S1: Yeah. I think one of the core reasons, obviously, the foundation model labs are going after code is because it is very powerful in general and can do, you know, in theory, all things software can over time. The flip side of that is it can do all things software can. Right? And so I joyously am already in the camp of a, having been over permissive with my agents such that it deleted data permanently and caused rework. So I'm like, [00:07:41] S0: oh, okay. [00:07:42] S1: I think I need some guardian spirits around it. Given your deployments today and talking to large enterprises, what is the state of deployment? Right? Like, much do you see that's within these more scoped, like, studio like platforms versus, you know, free riding coding agents? How much are you actually seeing in large enterprises and in different sectors?
00:08:09 · Maxim Bar Kogan
[00:08:09] S0: Yeah. So, I think right now, in our typical enterprise, we break it down to three categories. So, we break it down to various SaaS platforms that are typically more low code, where people build agents in this drag and drop way, and they're not really autonomous agents. I would think of them more as the automations. Then there are first party agents. People are building in their cloud, potentially because it's an application they want inside the company, or even a product they're planning to release to the customers that is agentic. And then the third category is very autonomous coding agents and assistants. Of these categories, I would say roughly at this point, over 50% is the autonomous coding agents and assistants in the average enterprise. Then probably 45% is those low code automations, and the last 2% are really the first party ones that they're building themselves, because obviously it's much harder to build effective agents, and it's much easier to adopt agents off the shelf or build them with low code. And that's what we're seeing. We do think that the automations are also the fastest growing category. So, it used to be that only developers would see cloud code growing like fire in our customer base, and now we're seeing a cloud co worker growing even faster. We're starting to see, to our own surprise actually, people adopting OpenCloud as a legitimate sanction tool in the company because the CEO is very driven to adopt AI. So, I think that today, autonomous Azure is by far the fastest growing category, today typically comes without any controls.
00:09:58 · Sarah Guo
[00:09:58] S1: So enterprises already buy, let's say, 100,000,000,000 of security today. They have lots of different protections at the endpoint and network and cloud and identity domains. What's relevant here for securing agents, or is none of it? Like, how do you think about the existing protection set?
00:10:24 · Maxim Bar Kogan
[00:10:24] S0: Security is always a space where you have some overlap between different tooling. In this case, and you have the concept of defensive debt as well. So, you want to have defenses at different levels of your technology stack to solve the problem. And that said, I think in this space, a lot of enterprises are kind of helpless because I'll take an example, the identity approach. Traditionally, if we have a software system that's running in our company, our first and most important control will be to limit what permission it has. And then no matter what, even if it goes wrong, even if it's compromised, it can't typically do stuff that it was originally allowed to do. But with these autonomous AIs, with these assistants, with these coding agents, we kind of want them to have our permissions because we want to tell Cloud Code to do something or Cloud Code work to do something, and we want to then go have lunch, we want to come back and see that it's done. And we want to give it so many diverse tasks as well that we kind of can't find the right set of permissions to do. So, suddenly our identity security software is not very useful. Then if you think about endpoint security or API security, if we tell our Cloud Code that we want to recreate a database and it should delete it and recreate it, that's great. That's gonna save our DevOps team and our platform teams a lot of time. It's a great benefit of Cloud Code. But if Cloud Code is working on an unrelated task and suddenly thinks that maybe the right thing to do is to delete our database and recreate it. Maybe we don't want that to happen. And unfortunately, our endpoint providers or API security tools, they don't know what cloud was thinking, why is it doing what it's doing, right? So, a lot of these existing tools, they don't have the context to understand what these very flexible, unpredictable systems are doing. If you're not building some kind of controls that are built for these systems, then you're either going to end up limiting them a lot, making them almost much less useful to the enterprise, or you're going to miss a lot of pretty dangerous things that they might be doing.
00:12:45 · Sarah Guo, Maxim Bar Kogan
[00:12:45] S1: As somebody who has worked in security for a long time, my first very traditional instinct on a problem like this is like, that sounds like a problem for a proxy with a policy engine. We make some rules and make the rules smarter. Like, why doesn't that work, or did you try it? [00:13:03] S0: There are a few things that, I mean, proxy's integration method, I would say. So, there are some AI systems where like you would want to integrate with a proxy if that's the easiest way to do it. But number one, there's a lot of systems where that's just not viable technically, because AI today runs on the cloud, on someone else's infrastructure, on your endpoint, and just proxy is not always an option. And the second thing is the question, okay, great, you're proxying, so you're seeing the data. You're seeing it, but that's not the hard problem. The hard problem is understanding what I should do now. Turns out that in the case of AI systems, that is the hard question. Like, what is the engine that needs to underwrite these different actions and say if they're okay or not? Because we need to be able to understand what another AI system is thinking, what is it planning to do, and then have our own opinion on that. And consider we're trying to understand some of the smartest models in the world are doing the right thing. So, who are we to do it? How are we going to do it correctly? Right? And so, that turns out to be a really difficult technical question. [00:14:11] S1: Part of the solution for Onyx has been training its own models. Like, what can you say about that?
00:14:18 · Maxim Bar Kogan
[00:14:18] S0: If you try today, let's say we're trying to build a solution to oversee and kind of control how other agents are operating. Maybe the first thing a lot of our listeners might think is say, well, I'll just ask Cloud Code to do it. In a sense, they would be right because Cloud Code is great. And maybe we can ask it to spawn a version of itself for every agent that we have and kind of keep monitoring everything that agent is starting to do. And if you think that there's a problem, intervene. So, that approach, obviously it's pretty naive, and there are some ways in which it totally fails, we could talk about, but it has some merit to it. So, it does seem intuitive that it's a good idea to have capable agents reviewing what other agents are doing, same as we have capable humans reviewing what other humans are doing. But then the problems that you're gonna run into is how do I make this work from cost, latency, reliability perspective? Because if I need to run an agent for every agent you're running as your security vendor, you're gonna be paying for me more than you're paying for your AI, right? So, it's pretty much a dealbreaker. And also, it's going be so slow, so you're not going be happy with whatever latency you're going to get. And so, the challenge then becomes how do I know what are the times where I need to interject with these smart agents to look at what's happening? And that's when actually what you want to do is you want to train very smart models that are, actually let me correct with that, very not smart models, but models that are just good at one thing. They're very small. They almost can't do anything else other than be able to say, should I have a smarter agent look at this? And if you manage to bake in that intuition into those small models well, in the sense that they don't miss a lot of stuff and they don't call that other agent too much, then you can get to a really good balance of we're very performant, we have smart agents overseeing things when needed, our costs are low and our latency is low. And then that becomes the challenge because you need to make sure that as the frontier models get smarter and the hardnesses become more evolved, you need to be able to have models that are on your side that are small and effective and continuously being able to say, now is the time.