No Priors: Artificial Intelligence | Technology | Startups
00:00:00 · Maxim Bar Kogan, Sarah Guo
[00:00:00] S0: As you're exponentially doing more things with the eyes, you're going to start having really bad actions happen. And we've seen some of that happen lately with agents accidentally publishing code and tokens that they weren't supposed to. Like, definitely, enterprises are starting to realize that risk has grown exponentially and that they don't have any way to stop the adoption. They just now have to do something to reduce the chance of these agent actions being illegitimate or incorrect. But we're allowed to look at a lot of historical data of how these agents have behaved, but enterprises today are not willing to have Anthropic or OpenAI give that historical data because they know these are very data hungry companies that will want to train on that data. [00:00:44] S1: Hi, listeners. Welcome back to KnowPriars. Today, I'm here with Maxim Bar Kogan, the co founder and CEO of Onyx Security, an Israel based startup of researchers, mathematicians, engineers building agents to watch the AI agents. We talk about specialized model training, mythos, alignment research, and the Israeli ecosystem in security and now AI. Welcome. Maxim, thanks so much for doing this. [00:01:08] S0: Thank you. Pleasure to be here.
00:01:08 · Maxim Bar Kogan, Sarah Guo
[00:01:08] S0: Thank you. Pleasure to be here. [00:01:10] S1: Everyone is much more concerned about security and the impact of AI on security than they were certainly a few months ago. The consensus risk story two years ago when you started the company was basically like DLP for chatbots. Like, what are employees putting into ChatGPT? Now we have clearly something that is not quite panic, but close to market wide panic. How did you decide to bet on agent actions when you started? [00:01:40] S0: Look, I think for us, the pivotal point was AutoGPT. I think AutoGPT kind of let everyone's imagination, including ours, run wild because it was a [00:01:50] S1: Can you remind listeners what that was?
00:01:52 · Maxim Bar Kogan
[00:01:52] S0: Sure. So AutoGPT, and I'm sorry if I don't know the guy behind it, but a huge fan, they created the first, as far as I know, the first really autonomous agent running on LLMs, right? So, an agent that would let LLM not generate text, but decide what to do and then give that agent an API access to do that thing, a tool to do it. And then we'd do that in a loop. So, it basically, in theory, could let agents do very complicated things, anything a person could do on a computer. Now, granted it didn't work that well, it was too early, the models were not good enough, GPT-four was not good enough, but I think it did give everyone a glimpse into the future of what if the models were good enough, and then basically using that same structure, we could have very capable agents doing stuff for us. I think that was, in many ways, cloud code today is not dissimilar to AutoGPT back then. I think they were a bit early again, before the models were ready, but the concept was right. And the thought that sticked with me was I was very eye pilled even back then. So, I was thinking, oh my god, models are gonna be way smarter than us. When that happens, how do we oversee these very smart agents that are smarter than us, they're very capable? How are going to feel easy about them doing stuff for us, especially when they start managing really important stuff. One day they're managing your water supply and your electricity, your power grid. Right? How do you control them? And that was like the thing I was kind of obsessed about that thought. I was also too early. So, I think at the time, enterprises were not using any agents. There were hardly any agents out there. And talking with a lot of security buyers at the time, they were like, oh, dude, you're way too early. Like, this is not something that's gonna happen. I asked
00:03:52 · Sarah Guo, Maxim Bar Kogan
[00:03:52] S1: you the same question. I said, is anyone going to do this before you run out of money? [00:03:58] S0: I think there was a good chance that I would've run out of money before because I think you were right. Like, I think there was an element of chance here, but then I think the market did happen. So, we had suddenly reasoning models that could do long horizon tasks. We had Cloud Code, which became like the really first widely used autonomous agent, and then we had Cowork and OpenCLO. And I think we're starting to see now that these types of agents that are very autonomous, even though everyone was afraid to build them, everyone started building these low code platforms that were much more limited, much more based on connectors, those platforms ended up being quite limited, so we didn't get the productivity gains from those limited platforms. But when we started getting the crazy benefits from these very unleashed agents that could do everything, that had much less controls baked into them, and even very large enterprises decided they're going to adopt it. Know, Anthropix revenue is coming from enterprises that are paying for cloud code to do a lot of the work that developers used to do. That was a bit of kind of how we started, and we definitely were in luck that very autonomous agents appeared before it was too late.
00:05:17 · Sarah Guo
[00:05:17] S1: So can you describe a little bit just because it's, I think, both close to impossible and then very useful in this period of AI to think about what is deployment right now and then, you know, what's changing about capability? What's the one liner on what the Onyxx product does today? And then, like, how you think about long term vision?
00:05:36 · Maxim Bar Kogan
[00:05:36] S0: Today, like, Onyxx is really does two things. Number one is we train models and build agents that can oversee other agents. And the goal of that is to say, okay, we need someone to be able to tell that all of these actions that are now happening by these AIs that we're adopting are legitimate because the number of these actions is growing exponentially. And so, things that we thought might be useful in the past, like a human in the loop, now that you're going to have 100x, 1000x, a million x of these actions, that's not going to work. And then we take that capability and we basically productize it in a product that we call the AI control plane or the secure AI control plane, where we come to Enterprise and say, hey, let's find all of your AIs and autonomous agents and hook them up to Onyx to this system where we can oversee what your AIs are doing, so that you don't run into the risk of, as you're exponentially doing more things with AIs, you're going to start having really bad actions happen. And we've seen some of that happen lately with downtimes that were caused by agents doing their own thing, agents accidentally publishing code and tokens that they weren't supposed to, and so on. So, definitely, enterprises are starting to realize that that risk has grown exponentially and that they don't have any way to stop the adoption. So, they just now have to do something to reduce the chance of these agent actions being illegitimate or incorrect.