00:04:58 · Tomer Schwartz, Rebecca Balan
[00:04:58] S2: Yeah. So a lot of different people participated in this research, and I gotta say that all of them are absolutely brilliant for finding this. It's kind of cool to see how different companies and different groups of researchers actually find similar things in parallel. My understanding of the timeline is that Calif actually found the first CVE, reported it to OpenAI. They realized they found some very similar variants of the same bug. They reported it to PHP together. And at the same time, in parallel, we had no idea about what's happening on the other side. We were looking into PHP at the same time we reported both CVEs to the PHP team. And it was the PHP team that decided to eventually say, hey, You know, this is such a critical finding. It's gonna take us so long to actually figure out what's the best way to to fix it. Why wouldn't all of us just join the same Slack channel and start working on this together? I think it was absolutely the right choice. We managed to do amazing work together. Honestly, I was fortunate to be on literally daily calls with the PHP team, with OpenAI and Calif just working together towards the patch. [00:06:07] S1: Yeah, must have been high adrenaline times for you guys. [00:06:10] S2: It was. I just want [00:06:11] S1: to double click on something you said, that a lot of different organisations found the bug at the same time. And you're saying this is because everyone's kind of using the same AI tools to do this, to search for vulnerabilities?
00:06:11 · Rebecca Balan, Tomer Schwartz
[00:06:11] S1: to double click on something you said, that a lot of different organisations found the bug at the same time. And you're saying this is because everyone's kind of using the same AI tools to do this, to search for vulnerabilities? [00:06:25] S2: Well, can't speak for everyone, right? But what I can say is that at Wiz, what we're using is a sort of custom harness that we've built. We call this Project Atlas, which builds on top of all the frontier AI models. It leverages both the the DeepMind capability, the the OpenAI capabilities, the the Anthropic capabilities. And in reality, what it what it allows us is to scale a lot of different ideas and try different potential attack paths at the same time. So it gives us a really broad scope to be able to pull up a lot of research ideas and a lot of potential issues. And from there, what we're trying to do is to distill it down to the most impactful ones and work on them. At the same time, I do I do know a little bit from our work with OpenAI and Calif. They're doing very similar things and trying to use their own models and and and new ideas to scale up research with AI, and they're finding, in many cases, very similar bugs to to the bugs that we're seeing.
00:06:25 · Tomer Schwartz, Rebecca Balan
[00:06:25] S2: Well, can't speak for everyone, right? But what I can say is that at Wiz, what we're using is a sort of custom harness that we've built. We call this Project Atlas, which builds on top of all the frontier AI models. It leverages both the the DeepMind capability, the the OpenAI capabilities, the the Anthropic capabilities. And in reality, what it what it allows us is to scale a lot of different ideas and try different potential attack paths at the same time. So it gives us a really broad scope to be able to pull up a lot of research ideas and a lot of potential issues. And from there, what we're trying to do is to distill it down to the most impactful ones and work on them. At the same time, I do I do know a little bit from our work with OpenAI and Calif. They're doing very similar things and trying to use their own models and and and new ideas to scale up research with AI, and they're finding, in many cases, very similar bugs to to the bugs that we're seeing. [00:07:24] S1: And how much of the the bug finding is really... Is it really AI versus, you know, human work that's supercharged? Right? Like, Calif found the bugs in a security audit says... You know, this is from our our security editor, Zach. He said that AI helped with the exploitation, but there was, you know, it was 90% human work. Calif's own line on this was that no model is simply capable of unearthing all bugs, at least for now.
00:07:49 · Tomer Schwartz
[00:07:49] S2: Well, I sort of agree. There's still an art and a science to this in how do you direct all of these models and how do you actually tailor them together to to be able to find such impactful bugs. But in reality, I didn't think that we could have found it without AI. So the the opportunity for a team of 10 of the most talented people that I know that are powering the the Wiz vulnerability research team right now, if we had taken all of them and told them, hey, maybe maybe find some some PHP RCE bugs, Maybe it it would have taken them a couple months of work, maybe even twelve months of work. And eventually, who knows? Maybe maybe they would have found the same the same finding. But what's cool about this is that we managed to take all of those people and tell them, hey, why wouldn't you try 20 different ideas on various different targets and just run them on top of Atlas and find a ton of different potential vulnerabilities and maybe do it over the weekend by the time we come back next week to see everything, we had a model working for us, trying a ton of different things that maybe a human researcher would not have found up. Right? So it's kind of cool to think that right now we can get 100x, 1000x the work of a researcher, and they can actually pick and choose what are the best what are the best research ideas and what are the best solutions and where do we need to pull this thread a little more. And in in many cases, with the capability of AI models today, it might be a full exploit already available where we just need to validate and we need to understand what the vulnerability is.
00:09:23 · Rebecca Balan, Tomer Schwartz
[00:09:23] S1: Yeah. I mean, it's great for defenders. It's also great for attackers. Right? And that's why we saw such a push to get this bug fixed because attackers are using AI to send out mass attacks and to 100x their own capabilities. Can you tell me a little bit more about Atlas? Atlas is your harness. Using, you said, OpenAI, Anthropic models. Are you using latest site? Are you using Mythos in that? Do you have access to that yet? Which models are you using and playing with? [00:09:51] S2: We constantly change it. With the pace in which AI models get better and better, we've to update this all the time, and there's also various different steps where we find that certain models are better at as opposed to others. [00:10:06] S1: Can you give me an example?
00:10:06 · Rebecca Balan, Tomer Schwartz
[00:10:06] S1: Can you give me an example? [00:10:08] S2: So let's talk about binary exploitation versus code comprehension. Right? These are very different tasks. If I take a very large code base and I'm looking for, hey, what kind of what kind of interactions do do we see here between components and what do they assume about how other components are using them? Maybe maybe there's one model that's really good about processing large chunks of data and compressing it into a whole bunch of insights that we can use later. But then if the the only thing we need is to to get to a certain part of the code base and figure out how to trigger this remotely, this is a completely different task. So we are trying to evaluate different models all the time. We we actually have a publicly facing benchmark that we we release. Fortunately, we also have access to unreleased models as well with the the labs that we're working with. So what we're trying to do is to just sort of do a best of breed kind of combination and see what's what's the best solution for this problem. [00:11:05] S1: How much of the mix is open models? Because I know a lot of people I've spoken to, including, you know, Clint along from Hugging Face, say that open models are really great cybersecurity because you don't run into any of the guardrails when you're red teaming or searching for vulnerabilities that you might with some of the frontier models. Is that what you're seeing as well?