00:00:00 · Max Justicz, Ruby Justice Thelot
[00:00:00] S0: I think that, like, marginal difference due to my skills is just gonna quickly diminish to the point where, you you know, if if you're completely unskilled, you'll be able to do the same thing. Like, I have a little bit of taste in, like, being able to aim the models at things that I think are worth looking at. But, like, I mean, what kind of mode is that? Like, I I I don't think that's gonna last very long. And pretty soon, just normal computer users will be able to find stuff like this. I'm I'm pretty confident. [00:00:32] S1: And we are back. We are live with Max Justicz. Very nominative determinism. He just found a major root code execution bug in apt, which is the tool that installs software on Debian and Ubuntu, which are two popular Linux distributions. Basically, it's a really important bug and really critical software. So you're able to find this quite easily with AI. Max, welcome to MTS. [00:00:59] S0: Yeah, thanks for having me, and great to meet you, Theo. Yeah, basically, I've been looking for security bugs kind of on the side for a long time. And shortly after I graduated college, I'd I'd spent a lot of time looking at apps specifically because I think package manager bugs are some of [00:01:17] S1: the worst. Right? Because they're they're
00:01:17 · Ruby Justice Thelot, Max Justicz
[00:01:17] S1: the worst. Right? Because they're they're [00:01:19] S0: in this main distribution channel for all the software that we use. So if you compromise the package manager, you compromise everything. Right? And I was really curious how powerful these modern AI tools were at finding bugs like the ones that I had previously found. So I kind of aimed just chat g p t, basically. G p t 5.6 sole at some of the code that I audited years before just to see if it could find anything that I couldn't. And after a couple hours, it came back with this ridiculous bug in apps that affects, like, all Ubuntu and Debian machines, where a network man in the middle can force those machines to install arbitrary software as root. And I came away from this with some realizations, basically. We're at the beginning of a phase when a bunch of really ridiculous bugs are about to to come out. And I think it's only gonna get worse for the next few months. And it really seems like there needs to be a massive effort to aim these really powerful AI models at critical software to to fix them. So maybe nothing new to your listeners, but this was certainly my wake up call. [00:02:22] S1: Well, why do you think there hasn't been more material cyber harm to ordinary people yet? There are things like the Binance cyber attack, which was, pretty huge, but still basically within historical norms for its cyber attack size. The markets don't seem to be expecting, like, major major cyber taps incoming. So what do you think is up with that?
00:02:47 · Max Justicz
[00:02:47] S0: I've been wondering that as well. I have some ideas. For one, it does seem to me like the frontier models are pretty effective at locking down access to some of the most, like, egregious vulnerability hunting requests. I was using Daybreak Blue, which is kind of the easiest to access elevated security permissions model that OpenAI offers. And even since finding this bug, they've really locked it down. Like, the same requests that I was making to find this bug are no longer working. So it does seem like the labs are at least getting a little bit better at locking down these models. I I haven't really been able to get Claude to do anything. It's super useful for me. That said, you know, I I also spun up an obliterated GLM 5.3 model just to see how it fared. And it it it was pretty good. Like, it it it was definitely a bit dumber than the g b d 5.6 models that I I was also playing with, but certainly powerful enough to to be dangerous and find kind of the the more shallow bugs. As for why we haven't seen more harm with this, I think... I wonder if we're rate limited on some of these harms just by the the number of actors actually trying to to to do harmful things. Like, the folks running ransomware campaigns are probably still running ransomware campaigns, and and maybe they're able to find some new targets. But, like, really, what it comes down to is I think that it just hasn't really been long enough yet. I think the tools just aren't in enough people's hands to to have caused major incidents. But I think we'll see them. I think it's kind of just a matter of time. Right.
00:04:37 · Ruby Justice Thelot, Max Justicz
[00:04:37] S1: So how long does it usually take to go from finding this bug to fixing it or bugs bugs like this? [00:04:49] S0: I think people are definitely just the rate limiting step there. I had reported this bug actually back in September. So it took five or six weeks for them to fix it and roll out a patch. But I think during that time, the maintainers were literally on vacation for part of it. So I think a patch could have been written and released later that day. So I I think the patches can come out quickly. It's really just a matter of having people, like, deploy them as urgently as they need to. [00:05:18] S1: So what should people who maintain packages be doing right now in light of all of these cyber incidents and cyber risk for models? [00:05:27] S0: It's very easy to sign up for Daybreak Blue or Anthropic. I think they call it the cyber... What is it? Cyber verification program or something like that. And you should just sign up and and aim it at your code base and and help it... Help you audit your your own code. I think that's kind of just table stakes now. And if you really wanna get fancy with it, I would spin up an obliterated GLM 5.3. And since there's a a startup called, I think, Obliteration dot ai [00:05:57] S1: Yeah. We needed the CEO. [00:05:58] S0: Oh, really? Yeah. I messaged him on LinkedIn the other day just to say thank you for making my life easier. But, yeah, I would recommend throwing $100 their way and aiming it at your code base as well. It's probably gonna be more effective.
00:05:58 · Max Justicz, Ruby Justice Thelot
[00:05:58] S0: Oh, really? Yeah. I messaged him on LinkedIn the other day just to say thank you for making my life easier. But, yeah, I would recommend throwing $100 their way and aiming it at your code base as well. It's probably gonna be more effective. [00:06:11] S1: Do you think that companies like Obliteration and those that offer obliterated versions of very cyber capable models are net good or net bad for the world? [00:06:20] S0: I think in the longer term, net bad to be providing kind of misaligned AI as a service. I think in the short term, it's actually good in that it's going to allow defenders to audit their code more quickly. So I think I think right now, it's... There's also just weird, like, second order effects. Right? Like, I was able to use that service in a way that might be raising some awareness to these issues. And I think that has a significantly good impact right now. So yeah, I think in the long term, the normalizing, providing access to AI models that are willing to do harmful things is is probably not good. [00:07:14] S1: Right. That makes sense.
00:07:14 · Ruby Justice Thelot, S2
[00:07:14] S1: Right. That makes sense. Ad[00:07:17] S2: I wanna shout out our sponsor, Lovable. You know, the app you've been meaning to build? The internal tool, side project, or product you'd otherwise lose a weekend to? With Lovable, you get software you can ship today. That includes an editable code base you can inspect and change, plus two way GitHub sync. Lovable also handles the backend infrastructure, including managed Postgres, auth, storage, hosting, payments, and more so you don't have to wire it all up yourself. And through Lovable's MCP server, you can create and deploy your project directly from the agents you already use. Turn ideas into software people love with Lovable. Lovable. Dev. Now on to the episode. [00:07:56] S1: So is there any way in which this should affect the things that ordinary users do with their computers to stay safe on the Internet?