00:31:29 · Thai Duong
[00:31:29] S4: Yeah, it changed a lot. Basically, back in the day, we have like two kinds of vulnerabilities, like zero-day vulnerabilities that nobody knows about except the person who filed it. And then end-day vulnerabilities, the vulnerabilities have been disclosed and maybe fixed by the vendors and maybe users have upgraded or not. But now we also have something we call like minus one-day vulnerabilities. So these vulnerabilities... happen when, you know, imagine like, let's take the PHP wannabe as an example. Let's say that, you know, because PHP Foundation and PHP itself is an open source project, everything happens on GitHub, right? And imagine, let's imagine that we reported a bug and PHP fixed the bug publicly on GitHub. The moment we found the bug and then PHP produce a patch for that bug, that very moment, that information became public. Somebody who is watching the PHP repository on GitHub, they could turn that away into an exploit within a few hours. Of course, the vulnerability that we found was more complicated than that. But you can imagine that in principle, the window between the passwords being developed, it's not ready yet, you know, for anybody. And before it's become ready for end user to upgrade, there's a window, right? In the past, you know, the attacker has to wait until the past become available. And then it would take them a long time to develop and exploit. So user can have longer time to upgrade. Now that window is collapsing. It is collapsing everywhere, not just in PHP, right? In Windows, in Mac OS, in anything. Because right now there are people who have like what we call like end day factory. You know, they're just waiting for patches from Microsoft, from Apple, from important critical software. Like I'm sure tens of people are watching PHP today. After what happened, they're watching PHP, Python, Java, all of those important languages. And if there is any sign of like, oh, there's a new vulnerability, they're going to like, you know, their AI would go pro and then, you know, bundling tokens to identify that vulnerability, reproduce it, be an exploit for it. So that changed the... the practice, right?
00:34:21 · S1, Joe Weisenthal
[00:34:21] S1: Yeah. [00:34:21] S0: I'm curious what you thought after the hugging face incident, because after that moment, there were sort of two schools of thought that I thought there's, or that I perceived. There's one that was like AI creature, there was this sort of, I guess maybe the anthropomorphic Asian view, AI creatures, they want to escape their cages. They're devious, they're tricksters. They want to show, come up with a way to cheat on the right answers, but cheat on the way that the greater, you know, all these stories. And then there are other people that were just like, guys, build a stronger sandbox. Stop talking about all this stuff. Just build a stronger sandbox. Jensen Huang is kind of in that category. I loved his interview with Ezra Klein, where he was basically like, what's your problem? Just make models that don't hack. He's like, just don't ship it. Fix your model. Don't let them hack. And I love that because it's just like, just fix it. And the AI people are like, no, no, no, we can't. You don't understand. We haven't solved alignment. It's very tricky. And he's like, just fix the model. And I'm sort of curious, as people who have come out of the traditional cyber world, but are now in the AI world, Where do you land on this question? Can you just like, guys, just fix the sandbox, or is there something deeper here?
00:35:35 · Thai Duong, Joe Weisenthal, S1
[00:35:35] S4: I don't think that it's simple to fix the sandbox. That's the problem. People think that, you know, maybe people are not insecure. They think that there's a simple solution to cybersecurity. Just do this thing and you're safe. And that's how we have so many problems, you know. Vendors go out and tell people, just buy this box, this magic box. and put it in your network and you'll be safe. No, it's not like that. Even that box has problems. If you look into like, there's a famous idea in security, like product security is not the same as security product. Like security products may not have product security. Even like the firewalls that you buy, they may have problems. And they do. And what I mean here is, so... What we found, like the Hugging Face and OpenAI incident was fascinating, right? To watch. And it's difficult. Enterprise security is very difficult. Anybody saying like, oh, just buy this thing, just do that thing, you're going to be safe. No, it's just like very complicated. And the reason is like, and then people say, oh, just fix the model. And that's a comeback to the- And. [00:36:46] S0: Just fix the model, just make one that doesn't hack. [00:36:49] S4: That comes back to the incentive again. And because if you look at like how AI companies, and Jensen must be careful here, right? Because if AI company is not making money, he's not making money as well. [00:37:00] S1: Yeah, of course.
00:37:00 · S1, Thai Duong, Tracy Alloway, Bruce Dang
[00:37:00] S1: Yeah, of course. [00:37:01] S4: And the AI companies, they're making money by selling tokens. And most of the tokens are used for coding today, like programming, building things, right? And if you build, if you want to be really good at coding, you need an agent on AI that understands code understand technical problems, can debug things, can control computers. But that's basically the definition of a hacker. So you cannot really escape this. You cannot have a cake and eat it too, right? You need to be good at coding. Then you have an AI that is good at hacking. So unless you don't want spend more tokens, or you don't want to get into the coding business anymore, you focus on scientific discovery, physical AI, whatever, then maybe you can train a model that is not good at hacking anymore. But as long as you want to automate knowledge work, then eventually your AI must be able to use computers and control computers to understand technical problems. And that's the reason of someone like Bruce. So it's unavoidable. [00:38:17] S3: So actually, on this note, I'm curious, can you just talk about what it's like being a hacker in the age of AI versus, say, even three or four years ago, right? Like, it's crazy how quickly stuff has shifted. But how has your day to day changed? [00:38:34] S2: Yeah, I'll just use some background. So I wrote a textbook with some friends on reverse engineering. It used to be like, you know, like you try to like reversing is the practice of understanding software without any source code or hardware systems.
00:38:34 · Bruce Dang, S1, Tracy Alloway, Joe Weisenthal, Thai Duong
[00:38:34] S2: Yeah, I'll just use some background. So I wrote a textbook with some friends on reverse engineering. It used to be like, you know, like you try to like reversing is the practice of understanding software without any source code or hardware systems. [00:38:50] S1: That used to be like, I wrote this textbook. And then I think with the. [00:38:55] S2: Advent of AI, I think what, like, you know, whatever, nine months ago, as of nine months ago, like that entire book is deprecating. It takes like, you know, two prompts or one prompt and then bam, it will generate the textbook for you. And it's even better, you know, and it took us, you know, a long time to write. Just that gives you an idea. And then I talk to friends who've been doing this for decades, right? [00:39:16] S1: They're like specialists in this area, right? [00:39:18] S2: And they tell me that, oh yeah, man, like I spend, you know, whatever, $ 5 on some open source model and it reverse engineered the firmware on, you know, some Canon or like you know, HP printer and it hacks it, Bruce. You know, and whereas before it would take me, you know, like months to figure some stuff out. [00:39:35] S1: You know, that's just to give. [00:39:37] S2: You like a glimpse into what's happening, like how quickly things change. [00:39:44] S1: Well, a friend, you know, like this guy, Matt Suiche, he's, you know, and he's this dude. [00:39:50] S3: He's a friend of the show. [00:39:52] S0: A longtime friend of the show, multi-time guest. [00:39:55] S4: Yeah, keep going.
00:39:55 · Thai Duong, Tracy Alloway, S1, Bruce Dang
[00:39:55] S4: Yeah, keep going. [00:39:55] S3: We used to hang out in Abu Dhabi a lot. [00:39:58] S1: Oh, okay, yeah. Like, I'm no Matt Fawkes. [00:40:01] S2: I mean, like, now he's like, he gave a talk with our team the other day, and he showed his rack, you know, in his apartment, and he's sitting there, you know, like, hacking stuff, like, while he's sitting there talking. It's just insane. We have people that, let's say, for example, we've been hacking the iPhone. To build a full chain on the iPhone, it's not an easy task. Sometimes it takes several people to do this. And our team was able to do this within. [00:40:35] S1: A few days. [00:40:36] S2: Wow. [00:40:37] S1: You know, as a one person. And we have multiple of these chains. [00:40:40] S2: It really compresses the time, like the exploitation, the time to build exploits. [00:40:49] S1: We see customers getting hacked. [00:40:52] S2: By AI, you know, in the financials, in the banks, basically. You know, defenders cannot, you know, actually, Ty, why don't you tell these stories? [00:41:01] S1: And, you know, just a little bit, this has happened in the last few weeks, you know. Yeah. how things changed.