00:28:31 · Rebecca Balan, Irene Solaiman
[00:28:31] S1: Yeah. [00:28:31] S2: The things that I worry about are what makes an open community targeted is also what makes it special. We aren't necessarily permissive. We want the average hobbyist in their basement who isn't... Doesn't have an institutional affiliation to be able to contribute. I mean, they are part of who made the Internet exist. I mean, this is, like, examples of Wikipedia, just this... How few people are keeping Wikipedia alive. That's the beauty of open contribution. And we also have... So there's a permissiveness and the wealth of information and data we have. So for Hugging Face, we have these really interesting evaluation data sets. OpenCode has these juicy third party API keys. The combination of that makes us prime targets. [00:29:16] S1: Yeah. I mean, it's it's... There's a there's a defense gap there. Right? So where do you draw the line between securing the open ecosystem and undermining the openness that makes it valuable? [00:29:28] S2: This is what we're we're noodling on on what is that trade off. [00:29:32] S1: What what kind of conversations are you guys having at Hugging Face? Like, anything that you can share. Like, what what do you... Like, what's... What are you noodling on? [00:29:38] S2: I mean, the conversations have gotten to a point where a lot of folks are... We we are responding to our community. We are just seeing so much interest in obliterated models. This is... It's can of worms, really dicey topic, and they they do instill fear in a lot of folks.
00:29:38 · Irene Solaiman, Rebecca Balan
[00:29:38] S2: I mean, the conversations have gotten to a point where a lot of folks are... We we are responding to our community. We are just seeing so much interest in obliterated models. This is... It's can of worms, really dicey topic, and they they do instill fear in a lot of folks. [00:29:55] S1: Can you just briefly explain to our listeners what an obliterated model is? [00:29:58] S2: Yeah. It sounds like such a funny word. Obliterated models are models that have had all of their guardrails removed. And already, some of these open frontier models are pretty permissive in their in their refusals and guardrails, but you... It can pretty easily just kinda fine tune that out and take those off and reupload them on Hugging Face. This is not a violation of our terms of service. And part of why we are keeping them up is we have so much feedback from cyber defenders and researchers on how this is their only option for either research or defense.
00:30:33 · Rebecca Balan
[00:30:33] S1: Yes. Yeah. I've I've also come around on obliterated models a little bit after talking to cyber people because, you know, I wrote about this. There's a company called Obliteration dot ai, I think it's what they're called. And and they basically just like host or will provide you with the ability to obliterate any model. So you just... There's no guardrails at all. And you can just imagine how horrible this is. Right? Like, you don't wanna make it easier for attackers. But at the same time, this makes it easier for defenders. And I guess, the argument is that the attackers are gonna obliterate... They're gonna do this anyway. It's important that the defenders have... Know how they're going to attack them, right, essentially, and are able to... It's really good for red teaming as well. You know, really helpful for, you know, when you're trying to do the most insane version of a hack that you're trying to test your systems on to have an obliterated model. Yeah, it's tough. But then if open model infrastructure becomes, you know, more costly and risky to maintain, who who, I wonder, would pay for the security? Like, would that potentially give large, well funded platforms an advantage over the smaller open source projects?
00:31:40 · Irene Solaiman, Rebecca Balan
[00:31:40] S2: I mean, there there's going to be just an advantage in the level of product support that you get from a closed weight lab. Most of these outside of the the really well resourced, like the Alibaba's, you know, we're still 200 people while we're undergoing this acquisition. So we're not gonna be able to provide that level of handholding. But for what folks need for for cyber defense at a time where there's a lot of... I'm I'm quite nervous about the state of the Internet. We we need to have some level of... This is a trade off. We need to have access to the folks who know what they're doing, who are getting refusals from the most frontier models. There's a dichotomy, unfortunately, between cybersecurity and AI safety folks, and both have well and good intentions. But a lot of security folks love open source because of the privacy, because of those on prem considerations, because of what they can do with it, and really have this ideological preference for open weeks in a way that is creating a new kind of open versus closed conversation. But the urgency from our perspective and from where I'm sitting at Hugging Face is that we just need to quit Defenders yesterday. And if that means giving folks a more permissive model, it seems that's the balance that we'll have to make. [00:33:05] S1: Is there anything that could convince you otherwise? Like, is there any type of attack that might happen that would make you think, actually, like, I don't want peep... Like, attackers to have access to, you know, powerful obliterated open models.
00:33:05 · Rebecca Balan, Irene Solaiman
[00:33:05] S1: Is there anything that could convince you otherwise? Like, is there any type of attack that might happen that would make you think, actually, like, I don't want peep... Like, attackers to have access to, you know, powerful obliterated open models. [00:33:20] S2: I mean, this is the visibility of who's doing the attacks and who's doing the defense. I wish that we had better ways. This is what we've asked for policy wise is better incident reporting and standards. We're so glad that we were able to be as explicit as possible, but it was quite difficult to figure out how to report it and to whom. So as folks both on the the frontier side but also who are doing defense, who are victims of attacks, are undergoing this, I would ask that they be explicit as possible in sharing openly what they're undergoing, if they have an understanding of what model was being used. To your explicit question of what would change my mind, I think it is a clear clear data that the risks of open weights being leveraged in attacks outweighs the possibility for defense. And a lot of this is compute constrained. I don't think there is enough compute available actor to do the level of large scale at this point, which could change in the near future. But in our level of urgency, the need for defense, for even if you've seen these European grocery stores, it might not be the highest priority for critical infrastructure, but for businesses to defend hundreds of thousands, if not millions, in damages seems important.
00:33:20 · Irene Solaiman, Rebecca Balan
[00:33:20] S2: I mean, this is the visibility of who's doing the attacks and who's doing the defense. I wish that we had better ways. This is what we've asked for policy wise is better incident reporting and standards. We're so glad that we were able to be as explicit as possible, but it was quite difficult to figure out how to report it and to whom. So as folks both on the the frontier side but also who are doing defense, who are victims of attacks, are undergoing this, I would ask that they be explicit as possible in sharing openly what they're undergoing, if they have an understanding of what model was being used. To your explicit question of what would change my mind, I think it is a clear clear data that the risks of open weights being leveraged in attacks outweighs the possibility for defense. And a lot of this is compute constrained. I don't think there is enough compute available actor to do the level of large scale at this point, which could change in the near future. But in our level of urgency, the need for defense, for even if you've seen these European grocery stores, it might not be the highest priority for critical infrastructure, but for businesses to defend hundreds of thousands, if not millions, in damages seems important. [00:34:39] S1: Would you support some kind of a a policy that restricted... I mean, if not restricting access to models, restricting access to compute? Compute?