00:23:49 · Irene Solaiman, Rebecca Balan
[00:23:49] S2: There you go. Yeah. And it's... If you do run it partially because of the one GPU, it is just orders of magnitude less costly than comparable models, and that goes a really long way for everyday biologists. [00:24:01] S1: Yeah. I mean, you you brought it to back NVIDIA for me. I promised I was gonna bring it back to NVIDIA, but you did it for me with Nemotron. So I I guess, you know, one of the the things we kinda talked about earlier is, like, who has the resources to back up a lot of the... If you're gonna be at the frontier of open models, like, need money. Like, this is expensive to train these models. Right? And of course, there's... You know, distillation helps, which a lot people give that a dirty word, but it is a very common practice in in AI training. But, like, does America need an NVIDIA like Behemoth to actually solve this and and not only compete with China on open source, but to compete with open AI and Anthropic here in The States?
00:24:40 · Irene Solaiman
[00:24:40] S2: I mean, this is a question of what does the overall ecosystem need. And this is... We are undergoing the regulatory process of joining NVIDIA, which I'm very excited about. And as this is a a thought process that I've had independently, but it's appropriate to give the disclaimer. I think we would thrive most in both open frontier and open local. So this is where you see on Hugging Face, the sweet spots of what people engage with with both likes and with downloads, we look at many metrics, are around like 20 to 40,000,000,000 parameters and with especially quantized or really souped up latest MacBook, DGX Spark, AMD, Halo, you can run these locally or the kind of one to 3,000,000,000,000 what we call open frontier. For an ecosystem, we we need the the smaller open weights that are very permissive, but we probably need the resourcing that something like an NVIDIA has to create a frontier American model that folks can explore, do more evaluations on, deploy locally, be a good alternative to our closed frontier.
00:25:51 · Rebecca Balan
[00:25:51] S1: You know, NVIDIA has been a big proponent of of open models recently. I mean, my personal theory is because, you know, the more people are are using open models, the more GPUs get bought, like, just individually or people need to use neo clouds. And, it's you coming at a time when a lot of the labs are, if not actively doing it, threatening to build their own AI chips. So it's an interesting space. But definitely Jensen Huang, CEO of NVIDIA, his embrace of open source has been, you know, parroted by a lot of other CEOs. We're coming to a point where open models, the use of them is not as, you know, stigmoid anymore. Right? Like, everyone's kind of admitting to it. It's becoming more mainstream. But at the same time, it's still facing threats, right? And so you and I spoke a little bit about the open code attack. So I'm not sure. I hadn't actually heard of this one by the time we talked because our algorithms are simply different. Yeah. But... And I feel bad about this because TechCrunch did cover it. But it was it was around disrupt time, so it was busy for me. But anyway, you know, mid October, OpenCode was the target of a hack. So OpenCode is an AI coding agent as part of the open model ecosystem. This was roughly a forty eight hour agentic attack targeting users' third party API keys. It compromised some accounts. There was some financial damage. So, you know, we've seen attacks now targeting Hugging Face and OpenCode. Granted, they were slightly different in motivation. One was human motivated, the Open Code, versus Agent Swarm motivated. But still, these platforms are, you know, potentially becoming bigger targets. Do you worry that the that security pressures could force more closeness? Closeness, close closeness in the open model ecosystem.
00:27:37 · Irene Solaiman, Rebecca Balan
[00:27:37] S2: This is why we need journalism, Rebecca. We all exist in the algorithmic prisons of our own making, and you are by far not the only person in our field to not have heard of this hack, especially it happened right before the PHP vulnerability exposure and just Black October took another level. The open code attack, we don't yet have public information on, who the attacker was. We have some ideas of the motivation. So it was around forty eight hours of intense agentic attacks that were seeking third party API keys. This is meaningfully different from the Hugging Face incident over the summer that we endured where, kudos to our infrastructure team, were able to recover. And it seems that open open code was able to recover, but we did not have compromises of user data, of financial data, and this is... There was financial damage here. [00:28:31] S1: Yeah.
00:28:31 · Rebecca Balan, Irene Solaiman
[00:28:31] S1: Yeah. [00:28:31] S2: The things that I worry about are what makes an open community targeted is also what makes it special. We aren't necessarily permissive. We want the average hobbyist in their basement who isn't... Doesn't have an institutional affiliation to be able to contribute. I mean, they are part of who made the Internet exist. I mean, this is, like, examples of Wikipedia, just this... How few people are keeping Wikipedia alive. That's the beauty of open contribution. And we also have... So there's a permissiveness and the wealth of information and data we have. So for Hugging Face, we have these really interesting evaluation data sets. OpenCode has these juicy third party API keys. The combination of that makes us prime targets. [00:29:16] S1: Yeah. I mean, it's it's... There's a there's a defense gap there. Right? So where do you draw the line between securing the open ecosystem and undermining the openness that makes it valuable? [00:29:28] S2: This is what we're we're noodling on on what is that trade off. [00:29:32] S1: What what kind of conversations are you guys having at Hugging Face? Like, anything that you can share. Like, what what do you... Like, what's... What are you noodling on? [00:29:38] S2: I mean, the conversations have gotten to a point where a lot of folks are... We we are responding to our community. We are just seeing so much interest in obliterated models. This is... It's can of worms, really dicey topic, and they they do instill fear in a lot of folks.
00:29:38 · Irene Solaiman, Rebecca Balan
[00:29:38] S2: I mean, the conversations have gotten to a point where a lot of folks are... We we are responding to our community. We are just seeing so much interest in obliterated models. This is... It's can of worms, really dicey topic, and they they do instill fear in a lot of folks. [00:29:55] S1: Can you just briefly explain to our listeners what an obliterated model is? [00:29:58] S2: Yeah. It sounds like such a funny word. Obliterated models are models that have had all of their guardrails removed. And already, some of these open frontier models are pretty permissive in their in their refusals and guardrails, but you... It can pretty easily just kinda fine tune that out and take those off and reupload them on Hugging Face. This is not a violation of our terms of service. And part of why we are keeping them up is we have so much feedback from cyber defenders and researchers on how this is their only option for either research or defense.