00:15:36 · Irene Solaiman, Rebecca Balan
[00:15:36] S2: I mean, yeah, the scarcity is not just to your point, the the availability, but the availability of what. And a lot of... There is a hunger for the latest NVIDIA chip because a lot of the newest models run easiest on them. In terms of what you can get your hands on, there are just a lot more libraries and resources, including on Hugging Face, and generally, this is the beauty of the open source ecosystem that will help you adapt your model to that chip. So we see these previous iterations of GPUs, so like the NVIDIA Blackwell, a lot of folks use for frontier models, and this is... We're gonna get a little bit technical here. It's also a question of the latency for what you're deploying, and this is the level of speed of response. For our incident over the summer, we deployed a NVIDIA quantized, so this is NVFP4, it's a four bit quantized model. It's not going to have that same level of sort of horsepower that a fully high speed kind of, inference will give you, but it'll get the job done, and it'll cost significantly less than what we estimate a... What we estimate the agents' warm cost. [00:16:50] S0: This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed Sponsored Jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed Sponsored Jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.
00:17:20 · Rebecca Balan
[00:17:20] S1: And and for the listener, if you don't know what Irene is referring to with the incident, the incident is obviously when OpenAI's, agent swarm, accidentally kind of hacked into Hugging Face. And Hugging Face was able to defend itself using some some open models rather than the closed models, which I think threw up some guardrails, which is something that we're... We'll we'll actually talk about that again because I wanna talk to you about the open code attack. But... And then another disclaimer, NVIDIA has acquired or is in the process of acquiring Hugging Face as well. So just, you know, note that. Another thing about NVIDIA, and I'm gonna come to it in a roundabout way, I wanna talk a little bit about China and competition for open model dominance. You know, the best open models, you've listed several of them. They're all Chinese at this point. What happens if Chinese models become the default for AI startups around the world? And actually, not just AI startups, like sovereign AI for governments. And, you know, we're seeing Rio De Janeiro in Japan's sovereign models are built on Qwen. So what is that... What happens? Is that a problem?
00:18:24 · Irene Solaiman
[00:18:24] S2: It's a... It's always a dance when we're talking about open models on... With the geopolitics. The geopolitics have become much more central even the last few months. It's a question of problem for whom. So people love open models no matter where they come from. I've had to scale up very quickly with our head of APAC. It's really just me and Adina working with Open Frontier Labs, these Chinese labs who are so talented, who work really hard, and, it's incredible to feel the ethos of belief in open weights. It would be even more exceptional to have a thriving global ecosystem and to have Open Frontier US models. We are working closely with with the Muse team, with Reflection, with RC. It's really hard to train an open frontier model, but there's a reason that governments are preferring models developed in The US, for example. There's a reason that GPTOSS still has a large amount of downloads on Hugging Face. There is a hunger for US models. How do we stoke that, I think, is a big a big question around the narrative. There's been a demonization of open source of them being... These models being more dangerous, which if you look at... Thanks for the the explanations and disclaimers, Rebecca, the last few months of cyber attacks, the most prominent ones have been frontier models, have been close weight models. And we're just balancing with what do we need to make sure that folks get access and have the tools in hand. And a lot of that... A lot of the time, it's open weight.
00:19:58 · Rebecca Balan, Irene Solaiman
[00:19:58] S1: You said that there's, like, know, there's a hunger for GPTOSS. There's a hunger for American open models. What... So... And a lot of people, a lot of companies, governments want to build on American models rather than Chinese. What what is the main reason? Is it just, like, the geopolitics of it all? [00:20:13] S2: So for... Also for context, g two s s was released by OpenAI in around Yes. August 2025. That's that's my bad for not to wait. [00:20:22] S1: Sometimes I'm like, we're all nerds here. Right? We all we all know what we're talking about. Right? [00:20:27] S2: But there have just been so many that I'd measure time and latest frontier generation. [00:20:31] S1: Believe that was 2025. No. [00:20:34] S2: August 2025 was due [00:20:36] S1: to open AI. When's the next open model? Come [00:20:39] S2: on now. We're also, you know, itching over here. [00:20:42] S1: Yeah.
00:20:42 · Rebecca Balan, Irene Solaiman
[00:20:42] S1: Yeah. [00:20:42] S2: In in AI time, that's a really long time for a model to be out there. There are there are much better open weight models since. So the fact that folks are still hungering for this shows us that sometimes it is a hesitancy to deploy Chinese models because of the geopolitics or or some misunderstanding. I think some of this misunderstanding is the projection that folks are using APIs for Chinese model or models developed in China, and then their data would be sent to mainland. We are very purposeful about none of that data being sent to mainland via inference. The partnerships that you see with Chinese based labs and AWS, for example, show you that there are ways to deploy these models without having your own GPUs but based in The US. Some of the the reasons that folks are using it outside using US and not Chinese based models, developed models could be around this this misperception, avoiding scrutiny, but it could also just be a preference for some of the values if you don't have those kinds of technical depths to be able to to fine tune. There are some differences, and there's some some thoughts on what refusals you get from models developed in China, for example.
00:22:03 · Rebecca Balan, Irene Solaiman
[00:22:03] S1: Yeah. I mean, the the basic one that I know about, and I think this is probably more of a function of their API than than actually the model. But I'm not sure. Maybe you tell me. Like, you know, earlier early on in the whole LLM discussion, you know, everyone was testing DeepSeek... Or not even DeepSeek at this point, probably. It was probably too early for DeepSeek. But they were testing some of the Chinese models, like asking about Tiananmen Square, and then they wouldn't answer. Right? So I'm not sure. Is that kinda what you mean, or is there other other refusals? [00:22:27] S2: Yeah. That's the classic one. And the part of the reason that you've not seen in-depth analyses is that cultural evaluations are so fraught. We've done a few ourselves, and to be able to do it justice would be a PhD level of work. But just from... And this is where we've moved somewhat to vibes versus evals to a combination of both is there... There's there's these kind... I think it's like Claude speak. There's different ways that different models speak. The kind of cultures that you're looking for to be emulated, you can you can fine tune and you can totally adapt and make that your own, but that is relatively intensive. Take some GPUs. Take some talent to make work. Yeah. [00:23:10] S1: Okay. So so GPTOSS, are there other open models from The US that are, you know, being downloaded or or used or really popular on Hugging Face at the moment?