00:20:42 · Rebecca Balan, Irene Solaiman
[00:20:42] S1: Yeah. [00:20:42] S2: In in AI time, that's a really long time for a model to be out there. There are there are much better open weight models since. So the fact that folks are still hungering for this shows us that sometimes it is a hesitancy to deploy Chinese models because of the geopolitics or or some misunderstanding. I think some of this misunderstanding is the projection that folks are using APIs for Chinese model or models developed in China, and then their data would be sent to mainland. We are very purposeful about none of that data being sent to mainland via inference. The partnerships that you see with Chinese based labs and AWS, for example, show you that there are ways to deploy these models without having your own GPUs but based in The US. Some of the the reasons that folks are using it outside using US and not Chinese based models, developed models could be around this this misperception, avoiding scrutiny, but it could also just be a preference for some of the values if you don't have those kinds of technical depths to be able to to fine tune. There are some differences, and there's some some thoughts on what refusals you get from models developed in China, for example.
00:22:03 · Rebecca Balan, Irene Solaiman
[00:22:03] S1: Yeah. I mean, the the basic one that I know about, and I think this is probably more of a function of their API than than actually the model. But I'm not sure. Maybe you tell me. Like, you know, earlier early on in the whole LLM discussion, you know, everyone was testing DeepSeek... Or not even DeepSeek at this point, probably. It was probably too early for DeepSeek. But they were testing some of the Chinese models, like asking about Tiananmen Square, and then they wouldn't answer. Right? So I'm not sure. Is that kinda what you mean, or is there other other refusals? [00:22:27] S2: Yeah. That's the classic one. And the part of the reason that you've not seen in-depth analyses is that cultural evaluations are so fraught. We've done a few ourselves, and to be able to do it justice would be a PhD level of work. But just from... And this is where we've moved somewhat to vibes versus evals to a combination of both is there... There's there's these kind... I think it's like Claude speak. There's different ways that different models speak. The kind of cultures that you're looking for to be emulated, you can you can fine tune and you can totally adapt and make that your own, but that is relatively intensive. Take some GPUs. Take some talent to make work. Yeah. [00:23:10] S1: Okay. So so GPTOSS, are there other open models from The US that are, you know, being downloaded or or used or really popular on Hugging Face at the moment?
00:23:10 · Rebecca Balan, Irene Solaiman
[00:23:10] S1: Okay. So so GPTOSS, are there other open models from The US that are, you know, being downloaded or or used or really popular on Hugging Face at the moment? [00:23:19] S2: We are seeing so much hunger for not just the Nemotron, but the general family coming out of NVIDIA and OpenWait side. So the Nemotron family is the the more lit... On the language model side. What is generally slept on in the open source community are biomodels and science and robotics. [00:23:41] S1: You guys just came out with a biomodel, Hugging Face. [00:23:44] S2: We did. [00:23:44] S1: Came out with your own. That's really cool. Carbon x. And it's quite a small model. You can run it on one GPU. [00:23:49] S2: There you go. Yeah. And it's... If you do run it partially because of the one GPU, it is just orders of magnitude less costly than comparable models, and that goes a really long way for everyday biologists.
00:23:49 · Irene Solaiman, Rebecca Balan
[00:23:49] S2: There you go. Yeah. And it's... If you do run it partially because of the one GPU, it is just orders of magnitude less costly than comparable models, and that goes a really long way for everyday biologists. [00:24:01] S1: Yeah. I mean, you you brought it to back NVIDIA for me. I promised I was gonna bring it back to NVIDIA, but you did it for me with Nemotron. So I I guess, you know, one of the the things we kinda talked about earlier is, like, who has the resources to back up a lot of the... If you're gonna be at the frontier of open models, like, need money. Like, this is expensive to train these models. Right? And of course, there's... You know, distillation helps, which a lot people give that a dirty word, but it is a very common practice in in AI training. But, like, does America need an NVIDIA like Behemoth to actually solve this and and not only compete with China on open source, but to compete with open AI and Anthropic here in The States?
00:24:40 · Irene Solaiman
[00:24:40] S2: I mean, this is a question of what does the overall ecosystem need. And this is... We are undergoing the regulatory process of joining NVIDIA, which I'm very excited about. And as this is a a thought process that I've had independently, but it's appropriate to give the disclaimer. I think we would thrive most in both open frontier and open local. So this is where you see on Hugging Face, the sweet spots of what people engage with with both likes and with downloads, we look at many metrics, are around like 20 to 40,000,000,000 parameters and with especially quantized or really souped up latest MacBook, DGX Spark, AMD, Halo, you can run these locally or the kind of one to 3,000,000,000,000 what we call open frontier. For an ecosystem, we we need the the smaller open weights that are very permissive, but we probably need the resourcing that something like an NVIDIA has to create a frontier American model that folks can explore, do more evaluations on, deploy locally, be a good alternative to our closed frontier.
00:25:51 · Rebecca Balan
[00:25:51] S1: You know, NVIDIA has been a big proponent of of open models recently. I mean, my personal theory is because, you know, the more people are are using open models, the more GPUs get bought, like, just individually or people need to use neo clouds. And, it's you coming at a time when a lot of the labs are, if not actively doing it, threatening to build their own AI chips. So it's an interesting space. But definitely Jensen Huang, CEO of NVIDIA, his embrace of open source has been, you know, parroted by a lot of other CEOs. We're coming to a point where open models, the use of them is not as, you know, stigmoid anymore. Right? Like, everyone's kind of admitting to it. It's becoming more mainstream. But at the same time, it's still facing threats, right? And so you and I spoke a little bit about the open code attack. So I'm not sure. I hadn't actually heard of this one by the time we talked because our algorithms are simply different. Yeah. But... And I feel bad about this because TechCrunch did cover it. But it was it was around disrupt time, so it was busy for me. But anyway, you know, mid October, OpenCode was the target of a hack. So OpenCode is an AI coding agent as part of the open model ecosystem. This was roughly a forty eight hour agentic attack targeting users' third party API keys. It compromised some accounts. There was some financial damage. So, you know, we've seen attacks now targeting Hugging Face and OpenCode. Granted, they were slightly different in motivation. One was human motivated, the Open Code, versus Agent Swarm motivated. But still, these platforms are, you know, potentially becoming bigger targets. Do you worry that the that security pressures could force more closeness? Closeness, close closeness in the open model ecosystem.