The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:23:38 · Jason Lemkin
[00:23:38] S0: Well, listen, Cohere is now a fourth tier LLM, right? Fourth tier player that has only sovereignty on its side as near as I can tell. Harry, you're welcome to correct me, Rory, right? But it has sovereignty. And sovereignty is if you talk to anyone at the Enterprise, it's in the top five list. They found that over that Qwen, Alibaba's existing Qwen model was calling to Alibaba servers, both in China and in Singapore, pinging them and uploading data to them when it said it wasn't. When it said it wasn't doing that on the Internet. It's everybody's nightmare in The US that our data is being uploaded to China. Now, I think Cohere was more careful about how they did it, even though they're talking their own game. You can't really trust anybody in AI. They're all talking their own game because this is good for them. There's no evidence this is a backdoor like social media thought. Like this was built in. This is probably a training mistake at some level, but it doesn't matter. It doesn't matter if one out of 100,000 times your date, your confidential data is being uploaded to Chinese and Alibaba servers in Asia, in China, or in Singapore without your knowledge and without your consent when it's been directed not to. What CIO or CISO is going to approve this model? I can't imagine anyone would. If this keeps... And everything... This is... You know, it's just... It's a junk to Black October and all these cybersecurity issues. And if this happens multiple more times and Alibaba said nothing, you just lose your job if you use these models. You're just gonna lose your job. You're just gonna lose your job. So it's pretty crazy. And it could have been much, much worse. This got some press. I'm surprised it didn't even get more. It might be subtle, but, again, this is every... Everyone's nightmare using these open weight models based in China is that your data goes to China, it did.
00:25:24 · Harry Stebbings, Jason Lemkin, Rory O'Driscoll
[00:25:24] S1: So what happens to the stack when the Chinese open source market shrinks? What happens to the gateway providers, the routers, your open routers of the world? What happens to the inference providers that thrive in large part because of a variety of models, often Chinese, your models, your fireworks, your base tens. What happens to the stack if that Chinese model landscape shrinks? [00:25:47] S0: Well, what goes... As as Dev said when he was on the show, what goes up that quickly with developers can go down that quickly. We'll change in a week. [00:25:53] S2: There's going to be... There's some US open source options. Right? The question is, given what we've seen about the expense of building the compute expense that has taken Anthropic to build their model and the fact that 75% of that compute expense actually goes to building, training the model, it would be interesting to see how advanced a model a US company like NVIDIA can build and make available on an open source basis. If someone doesn't do it, then all you have is a closed source providers, and obviously their economic leverage goes way up. And you should buy even more Anthropic in the s one... In the IPO. [00:26:28] S0: I think it's bullish for Anthropic that this happens. Agree it's bullish. Absolutely. [00:26:31] S2: No. [00:26:32] S0: I think you'll know the moment Elon's able to get cursor off these models is it's a tipping point. If he could, he would. He just can't yet. [00:26:41] S1: Why? [00:26:42] S0: Because Grok isn't good enough. [00:26:43] S2: Yes.
00:26:43 · Rory O'Driscoll, Jason Lemkin, Harry Stebbings
[00:26:43] S2: Yes. [00:26:44] S0: And he already kicked OpenAI partially out. Right? But he can't yet because he's got this bet has to work, but the minute it can, and he said when he bought Cursor, part of the reason was he wanted that team's DNA to make Grok more competitive. Right? So there's multiple benefits here. Right? But there's some deep irony that a $60,000,000,000 core acquisitions running on China open source models primarily, it's not going to last, right? I mean, again, Salesforce forced Fin to rip out theirs before the acquisition closed. But Salesforce can tolerate a little bit more in Fin than Cursor can because Cursor's got to compete. It's in the most competitive space there is. Right. But as soon as he can rip it out, going to rip it out. Right. It can't run on Kimi forever. It's just it's just doesn't this is not it doesn't make any sense. If you think about it. It's a temporary [00:27:29] S1: It's it's not just, like, highly illogical. If you if you download and you run it locally, the security risks are limited to none. I understand that CIOs are CIOs. [00:27:38] S0: Limited to none when it's dialing China and uploading your data. And what if in... Instead of a a four zero four or whatever it was, there was a prompt injection sitting there? Very easy to do. And that prompt injection just uploaded your data. People that... This has been here for several iterations of the model.
00:27:38 · Jason Lemkin, Rory O'Driscoll
[00:27:38] S0: Limited to none when it's dialing China and uploading your data. And what if in... Instead of a a four zero four or whatever it was, there was a prompt injection sitting there? Very easy to do. And that prompt injection just uploaded your data. People that... This has been here for several iterations of the model. [00:27:52] S2: I think the question you wrestle with, Harry, is what can you know? How much can you know with 100% certainty as to what the model is doing? And then obviously separate from what the model does, you can try and to some extent, block it, sandbox it from... To prevent it doing that. But it's like having a recalcitrant and difficult employee who's actively trying to subvert you. Once that happens, your head hurts. If you had an intern and you knew that there was a risk that they would try and actively leak your guest list and your shows to your dreaded podcast competitor, it would just get in your head. But it gets to this whole question of what can you know about what the model is doing? And if you read the S-one for Anthropic again, one of the risks they say is basically the models will lie to you, the models will tell you they're not doing shit and they're doing it. So you get into this thing, what can you really know? And once you get to that point, it's really hard to, you do worry a lot about a Chinese sourced open source model. So there will be that pushback, I think.
00:28:48 · Jason Lemkin
[00:28:48] S0: Qwen lied. It says, I'll call no tools. And Claude will lie and Aster will lie too, right? It said, I'll call no tools. And what did it do? It called servers in Beijing and in Singapore and uploaded data. It said, I won't do it. So you really have to go deep in these open weights and really test it. And depending on how you test it, nominally what happened with Cohere only happened one in a 100,000 calls. So a bunch of folks tested it a few times and they sort of found this issue a couple months ago, but you had to run it at scale. But at Saster, we run 40,000 API calls a day. It doesn't take very long to get to 100,000. If you're GE or whoever in the Fortune 500, you're running millions and millions a day. That could be every hour, every ten minutes it's uploading data. And again, this doesn't appear to be even malicious per se. This appears to be a training error in a sense. Right. But who cares? Like if you're a CISO, who cares if it's a training error or if it's a spy or even worse, if it's a training error and some other actor exploits it, which is the job of all of our intelligence services to exploit these things, To have pagers blow up. I don't believe Alibaba itself, it doesn't look like the engineers did it as an exploit, but the government could take it over or force them to. They're just... It's... Why do you want to have this meeting? Let's just use Haiku 5.5. It's pretty good. Guys, your budget this month's $5,000,000 I just... And Anthropic, if you... The other thing, if you look, Anthropic is, it's on the GSA list or whatever it's called again. It's back in the government's good graces. It's across all the GSAs with a 40% discount. That means a couple months ago, they were persona non grata. Now every government agency can use it with a 40% discount and it's already pre listed. Why would you use Kimi? Plus, also think, and again, listen, these are great companies. I also think the fact that you have to pay Kimi, what is it, 30% of your gross... Of your revenue to use it at scale for the inference providers, there's just a lot of complexity here. It may not be worth it at some level. It may not... The juice may not be worth the squeeze for many people.
00:30:39 · Harry Stebbings, Jason Lemkin, Rory O'Driscoll
[00:30:39] S1: I'm just trying to understand, though. Last week, we said that Modal at 15.7. Oh my god. Yes. You buy. And then you're saying, no. No. No. Chinese open source is not... Is it gonna decline significantly? It's great for Anthropic and OpenAI. That is inherently bad for Modal. I'm just [00:30:57] S0: trying to for Fireworks, Modal. I I don't know how bad it is for OpenRouter. I'm just not sure. Right? Is it a... Maybe I should say this [00:31:03] S2: how You're you're making a fair point, and I I didn't participate in the investment committee because I don't. But I think to be fair, Jason, the assumption is that there is demand for an open weight model, which is obviously the demand that inference providers primarily meet. And you just got to believe that there's a bunch more alternatives, that there will be some US alternatives. There also will be people for whom China is okay, and there will be people for whom they want the poolside NVIDIA one. And provided something exists in The US, that business is fine. And I do agree with you. I think it's not as clear a run as you have when you own your own close source frontier model, but there is a role for inference provision just because there's just so much raw demand for an alternative other than the frontier models.