00:47:12 · Vincent Weisser, John Coogan
[00:47:12] S3: Yeah. So basically, it's like for for all the needs, like, from training to inference, you need obviously compute. In a lot of ways, like, our customers, like, are doing a lot of these, like, large scale training runs with us, like, leveraging our tools, like, including our compute. Right? So it's like pre training, fundamental and parameter models, like, for what we did with the RC, like Q4 last year, or really large also post training runs and RL runs. Right? So it's like increasingly basically, we abstract all of that info away. So basically, people can just, like, hit a button Mhmm. To do a post training run without having to, like, log into GPU cluster and and managing the bare metal GPUs. So we basically abstract a lot of this, like, GPU complexity away for but but yeah. Like, for that, we basically also ramping up our own clusters to basically just, like, serve hundreds and and thousands of customers, like, in parallel on those. [00:48:04] S1: Yeah. So what you're saying is, like, there might be a situation where I have a I have a 50 k budget like you gave with the ramp example. I have a 50 k budget to do some post training, and and I need to deploy that 50 k into, like, thousands of GPUs, but only for a short amount of time. And so Exactly. The budget's actually going to be relatively small even though if I were to buy all of that, it would be it's way more than 50 k, millions, millions of dollars. So you sort of absorb some of that cost, orchestrate it, abstract it away, and then you can deliver for the customer. Just a slice of it.
00:48:04 · John Coogan, Vincent Weisser
[00:48:04] S1: Yeah. So what you're saying is, like, there might be a situation where I have a I have a 50 k budget like you gave with the ramp example. I have a 50 k budget to do some post training, and and I need to deploy that 50 k into, like, thousands of GPUs, but only for a short amount of time. And so Exactly. The budget's actually going to be relatively small even though if I were to buy all of that, it would be it's way more than 50 k, millions, millions of dollars. So you sort of absorb some of that cost, orchestrate it, abstract it away, and then you can deliver for the customer. Just a slice of it. [00:48:38] S3: So so basically, I think there's this, like, also big unlock, like, also on the inference side, right, where customers don't need to, like, front load hundreds of millions and and CapEx spend for large data center commitments. [00:48:49] S1: Yep.
00:48:49 · John Coogan, Vincent Weisser, Jordi Hays
[00:48:49] S1: Yep. [00:48:50] S3: So we basically also increasingly saw a lot of inference. So it it really is the full stack, and I think a lot of the compute like, we we have, like, quite a a clear, like, visibility into the demand just because, like, people come with us with large requests or or large contracts to basically do larger training runs and deployment. So we we basically just, like, need to secure the compute to, yeah, to be able to fulfill all those. And I think, like, right now, we would probably be at, like, many orders of magnitude more compute like compute, but also revenue if compute wouldn't be as constrained as it is right now. So it's like right now, I think, like, in the in the current environment, it's like, basically GPUs are pretty sold out [00:49:29] S1: Mhmm. [00:49:30] S3: For the next few months. And for almost like the rest of the year, especially if you want to like stand up data centers in in The US. So I think this is actually like I think the the the main bottleneck to to scale right now for anyone in AI. And this is I think what we've been able to really scale through extremely well because we've basically partnered with almost every data center out there since, like, over two years. So basically, you're able to send up, like, data centers in in different geographies and orchestrate them globally. [00:49:59] S1: Yeah. [00:50:00] S0: Any predictions, forecasts for American open source over the next six months?
00:50:00 · Jordi Hays, Vincent Weisser, Rodrigo Liang
[00:50:00] S0: Any predictions, forecasts for American open source over the next six months? [00:50:07] S3: Yes. No. Really great question. I we joined also, like, NVIDIA to help them train Nematron. So they're doing the Nematron Alliance and Coalition. So we are one of, like, think 10 or so partners. So it's like so like like, I can't I can't speak about some of that like all the things there, but I think, like, in general, both on that front and other things we're saying [00:50:27] S4: We won't [00:50:28] S0: tell any we won't tell [00:50:29] S3: any. Wait. [00:50:30] S0: Wait. Anything Just between us. Between us. Just the three of So
00:50:35 · Vincent Weisser
[00:50:35] S3: basically, I think like well, like, in some ways, like, leading American efforts actually are Nemotron and actually like Aussie with the Trinity models. So both of those we're like very deeply involved in. And so I think those will continue to to like lead like with like American open source. Like we have like some of our own models cooking. And and then, like, we help a a ton of customers over there. So I think there will actually be, like, a huge resurgence of, like, Frontier open models coming out of The US, also led by players like NBA itself and a lot of our customers and partners. So, yeah, I think, like, in in general, I think the interesting geopolitical question is actually what happens with Chinese open malls. Right? It's like like, obviously, we've seen news that, like, China might consider, like, export restricting them, like and and and, like, there might be policy dimensions on on both the Chinese and The US side. Right? So it's like, I think this is, I think, actually, in some ways, the motivation to build a stack and and to also push American open source forward is to have, like, less reliance on malls that might get banned any day. Be it all more close. Right? It's like I I think this is something increasingly, like, a lot of enterprises are starting to to realize that, like, there's actually huge dependence, right, in the sense on like, mainly from the policy side. So this is something I think where we want to just, enable everyone to have, like, their own end to end FrontiOpenStack to be able to create their own models. And ultimately, think like there there will be like there's obviously a lot of like now, like also Neolabs and and LoftMRI actually like thinking about things quite in a quite open way. So I think it will be surprised that this may be one of the hot takes is to see more Neolabs release open models. Like a good example that we also partnered with recently was a poolside, for example. People were expecting them to be closed, but they actually opened up their models. So I I think there will be more of those, especially given there are, like, hundreds of well funded Neolabs now. Yeah. I think there will be huge resurgence just like of, like, the age of research in terms of, like, these nail labs, like, pursuing truly novel directions. Right?
00:52:40 · Vincent Weisser, Jordi Hays, Rodrigo Liang
It's, like, going where, like, the big labs almost can't go necessarily, taking big, like, bold bets that also might not pay pay off all of them. Right? Or, like, if you have 100 shots on goal, like, some of them will probably pay off and result in interesting outcomes. So I think this is, like, really also, I think, what we are empowering, right, is is kind of, like, helping these new nail labs and and big AI natives and enterprises to move much faster and not have to basically all build the same stack each by themselves in house. Because obviously, a of them are like tiny teams of like tens of people instead of like thousands. So we're able to like help them move much faster basically, But I'm adopting our sec. [00:53:20] S0: When you hear of a, you know, major American tech company, let's say, someone [00:53:29] S4: in [00:53:29] S0: the mag seven that has extra capacity, do you reach out to them? Do you get do you do you do you ping them? Are you at the scale yet that that a deal would be interesting to them? Or are they really searching for these, you know, $10,000,000,000 plus opportunities?