01:29:33 · Rodrigo Liang, John Coogan, Jordi Hays, Vincent Weisser
[01:29:33] S4: I'm doing good. How are you guys? [01:29:35] S1: We're doing fantastically, but it sounds like you're doing better. Give us the news. What happened? [01:29:39] S4: You know, we're super excited. We did a big big funding announcement today with a billion dollar round that we did a first close on at at an 11,000,000,000 valuation. So super excited about announcing that. [01:29:53] S1: Incredible. There we go. There [01:29:58] S4: we go. There we go. [01:30:00] S0: This reminded me there was somebody posted, like, a couple weeks ago something to the effect of, like, every day there's a new billion dollar deal in chips from some company you've never heard of. Market. I guess the trend the trend continues. Yeah. Talk about [01:30:16] S1: it. Yeah. Yeah. Is this an overnight success? When did you start the company? How long has this been overdue? [01:30:22] S3: It's an
01:30:22 · Vincent Weisser, Rodrigo Liang
[01:30:22] S3: It's an [01:30:23] S4: overnight nine year success. Know, I've been I've building chips for thirty years, the high performance product, and we started this company in 2017. So no, but it's been a great journey. It's just, again, so noisy. So there's so many companies out there doing all sorts of awesome things, but look, Inference is now right in front of us. It's kind of broken everything open. It's what we do, and it's been what we've done for a long time. The new training run, and so people that are kind of been thinking about building just for training, well, now it's the time for inference. This is where we shine, and we shine, and investors are putting all their money behind that, and so we're excited about that.
01:31:05 · John Coogan
[01:31:05] S1: How how walk me through the the phases of I mean, you have some of the biggest companies in the world you're working with. JPMorgan Chase selected you for secure on prem AI inference. And I'm interested in the thought process that large Fortune 500 companies, Fortune 100 companies are going through right now. I imagine that there's a lot of explore with frontier models, very expensive. They're token maxing. Then they shift to, okay, this agentic workflow is working. Let's use an open source model for it that is, you know, either the open source frontier is caught up and can do this particular task. Maybe there's a fine tune or some RL on some private data that makes the the the system perform better. When is the conversation shifting to actually actually optimizing the inference stack, the hardware? What are what what is what's the conversation like there? What are the savings that people are focused on?
01:32:05 · Rodrigo Liang
[01:32:05] S4: Well, I think what people are starting to realize after they started using ChatGPP and Yeah. Drop the the frontier models is, there's a bunch of things I can do there, and it's great. You know, we'll use it, and it's going to be part of their business for a long, long time. Mhmm. Right? And hyperscale is offering all sorts of different services with different GPUs and GPUs and training chips. They're offering different things there, again, on the cloud. What's happening now is this next phase of production is people starting to realize, okay, what happens to the stuff that I'm not comfortable releasing to these models? Like my private data, my security, the stuff that I want your models to learn and then somehow become part of the global knowledge, right? And so what happens with that? Right? And so I don't know if I have, you know, you know, my clients' banking information, you know, I have, you know, some some very regulated information. What do I do with that? Right? And so people are starting to realize, okay, there is a whole another class of workloads. Whole another class of applications that you've got to go and do private. With someone like JPMorgan, which is fantastic, they are the top of the top when it comes to enterprise and IT and technology, fantastic partners. They selected us as the ones that can come in and go into an environment like JPMorgan, do full on prem secure private. They can bring their open source models. They can fine tune into it. They can deploy that privately security and control the models and control it forever. So I think we're starting to see the landscape starting to steady, right? There's some things you use the Anthropix for, there's some things that you're going to use the hyperscale clouds for, and there's some things you're going to say, Hey, let me run on prem and let me control the outcome. Especially since I don't know how the regulatory environment is going to settle, let me just be secure for that. Let me just be safe and just run those things on my own models in my own environment with my own security around it and then be done.
01:33:54 · Jordi Hays, John Coogan
[01:33:54] S0: Yeah. You can think, you know, pharma company being like, would love for all the other large pharma companies to share their data Yeah. With with the Frontier Lab. Totally. But I'll I'll pass. [01:34:05] S1: Mhmm. Does what does partnership with you look like for a large organization? If I have, you know, a large IT footprint already, I probably have data centers, but they might be more CPU based workloads. Maybe I have some GPUs, but I'm not doing, frontier inference. Not fully scaled up, but I have some are are you consulting with them at the level of, like, where they should be buying land for their next data center? How they set up a powered shell? Who they should partner with? Like, How deep in the stack will you go with a company that you're working with?
01:34:43 · Rodrigo Liang
[01:34:43] S4: Well, look, if you look at the AI ecosystems up to date, we've been really focused on hyperscale clouds, putting out the neo clouds. The frontier models, we're really focused on building those out, you know, gigawatts here, megawatts there. I mean, you know, just this large scale deployment. And then what you're still thinking about is, well, how do I get as many tokens out per, you know, megawatt per kilo you know, kilowatt, megawatt, and gigawatt that I have. Right? And so that's kind of we're very focused on that. Some of them will play into that are really, really efficient models. We run the big models really fast. That's really what we're known for. Mhmm. Huge models. Right? You know, the battleground for power and data center and all that is about huge models. It's not about the little models, it's about the trillion, multi trillion per model models. That's what we focus on and we run that faster than anybody else in the world. But then you flip it over to the enterprise, and for them, it's not as much about how many thousands of racks are going to fit into an environment. It's about data privacy. It's about security. It's about having the capacity when I want it. It's about low latency. It's about these things that, again, you go back to traditional computing. It's what they've always wanted. They always wanted low latency or high performance, all security They and always wanted that. What we can do is go on over is that because our our rack is a 10 kilowatt air cooled rack compared to what traditional GPs are 140 kilowatts per rack. With 10 kilowatt air cooled rack producing state of the art AI.