Unsupervised Learning with Jacob Effron
00:10:15 · Jacob Effron
[00:10:15] S1: Makes total sense. I guess I'd love to take our listeners, like, inside Nebius. I feel like there's so many interesting parts of the business. But maybe one place to start would just be, I imagine as you're, like, getting, you know, new NVIDIA chips that come in, like, there must be a bunch of stuff you have to do within the business to to get this all all up and running. So, like, what does that look like from, you know... I don't know. You're talking to NVIDIA folks, and they're, you know, giving you early insight into into the next generation of chips to, like, you know, actually getting these live for a set of customers?
00:10:43 · Danila Shtan
[00:10:43] S0: That is a mess. No. I mean, like, in a good way. You you know? Like, this whole this whole market, this whole field is running at such a pace that everyone is trying to catch up. Right? And NVIDIA, despite being in the lead of this, essentially leading and showing the direction for this market, is not the exemption. So first, NVIDIA folks, like, way before show you the prototypes of what they want to build in the next, like, year. Right? They kind of show you the schematics, like how it will be, like, optimized internally, where is your PCI connections, how would that look like, what is the future platform would look like. Then sometime later, they come to you and said, oh, like, we've optimized things. It's now gonna be different. And you're like, all right. Then they come to you and like, oh, we have, like, the first lab samples. Do we want to have a look? Yes, of course, we do want to have a look. Oh, by the way, it doesn't look like the stuff you've showed us two months ago. Oh, we've changed stuff because, like, we found, like... Right. Because they are so compressed in time, same as they... Everyone else. They are constantly iterating while these chips are being produced. Right? So they've... It's not your, like... You know, there is, like, an ocean in the consumer market. Like, Apple iPhone that you see today was actually planned three years ago, and final design more or less was settled, like, a year ago. And they take the final year to polish it to build this product. Like, AI market is running at such a pace, like, that whatever, like, they announce is it's indicative. And then teams are constantly
00:12:27 · Jacob Effron
[00:12:27] S1: Well, the architecture and the workflows are changing. Right? So I'm sure they're iterating as as that evolves.
00:12:31 · Danila Shtan
[00:12:31] S0: Like, they're iterating, like, going further and further and further. NVIDIA is a behemoth now. Right? Whatever they do influences the whole market, supply chains, just everything. So they have to, like, live with that. So at some point, you get a sample, and it's more or less like what you'll get in production. Then you start building. Like you're testing everything. You like incorporate support for the new hardware, for the new restrictions or capabilities into the software platform. You figure out a way to isolate workloads from from the chat. We are we are a cloud. We are not providing bare metal hosts. Right? Like, so we need to split. We need to make sure that our virtualization works, that our storage works, that our networking works. Sometimes we are on the bleeding edge of this. So we, like... For some of the generations, we were probably the first team... Like, maybe one of the first teams that actually built, like, a cluster of a certain size. So we've hit some issues with, like, with the software that no one has seen before. So we had to report that back to NVIDIA, like, propose a patch, get there, like fix from it, make sure that it runs, etcetera, etcetera, etcetera. So like, we kind of feel like we are at the forefront of it. But all in all, it's a pretty normalized process. It may sound messy. Right? There are like multiple iterations. You go back and forth. You're in constant contact with like NVIDIA hardware folks, with NVIDIA software folks. You're trying to solve the issues on your end at the same time, incorporating into a more complex platform, thinking how can you expose that to customers. Because, for example, working with, like, your usual black hole chip, like, you know, the b three hundreds, and then transferring that experience to GB three hundreds, which I recall different architecture in terms of isolation. You are talking racks rather than servers. So you need to find a way how to do maintenance on the rack instead of the server, which you just switch off and you move the workloads to another one. Despite all of that, this kind of works all the time. And every day, you like push it a little bit further. And then in the end, yes, there is a product and the customers use it. And that's it.
00:14:55 · Jacob Effron, S2
[00:14:55] S1: One thing I think it would be so interesting to get your thoughts on is just the the differences in, you know, kind of what end customers end up wanting. Like, obviously, it can be easy to think, okay, everyone just wants to, you know, come to you and and get similar things. But I imagine you segment your customers or think about differences in what is important to different customers. How do you think about the key differences in these customer desires today? [00:15:18] S2: Hi. I'm Anne Maris, and I'm the producer of Unsupervised Learning. Thank you so much for listening to the show. And if you've enjoyed the conversation, please subscribe. Subscribing helps us grow, which helps us continue to lock down some amazing guests. Thank you so much for your listenership, and see you next episode. [00:15:35] S1: How do you think about, like, the the key differences in in in these customer desires today?
00:15:35 · Jacob Effron, Danila Shtan
[00:15:35] S1: How do you think about, like, the the key differences in in in these customer desires today? [00:15:42] S0: It's not that different, to be honest. Because, like, every customer is, like, its own unique flower. Right? But at the same time, workloads in principle are not that different. It's either training, and you kinda know how training looks like, or inference, and you kinda know how inference looks like. Circling back a couple of years ago, for example, we kinda underestimated the storage component. We thought GPUs are the most important thing. And they are. But alongside the whole training angle, you see a massive demand for storage. And it's not for the, like, amount of, AGDs or VMUs that you can provide to the customer. It's about throughput. It's about latency. In my prior life, pre AI workloads and infrastructure build outs, I was talking about gigabytes per second, if we were talking about network attached storage. Right? It was like... It's really... Right now, we have customers routinely kinda like hitting our... Like a common share, like one volume, right, connected to a number of GPU servers with like, I don't know, five, six, 700 gigabytes per second of throughput. This is extreme. Like two years ago, I didn't even think about that. And now this is more or less like something we expect. At the same time, if you are targeting these numbers for the highest demand... Like, the the most demanding customers, your baseline is so high, you kinda can serve anyone. [00:17:20] S1: Yeah.