Unsupervised Learning with Jacob Effron
00:06:57 · Jacob Effron, Danila Shtan
[00:06:57] S1: You know, people have come to these AI cloud businesses in different ways. And, obviously, you know, I felt like the index team was, like, legendarily really, really strong, you know, on on the engineering side. You know, a lot of folks building AI clouds came from, like, crypto or energy or, like, not not necessarily the same, you know, maybe technical heritage. To what extent, I guess, was the Yandex experience really relevant? And to what extent was it like... You know, it turns out no one knows how to build a GPU cloud because it's all kinda new and, you know, there's there's... Everyone's learning on the fly. [00:07:27] S0: First, we had and still have the core team that supports the hardware side of business that came from Yandex that had experience operating hardware, like, infrastructure at scale, like, real scale. Hundreds of thousands, millions of servers, span over multiple facilities, etcetera, etcetera, etcetera. These people, like, were at the core of our build outs, like initial build outs and expansion, and they're still with the with the company. So this definitely helped. I think, like, we would be way less successful if not for that core team. [00:08:04] S1: Yeah.
00:08:05 · Danila Shtan
[00:08:05] S0: On the software side, it's interesting because it's a two fold. Like, because like with hardware, it's very straightforward. We needed people with experience. We had people with experience. They started building out from the get go. On the software side, it's... It was a weird one. First of all, we had to rewrite everything from scratch. So our previous experience was kinda like garbled. Second, we had to grow so fast that by, for example, by the end of twenty twenty four, when the new platform launched, like a significant portion of our engineering team, software engineering team, was not from Yandex anymore. Because we had to hire during this period extensively. As a team, we were working on this cloud platform from, like, from the initial idea. Right? So we kinda just went with it and doubled down on software. Like, we are not doing a managed bare metal. We are not doing a patchwork of third party services. We want to build a holistic cloud experience, the holistic cloud platform. And we kind of doubled down on that even with the whole rewriting of platform, like doing it from scratch, etcetera, etcetera, etcetera. And of course, like having a ton of talented engineers on the team obviously helped. Like, just for the scale of it. Right? You've mentioned, like, people who, like, came from crypto or came from other avenues of life. They are mostly concentrated on how to... Like, their idea is like, you need to buy GPUs, you need them to run, and then you'll find the buyer. In our case, for example, in 2024, when our hardware team that actually, like, procured GPUs, did the build outs, we're working on our own servers and chassis, writing firmware and stuff, building the network. That whole team was roughly, like, 40 to 50 people. Yeah. The software organization was already 200. Right? So we kind of concern... Like, think of ourselves as a software first company. We are building the cloud on the solid foundation of hardware. Yeah. We're not a hardware first company.
00:10:15 · Jacob Effron
[00:10:15] S1: Makes total sense. I guess I'd love to take our listeners, like, inside Nebius. I feel like there's so many interesting parts of the business. But maybe one place to start would just be, I imagine as you're, like, getting, you know, new NVIDIA chips that come in, like, there must be a bunch of stuff you have to do within the business to to get this all all up and running. So, like, what does that look like from, you know... I don't know. You're talking to NVIDIA folks, and they're, you know, giving you early insight into into the next generation of chips to, like, you know, actually getting these live for a set of customers?
00:10:43 · Danila Shtan
[00:10:43] S0: That is a mess. No. I mean, like, in a good way. You you know? Like, this whole this whole market, this whole field is running at such a pace that everyone is trying to catch up. Right? And NVIDIA, despite being in the lead of this, essentially leading and showing the direction for this market, is not the exemption. So first, NVIDIA folks, like, way before show you the prototypes of what they want to build in the next, like, year. Right? They kind of show you the schematics, like how it will be, like, optimized internally, where is your PCI connections, how would that look like, what is the future platform would look like. Then sometime later, they come to you and said, oh, like, we've optimized things. It's now gonna be different. And you're like, all right. Then they come to you and like, oh, we have, like, the first lab samples. Do we want to have a look? Yes, of course, we do want to have a look. Oh, by the way, it doesn't look like the stuff you've showed us two months ago. Oh, we've changed stuff because, like, we found, like... Right. Because they are so compressed in time, same as they... Everyone else. They are constantly iterating while these chips are being produced. Right? So they've... It's not your, like... You know, there is, like, an ocean in the consumer market. Like, Apple iPhone that you see today was actually planned three years ago, and final design more or less was settled, like, a year ago. And they take the final year to polish it to build this product. Like, AI market is running at such a pace, like, that whatever, like, they announce is it's indicative. And then teams are constantly
00:12:27 · Jacob Effron
[00:12:27] S1: Well, the architecture and the workflows are changing. Right? So I'm sure they're iterating as as that evolves.
00:12:31 · Danila Shtan
[00:12:31] S0: Like, they're iterating, like, going further and further and further. NVIDIA is a behemoth now. Right? Whatever they do influences the whole market, supply chains, just everything. So they have to, like, live with that. So at some point, you get a sample, and it's more or less like what you'll get in production. Then you start building. Like you're testing everything. You like incorporate support for the new hardware, for the new restrictions or capabilities into the software platform. You figure out a way to isolate workloads from from the chat. We are we are a cloud. We are not providing bare metal hosts. Right? Like, so we need to split. We need to make sure that our virtualization works, that our storage works, that our networking works. Sometimes we are on the bleeding edge of this. So we, like... For some of the generations, we were probably the first team... Like, maybe one of the first teams that actually built, like, a cluster of a certain size. So we've hit some issues with, like, with the software that no one has seen before. So we had to report that back to NVIDIA, like, propose a patch, get there, like fix from it, make sure that it runs, etcetera, etcetera, etcetera. So like, we kind of feel like we are at the forefront of it. But all in all, it's a pretty normalized process. It may sound messy. Right? There are like multiple iterations. You go back and forth. You're in constant contact with like NVIDIA hardware folks, with NVIDIA software folks. You're trying to solve the issues on your end at the same time, incorporating into a more complex platform, thinking how can you expose that to customers. Because, for example, working with, like, your usual black hole chip, like, you know, the b three hundreds, and then transferring that experience to GB three hundreds, which I recall different architecture in terms of isolation. You are talking racks rather than servers. So you need to find a way how to do maintenance on the rack instead of the server, which you just switch off and you move the workloads to another one. Despite all of that, this kind of works all the time. And every day, you like push it a little bit further. And then in the end, yes, there is a product and the customers use it. And that's it.