Unsupervised Learning with Jacob Effron
00:34:29 · Jacob Effron, Danila Shtan
[00:34:29] S1: Yeah. Makes makes sense. I mean, I guess there's a lot of dialogue around the the bottlenecks to the AI build out today. And I think people talk about a bunch of different ones, you know, whether it's power or labor to actually make, you know, data centers, cooling, networking, you know, HBM, GPUs, I guess, as as a as a overall. Like, how do you think about what the what the key bottlenecks are today? If you were to to kind of rank the... Across all of them, the the the few that feel most most poignant right now? [00:34:55] S0: I'm sorry to disappoint. There is one single bottleneck, and that's your ability to execute. Because, yes, all of those are issues. Like, there are GPU shortage in the market, components shortage in the Nowadays, it's CPU shortage in the market. Like NVMe shortage in the market. There are, like, not enough facilities, not enough electricity, not enough not enough qualified, like, workforce to help you build it out. Not enough, like, small components like network transceivers, for example. What else is not in abundance? Oh, oh, well, you know, like, those large scale... I'm not sure. I, like, I I know the proper English terminology, but, like, you know, those machines that actually convert electricity in place like transformators. [00:35:48] S1: Oh, transformers. Yeah.
00:35:48 · Jacob Effron, Danila Shtan
[00:35:48] S1: Oh, transformers. Yeah. [00:35:49] S0: Transformers. Yeah. Like, do you know the lead times on those these days? Oh, years. Right? So, like, everything is a problem. And I think the single bottleneck and at the same time, the single, like, mode that you can impose on that is, like, your ability to execute around all these problems and concerns, your ability to source, your ability to cut corners wherever appropriate, your ability to execute in general, your ability and flexibility to move stuff between facilities. Okay. We're gonna move this one from here to there, and then we're gonna move it back, kind of like, yes, we'll postpone this one, but this means we'll be in time with this one. This is important, etcetera, etcetera, etcetera. So I think that, like, this particular field, like an experienced project manager, is probably in highest demand I've ever seen throughout my career. [00:36:46] S1: Yeah. That's... I I guess the the the bottleneck is really good project managers, which Yeah. [00:36:50] S0: Exactly. Like, your your overall ability to, like, have this large context of, like, problems everywhere in mind and navigate that becomes a moat.
00:36:50 · Danila Shtan, Jacob Effron
[00:36:50] S0: Exactly. Like, your your overall ability to, like, have this large context of, like, problems everywhere in mind and navigate that becomes a moat. [00:37:03] S1: Yeah. It's interesting because you obviously have, you know, the hardware itself changes... Changing. You have a bunch of workloads changing. Obviously, we have much more long running agents. We have more post training going on, you know, people running open source models. How does that actually impact the way that, you know, the overall, like, I guess, the overall build out of the clusters themselves? To what extent is it like, hey, you know, a workload's a workload and, you know, we can we can run it?
00:37:27 · Danila Shtan, Jacob Effron
[00:37:27] S0: No. The workload's a workload. Like... Exactly like you said. It is slightly different, But in a sense, you need a high enough baseline to not be constrained by that. And then the rest is just your optimization problem. The way you pack things. The way you split things. Right? Like you don't need whole nodes for some of the workloads. So you need to collocate workloads. You need to have the right primitives. You need to expose those through the platform, etcetera, etcetera, etcetera. But in a sense, like differentiated workloads almost never like impose any changes on on on how you build on the lowest level. Yes, of course, like nowadays with like these new architectures, like what is it called? NVIDIA LPX, like the outcome of the grog deal. Right? This like adds to it. But in a sense, if you have experience running other generations, this is just another like 0.5 generation. Right? Yes, this will be a little bit novel and it's connected in a in a weird... Like, compared to the previous one's way. And it kinda is... Like, there are some restrictions when you can run and not run on that. But efficiency gains. And this is like just a, like a niche local addition to the platform to what to what you offer to the broader market. But in general, no. There is like way less custom, like custom solutions that you would expect from that. The workload is a workload. [00:39:17] S1: Why didn't the hyperscalers, like, move into this market faster or win this market, you know, more convincingly early on in in in this first chapter, at least?
00:39:26 · Danila Shtan
[00:39:26] S0: First of all, sometimes there is a notion that they didn't even try. There is a solid amount of business that hyperscalers get from out of this market. Like huge deals, they are very competitive. Sometimes people say, Oh, have you seen AWS prices? Well, these are public prices. I assure you. Like for proper deals, they are doing proper prices and they're very competitive. And like they're very aware of the market. I think in a sense, like there are two two angles to that. First, like what I've mentioned before, a lot of these companies need to juggle like their internal consumption and what they can put for, like, for for the broad market. And there is... Like, we are in chip shortage world. Right? So, like, if NVIDIA could have bought... Like, could have produced, like, 10 x the chips, probably this this this whole market would have been shaped differently. Because they... Like, Microsofts and Googles of the world would both buy capacity for themselves and for the broader market to operate. This was partly... Like, simply not enough capacity for their scale. Right? Because there is not enough capacity for their scale, this is a relatively small business for them. AWS revenue is what? Like, billion or something? I mean, like, a business for, like, $2.03, 4,000,000,000. It's not insignificant paying margin. But they compared to what they... Their conservative, large scale established businesses, it's not that easy either.
00:41:08 · Jacob Effron
[00:41:08] S1: Yeah. But your point is basically like they're, you know, they get a chip allocation that is like, you know, not as large as they'd want. They have to use some of that, you know, a large amount of that for training themselves. There's only so big that that inference business or that that the cloud business can be given all that. And Yeah. Yeah. It's interesting to figure out how to prioritize. Do you go after like the few folks with large contracts or do you try and get this like mid market enterprise up and coming segment? But I guess you're you're kind of doomed either way if if you only have a certain number of chips to Yes.