Unsupervised Learning with Jacob Effron
00:00:00 · Jacob Effron, Erik Bernhardsson
[00:00:00] S0: How much of, like, an an organized marketplaces compute these days versus, like, the Wild West? [00:00:04] S1: I went to GTC last year. It was like an open air drug market, but, like, GPUs. Walking around, like, talking to people, trying to get GPUs, and meeting people in hotel lobbies, exchanging numbers. It's very far from an efficient market. I mean, some people are working on this. Compute comes in many different shapes or forms. I also feel like structurally, it will be a little bit, like, sort of unwieldy for a long time. Like, it's just not something you can, like, sort of buy and sell in an exchange because it's not like a fungible contract.
00:00:04 · Erik Bernhardsson, Patrick Chase
[00:00:04] S1: I went to GTC last year. It was like an open air drug market, but, like, GPUs. Walking around, like, talking to people, trying to get GPUs, and meeting people in hotel lobbies, exchanging numbers. It's very far from an efficient market. I mean, some people are working on this. Compute comes in many different shapes or forms. I also feel like structurally, it will be a little bit, like, sort of unwieldy for a long time. Like, it's just not something you can, like, sort of buy and sell in an exchange because it's not like a fungible contract. [00:00:34] S2: Welcome to unsupervised learning. As you can probably tell from my voice, I am not Jacob. I am Patrick Chase, GP at Redpoint. So awesome to have Eric on unsupervised learning for the second time. He is the founder and CEO at Modal. I have learned so much from working with him over the last three years. He is one of my favorite people to jam on AI ideas with. He has an incredible ability to move seamlessly from the lowest altitude of developer shipping code to the highest altitude of where the market is going and building what is missing. We talk about the mobile journey over the last three years, the crazy technology they have built to make it all work, inference, sandboxes, and what's coming next. Hope you enjoy the episode. For you guys, there is obviously a ton of AI compute demand and lots of people going out and trying to reserve, you know, GPUs and CPUs. What's it like for you guys finding compute in in the market today?
00:01:29 · Erik Bernhardsson, Jacob Effron
[00:01:29] S1: It's pretty rough. Mean, like, you know, not rough in the sense that like we can actually find compute. I think there's a little bit of a misconception that like we're sort of... We are certainly scarce, you know, in terms of compute in certain ways, but like there is capacity you can get. You just have to look a little bit further, accept higher prices, accept longer terms, etcetera. And and we do that a lot. I mean, probably do like two compute deals every week at this point, if not three. And and so we we keep to, you know, stacking up, you know, more and more compute. And and I really think that's the beauty of model is we can aggregate all this demand and and manage the capacity in aggregate much more efficiently. And and the value prop of model has always been like, don't think of my capacity. Like, we had a lot of that, like, annoying stuff to sort of the day to day knife fighting of of getting capacity and working... We work with 25 different clouds at this point and adding more all the time. [00:02:18] S0: How much of, like, an an organized marketplaces compute these days versus, like, the Wild West, like, random people that somehow have capacity and, you know, the late night text messages?
00:02:18 · Jacob Effron, Erik Bernhardsson
[00:02:18] S0: How much of, like, an an organized marketplaces compute these days versus, like, the Wild West, like, random people that somehow have capacity and, you know, the late night text messages? [00:02:27] S1: It... It's it's a lot of late night WhatsApp groups and, like, it's pretty unorganized. Like, I mean, I went to GTC last year or so earlier this year in San Jose, the NVIDIA, big conference they're doing. And it was like an open air drug market, but, like, GPUs. Like, it's just like walking around, like, talking to people, trying to get GPUs. And it it... You know, meeting people in hotel lobbies and, you know, exchanging numbers and, you know... I mean, at some point, we... I think we got connected with someone who had GPUs because I think our VP of sales cousin was golfing with someone, and and we got connected. Like, there's many such cases. Like like, it's very far from an efficient market. I mean, some people are working on this, and I think over time, it'll get a little bit more efficient. But it... It's very heterogeneous. Like, compute comes in many sort of different shapes or forms. So I also feel like structurally, like, it will not... Like, it will be a little bit, like, sort of unwieldy for a long time. Like, it's just not something you can, like, sort of buy and sell on an exchange because it's not like a fungible contract. Yeah.
00:02:27 · Erik Bernhardsson, Jacob Effron
[00:02:27] S1: It... It's it's a lot of late night WhatsApp groups and, like, it's pretty unorganized. Like, I mean, I went to GTC last year or so earlier this year in San Jose, the NVIDIA, big conference they're doing. And it was like an open air drug market, but, like, GPUs. Like, it's just like walking around, like, talking to people, trying to get GPUs. And it it... You know, meeting people in hotel lobbies and, you know, exchanging numbers and, you know... I mean, at some point, we... I think we got connected with someone who had GPUs because I think our VP of sales cousin was golfing with someone, and and we got connected. Like, there's many such cases. Like like, it's very far from an efficient market. I mean, some people are working on this, and I think over time, it'll get a little bit more efficient. But it... It's very heterogeneous. Like, compute comes in many sort of different shapes or forms. So I also feel like structurally, like, it will not... Like, it will be a little bit, like, sort of unwieldy for a long time. Like, it's just not something you can, like, sort of buy and sell on an exchange because it's not like a fungible contract. Yeah. [00:03:29] S0: Mean, the key differences being, like, I guess, the the kind of actual powered shell or size of the cluster and, like, the chips themselves or... Because that... You know, to your point, there's lot of people that have been trying to build, hey. You know, compute is this fungible single asset that you could build a marketplace around. What what are some of the kind of the key differences you observe?
00:03:29 · Jacob Effron, Erik Bernhardsson
[00:03:29] S0: Mean, the key differences being, like, I guess, the the kind of actual powered shell or size of the cluster and, like, the chips themselves or... Because that... You know, to your point, there's lot of people that have been trying to build, hey. You know, compute is this fungible single asset that you could build a marketplace around. What what are some of the kind of the key differences you observe? [00:03:46] S1: I I think a lot of this actually boils down to reliability. I I mean, first of all, it's like, you know, differences in like disk and networking, and of course CPU and memory and things like that. Like all these things vary and region or like location, like they're all over the world. Like we have GPUs in The US, of course, like all over The US, but we also have GPUs in like Indonesia, like, all over the year... All over Europe, etcetera. So so location is is one of those big factors. And then... But but also reliability. And and and that's another thing we do exceptionally well is, like, we aggregate all this capacity. And then when Neo Clouds die or, like, when they have failures, which they often have because some of these are, like, former crypto miners who haven't really thought about, you know, running a twenty four seven operation, we can just take it out of rotation or, like, when GPUs fall off the bus or whatever. Like, we... In many cases, we just file a ticket automatically. Someone goes and looks at it.