Lenny's Podcast: Product | Career | Growth
00:24:01 · Lenny Rachitsky, Anish Acharya
[00:24:01] S0: Such an interesting insight. So, just to make sure I understand what you're describing here, you're thinking there's going to be this split between within an org of function and model, where specific functions that have a lot more upside and potential leverage go for the frontier models, and you described these kind of product sales, engineering research roles. And then, for other functions you don't need Mythos, you don't need Astra, I think is that the latest one that's coming? [00:24:29] S1: Yes, Astra. Yeah, yeah. [00:24:30] S0: And so it's both, you're saying the AI doesn't need to be the frontier model and the people don't have to be the smartest people in the world to do really well in that world. [00:24:37] S1: Yeah, and I don't want to be diminutive. There's extraordinary people in those job functions. I just think that they're bounded upside problems that they work on. You can only kind of close the books correctly, you can't close the books 100X better. So yeah, that's exactly what I'm saying. [00:24:53] S0: I wonder if it's connected back to that discussion we had earlier about it's verifiable. If it's a lot more verifiable, you don't need the frontier model versus, I don't know, the potential upside.
00:24:53 · Lenny Rachitsky, Anish Acharya
[00:24:53] S0: I wonder if it's connected back to that discussion we had earlier about it's verifiable. If it's a lot more verifiable, you don't need the frontier model versus, I don't know, the potential upside. [00:25:04] S1: I'm not sure. I mean, I think for a ... Verifiability is a good question because I think for a drug development company, you have infinite upside, it is verifiable, but closing the books is also a verifiable problem, which has limited upside. I think the question is really how much upside is there and how hard is it to calculate it? Here's a nuance. Customer support, a customer may call in and report a bug. That bug may actually be the first breadcrumb to a thing that changes our entire organization. And if the CEO was on that call or the smartest person, they could follow the trail. But if we actually have this "mid-IQ generalist," then not, that actually makes the case for as a basket of problems always use frontier intelligence. And I can of course make the case for the mid-IQ basket as well, which is simply that we've crossed an intelligence threshold for almost every economically useful problem, and anything beyond that threshold is simply waste. [00:25:57] S0: Yeah. And I think the point people forget is that the models that are not the frontier models, that was what the frontier was, I don't know, six months ago, and we were so impressed and we loved it. It was like, "Holy shit, it can do all this." And now just because there's something better, we don't give those models as much credit.
00:25:57 · Lenny Rachitsky, Anish Acharya
[00:25:57] S0: Yeah. And I think the point people forget is that the models that are not the frontier models, that was what the frontier was, I don't know, six months ago, and we were so impressed and we loved it. It was like, "Holy shit, it can do all this." And now just because there's something better, we don't give those models as much credit. [00:26:12] S1: You're right, because it's sort of like, "This thing is so crazy. This is AGI." And then, a day later it's like the old thing that you throw in the dustbin. [00:26:22] S0: So along those lines, I hear people jokingly call you a model sommelier. Tell us with your sommelier credentials, what's the current state of the model art? What is each model great at? What are model's terrible at? [00:26:38] S1: Yes. Well, as you know, the secret of every sommelier is 10,000 hours or maybe 10,000 bottles. So, I think that the secret of being a model sommelier is just using them all. [00:26:47] S0: Drinking them all.
00:26:47 · Lenny Rachitsky, Anish Acharya
[00:26:47] S0: Drinking them all. [00:26:48] S1: I push myself really hard to ship something with every new model that comes out, and I think you learn so much. I think for people who believe the models are commodities or totally fungible, you just haven't actually used the models. And for example, in the last few weeks I've been obsessed with Qwen 3.8-Max. Qwen is an awesome model. It's very good at long horizon tasks, and it's actually really creative. It's a great storyteller. So, I've been using it to create these impossible documentaries, mostly of planets of the Star Wars universe. I did Tatooine, which turned out great. I did Bespin last night, and it can just work for four or five hours. It tells a great story, it generates all the video using the MiniMax model via Fal, generates audio via Eleven. It actually directs the movie, the five-minute movie, and then it cuts all of the clips in. It overlays it, it watches it. It's just extraordinary. It's got a totally different shape than GLM-5.2. GLM-5.3 just came out, which I used for a bunch of product work. It doesn't have a vision component, and it's sort of like this neurotic PhD you put in the corner, and they both have the role. This is a little bit of these tension of models. It's not that one is ahead of another, one is more intelligent, it's rather one has a mind that's shaped in one direction, perhaps creativity and openness for Qwen, and others that are shaped in other directions like neuroticism and precision like GLM-5.3. So, I just make something with every model and that's how I build my intuition.
00:28:22 · Lenny Rachitsky, Anish Acharya
[00:28:22] S0: How important is this habit, do you think, for people? Because I hear a lot that it's really important to be using these models. And a secondary question is how do you come up with what to do with these models? Because a lot of people want to try these things. They're like, "Okay, what do I do?" [00:28:37] S1: I know. [00:28:37] S0: Any suggestions? [00:28:38] S1: Well, I think almost all of us have got ... The most insufferable thing for a long time was your app idea friend. Every time you went to have a beer they're like, "Let me tell you my app idea." You're like, "Oh, here we go again." We have got to be the app idea guys now, and all the silly ideas, actually, especially the silly ideas, because those are the ones that often have the most alpha, are the ones that we should all be building. So, I've probably got two dozen apps that I've built, I've got one or two large apps that I iterate on. And I think that if you don't have a chassis with which to use the models, it's just really hard to come up with an idea from scratch every time. So, I'd say work on something. It's actually better if it's not important with a capital I, and then keep finding new ways to invest in it and add to it with a model as a kind of tool rather than as a goal. Does that make sense? [00:29:27] S0: Yeah. Maybe even zooming out, I've been thinking more and more one of the most important habits to build right now is to, whenever you're about to do something, ask yourself, "How can AI do this for me?" [00:29:38] S1: Totally.
00:29:38 · Anish Acharya, Lenny Rachitsky
[00:29:38] S1: Totally. [00:29:38] S0: I'm visualizing this input to response, the whole idea of there's a space between input and response, and meditation helps you think more deeply before you respond. And I feel like the trick now is insert in that moment, "How can AI help me with this?" [00:29:53] S1: I mean, I'm also a longtime meditator. We should talk about that if you like, but yeah, I think that's right. I think in our day-to-day knowledge work, sometimes it's less obvious to me. I guess my mind is I've always been a consumer product person, I love products. So for me, it's easier to think of a new feature to add to my DJ streaming app or my Google Reader for X that I use than it is to find a part of my life to automate. But you're right, there's some fun examples. I posted about one a few weeks ago where I had my laptop transcribe everything that was happening in the kitchen, and then it would award or detract screen time from my son's iPad depending on whether he was being good or bad. So, it was a fun little social experiment. It also had a fun outcome, which is that he recorded a video of himself saying, "I love you, Dad," over and over, and put it next to the mic. [00:30:42] S0: To hack the metrics? [00:30:43] S1: To hack the metrics. So yes, anything becomes a measure it's no longer useful, but it was just a cool little social experiment. I think those things are super fun. [00:30:54] S0: That is hilarious. So, one of the measures was how often he said, "I love you," and that was going to give him more free time?