Lenny's Podcast: Product | Career | Growth
00:21:06 · Lenny Rachitsky, Casey Winters
[00:21:06] S1: Makes me think about some of the best researchers I've worked with, and they often tell me, like, we don't need to do research on this. We have enough information. Like, we we have other things we should be doing. [00:21:13] S0: I I love it when they say that. Yes. [00:21:15] S1: Yeah. This would have been a really good segue to another topic I wanna talk about, which is around gut instincts versus team expertise, which we're gonna get to, but I have a couple more questions along these lines. Sure. You talked about interviewing PMs at Whatnot. Can you actually describe Whatnot just because you mentioned a couple times just so people know what we're talking about? [00:21:31] S0: So Whatnot's a a live streaming marketplace, mostly focused on the collectibles market. So, you know, sellers, whether it's like, you know, baseball cards or women's handbags or sneakers, you know, they'll go live on video and show you the products they have that they're selling, and people who are watching the streams can engage with the sellers and bid on, you know, different items. So kind of like Twitch meets eBay would be the you describe it. [00:21:54] S1: Awesome. And I'll just mention I'm a tiny angel investor in that company. [00:21:56] S0: Oh, that's why you brought it up. Okay. I get it. Yeah.
00:21:56 · Casey Winters, Lenny Rachitsky
[00:21:56] S0: Oh, that's why you brought it up. Okay. I get it. Yeah. [00:21:59] S1: I just wanted to make sure people understood what this word was because it's like a a fun word, whatnot. But what I wanted to actually get to is you said you were interviewing a lot of product managers for them. Yeah. What is your approach to interviewing PMs? What do you find is a is a really good signal for someone that's gonna be successful? [00:22:14] S0: I feel like the whole thing's gotten so performative. [00:22:17] S1: Mhmm.
00:22:17 · Lenny Rachitsky, Casey Winters
[00:22:17] S1: Mhmm. [00:22:17] S0: It's like interviewing is handing out Oscars based on who's prepared the best tell me about a time response versus actually assessing, like, who can choose the job we're hiring for. So it's like most PMs are better PM interviewers than PMs now. There's this quote from a movie called The Way of the Gun. I don't know if you've ever seen it. Nope. But Benicio del Toro says in it, it's like, these days, they wanna be criminals more than they wanna commit crime. Mhmm. And I think about that quote a lot when it comes to interviewing. So I would say I'm a bit contrarian here in my approach. I don't ask about your work history. I don't care about your perfectly practiced answers. I'm gonna give you real scenarios that I expect from the role. I wanna hear how you'd approach them. And if you can't come up with a few reasonable ideas, figure out how to test them quickly without analyst support or research, I'm just not interested. And I'm not saying this is a perfect approach. Interviewing is a very lossy format. But the more I can see them do the job we're hiring for with questions, whether it's a presentation, a prompt, that gets me the most comfortable that they know the job they're signing up for, that they've shown in a practice scenario they can do it, and they actually enjoyed it in some way. [00:23:28] S1: Are there, like, red flags you look for in these interviews? Like, one is maybe said over, like, I can't even begin to answer this without research and data. Is there anything else?
00:23:28 · Lenny Rachitsky, Casey Winters
[00:23:28] S1: Are there, like, red flags you look for in these interviews? Like, one is maybe said over, like, I can't even begin to answer this without research and data. Is there anything else? [00:23:36] S0: Some things I pay attention to is if they're talking about solutions that are gonna take a long time to get signal from. [00:23:44] S1: Mhmm. [00:23:45] S0: You know, whether it's months and months of engineering time before you actually, you know, see the impact on users or or the business, I think that's, you know, always great. If they're not factoring in the amount of time it will take, in general is not, you know, a good sign. And not thinking a bit more holistically about the types of metrics they expect to improve versus track to make sure, like, they haven't gone down. Those are some things that I pay a lot of attention to. [00:24:11] S1: How much do you think, they're intimidated by being interviewed by Casey Winters? How much do you think that factors in? [00:24:16] S0: I think you overestimate my importance or awareness in the community. I think most people have no idea who I am when they start talking to me. [00:24:25] S1: Mhmm. Well, I don't know. Another thought is there's this mean that g p t four is gonna... Or five or six or 10 is gonna replace product managers. And anytime some new feature comes out, people put out these videos of, oh, look. They're doing all the PM's jobs. What's your take on the future of PM in in AI, if any?
00:24:25 · Lenny Rachitsky, Casey Winters
[00:24:25] S1: Mhmm. Well, I don't know. Another thought is there's this mean that g p t four is gonna... Or five or six or 10 is gonna replace product managers. And anytime some new feature comes out, people put out these videos of, oh, look. They're doing all the PM's jobs. What's your take on the future of PM in in AI, if any? [00:24:41] S0: Well, I think if you thought the PM job was just filling in the latest, you know, Reforge or Shreyas framework and then getting, you know, that automatic Thang promotion every year and a half [00:24:52] S1: Mhmm. [00:24:52] S0: Then, yeah, you're gonna get replaced by AI. But I think the real PM job is the least likely to get replaced over time because you need, you know, real subject matter expertise. You need to, you know, be trading off a lot of different types of things and and making good decisions for the company and and for your customers. So in terms of using AI now as a PM, I'd actually be cautious in, like, the current iteration of the cycle for PMs. Yep. It's a tool that's trained on sounding smart rather than always necessarily being smart. I was at Eventbrite the other day, and someone was telling me how they were loving the new Notion AI integration. And she asked me if I'd used it. I said, no. She's like, oh, you totally need to. Hey. Go ask you your bio. It's really cool. And I said, alright. Let's just do it on [00:25:39] S1: your screen.
00:25:39 · Lenny Rachitsky, Casey Winters
[00:25:39] S1: your screen. [00:25:40] S0: So she did, and, like, my bio said I started my career at Google and worked on Google Trends and a bunch of other products, none of which happens. Like, it's just complete nonsense. It did have some stuff that did happen. It knew I worked at Eventbrite, and it knew I worked at, you know, like, Pinterest and stuff. But, like, like, half of it was just completely made up. It sounded plausible, but it wasn't actually, you know, true. I think where I'm more inclined for PMs to try to get leverage out of something like GPT four is a lot of the tedious work that's maybe not their specialty to begin with. So, you know, if you're not great at, like, Excel or Google Sheets and you need to model something, you can ask it how to format something. That'll probably be perfect. Or, like, how to get some Zapier integration to work. Like, it probably knows how to do that really well. So I think at this point, it's a pretty good no code, low code tool, and it's obviously gonna become a lot more than that. But for a lot of these other use cases that people are touting, there's a decent chance confidently tells you the wrong thing. And and that would scare me as a PM. [00:26:45] S1: I've had the exact same experience. I had it, describe what I've done, it was, like, 70% mistaken. Yep. Is there anything you've found value in yet with GPT of any kind in your your work, or is it still kind of just poking around in novelty for you?