Lenny's Podcast: Product | Career | Growth
00:42:28 · Noam Segal, Lenny Rachitsky
[00:42:28] S0: When we asked people things like, are they enjoying their job? As we talked about earlier, most people said that overall, yes, they're having fun in their role. I think what's going on here is that again, people are lacking optimism compared to what they were. Optimism has gone down. So there's a big difference between how people are feeling now about their role, given what they've achieved in their careers, and where they are on the ladder compared to what they're extrapolating might happen in a year, in three years, in five years. And what people are extrapolating might happen is clearly more negative than their current experiences. So yeah, I mean all in all, we're doing okay as an industry. We're enjoying, we're trying new things, we're experiencing a complete paradigm shift in how we work. And that's fun for a while, but it seems like people feel like this is not going in the best of directions. And of course I hope we're all wrong. I hope we're all wrong. [00:43:48] S1: Me too. As a natural optimist, I'm like, "It's going to be all right." So we'll see.
00:43:53 · Noam Segal
[00:43:53] S0: I'm a natural optimist too, and I'm realizing so far we haven't been in too much of optimism mode, so I hope we manage to get to some of that as well. This was a very interesting one to me. In last year's survey that we did on the impact of AI specifically on productivity and quality, people were rather bullish about the technology and how much better it makes them, not just more efficient, but better. And this time around, we saw some interesting findings. Let me start with this. And this is the initial view, but it leads to something very different than I thought would happen. When we simply asked the question, how much better are you at your job thanks to AI? Unequivocally, the answer was yes. You can see here that 97.2% of people are saying that AI is making them better at their job, and close to 50% of people are saying that AI is making them very much or extremely better at their job. So it's very clear that high level, however you interpret better, which we'll get to in a second, people feel like AI has had an impact. But when you dive deeper into what that means, and you go below the surface level, as you can see here, where doing better actually means something rather different than what you might think. So when I heard people say they're doing better, I thought they meant that literally, I thought the quality of their work was better, that AI was contributing to better quality of work. In fact, it's the exact opposite. What we heard from people were two concerning things. The first is that I can do more faster, but not better. So again, AI has lowered the floor of what's possible and enabled people to put out a lot more work than they ever have before, whatever that work is, whether it's PRs, PRDs, research projects, prototypes, but the output isn't of higher quality. And perhaps the even more concerning effect, which we heard about, the deeper cost of all of this is the cost of using AI on thinking and judgment. People essentially told us, "My brain is rotting. My work feels worse." And this is related to phenomenon I've heard about recently, is it cognitive rot? I think is what I've heard. Basically, this phenomenon where you see the initial output of an AI model and you just accept it. You don't apply your judgment to it.
00:48:17 · Noam Segal, Lenny Rachitsky
You don't apply your thinking to it and you slowly let your mind, your involvement in your work, your agency collapse into this rotting state, so to speak. [00:48:36] S1: Yeah, this is such an interesting takeaway. There's a positive note to this. As you said, I'm looking at the numbers here. 82% of people say AI is making them at least moderately better at their job and nearly 50% say very much or extremely better, which may correlate with that original, just this bifurcation. The people that feel like AI is making them better at their job, probably the people feel most energized. So whether it's true or not, so I think there's a couple happy points. That's one. People feel AI is making them better. They feel that. So that's real even if it may not be. And at the same time, I think they're self-aware the quality is an issue. And also, maybe I'm atrophying in my ability to say code or write strategy docs. So at least people are self-aware that this is happening, not just I'm killing it, all my stuff is higher quality and it's much better than ever.
00:49:28 · Noam Segal
[00:49:28] S0: They are self-aware, but among other things, I'm a certified coach and I would suggest that beyond being self-aware, we need to act on this. There are several explanations I could give to this phenomenon. One of them might be that the honeymoon period with AI is over. We thought it was perfect and the best thing that ever happened to us since sliced bread or whatever typical Americans say, but we've realized it's not quite there yet. And yes, the models are the worst they're ever going to be, they're only getting better, but there's still- And yes, the models are the worst they're ever going to be, they're only getting better. But there's still so much they need to get better before we can really say, "Yeah, my work is just so much better and I don't need to be as involved with it." You had Simon Willison on this show talking about how people are worried about skill atrophy. That they're not learning enough and that we need to be consciously resistant to this. And not let ourselves cognitively rot. And I think this data suggests that we're losing that fight a little bit. We need to put a little bit more energy into it, into fighting this cognitive rot that we're feeling. And to make sure that as the bar continues to rise and this technology continues to evolve, that we are meeting that with deliberate practice, deliberate thinking, deliberate judgment.
00:50:59 · Lenny Rachitsky, Noam Segal
[00:50:59] S1: People are just so bad at that. Just like if I have an easy button, I will press it. I know I shouldn't, but I'm just going to eat those Doritos. So that's the challenge I think for people. It's just like because you could do it the easy way, it's so hard to resist. And that connects again to this ladder that's being raised that people are just getting into this stuff. It's just they have to make themselves do it the hard way to learn how to do it for real. And I'm really curious how that plays out with junior folks in the market learning how to actually code, how to do peer writing, how to write a [inaudible 00:51:28] from scratch with your fingers versus talking to Claude about it. [00:51:31] S0: Yeah, there's so much to say here. I think people miss feeling smart sometimes. I think I thought once upon a time about myself that I'm smart. And then you encounter these models, most recently a Fable 5 that just came out for the second time and you don't feel that smart anymore and you sort of lose the sense of self-efficacy that you used to have. And the second thing to say about this is that every time you, you, not the AI, solve a problem and get over a barrier, it increases, it raises your baseline of self-efficacy, of self-confidence, of self-belief. And every time you offload that to your favorite AI model, you are lowering that baseline and your thinking and your judgment is watching. And that's a serious problem.
00:51:31 · Noam Segal, Lenny Rachitsky
[00:51:31] S0: Yeah, there's so much to say here. I think people miss feeling smart sometimes. I think I thought once upon a time about myself that I'm smart. And then you encounter these models, most recently a Fable 5 that just came out for the second time and you don't feel that smart anymore and you sort of lose the sense of self-efficacy that you used to have. And the second thing to say about this is that every time you, you, not the AI, solve a problem and get over a barrier, it increases, it raises your baseline of self-efficacy, of self-confidence, of self-belief. And every time you offload that to your favorite AI model, you are lowering that baseline and your thinking and your judgment is watching. And that's a serious problem. [00:52:26] S1: So maybe just wrap up this takeaway, the sentence used in this report I love, I'll just read. The productivity gains are real, but the quality of the work and the sharpness of the person producing it are taking a hit. [00:52:37] S0: Absolutely. And maybe we'll get to this in the end, but we should keep a close eye on this and really try to work on our own self-efficacy. [00:52:47] S1: We need evals for people. Are humans becoming- [00:52:49] S0: Evals for people. Let's do that. [00:52:52] S1: [inaudible 00:52:52] your smartness. Okay. Let's talk about takeaway number five, which I also though was extremely interesting, my God. So much interesting in this. Let's talk about the fifth takeaway.