Lenny's Podcast: Product | Career | Growth
00:03:02 · Elizabeth Stone, Lenny Rachitsky
[00:03:02] S1: Brian's amazing, so I'll let that one go. [00:03:04] S0: Yeah, he is. And then there's just all these fancy AI people that are just coming in hot. So, it's been two and a half years at this point. A lot's changed. Obviously, AI. Something AI is allowing people to do is everyone can be everything now. This idea of PMs can ship code, designers can write PRDs, and engineers can product and everyone's everything. There's a bunch of elements of this conversation. One is that I've heard from people that there's also this kind of confusion and frustration of what is my job anymore. What am I responsible for as a PM, as a designer? Is that something you've experienced?
00:03:44 · Elizabeth Stone
[00:03:44] S1: I hear it within Netflix for sure. I think anytime a new technology comes along, especially one that's as transformative as GenAI, you go through a storming phase before you go through the forming phase of things. And I think we are in the middle of that right now. I don't think that means we should put AI back into the box and say, let's not use it, because this is complicating all of our preconceived notions about our roles. But I do think it means we have to be much more thoughtful about how do we get the benefits while reducing the costs. I think it's a great thing that people are experimenting with. How can I develop an idea faster, prototype an idea, put together an initial set of code that would allow us to test it? Do I believe that means anyone should be shipping code to production, that everyone should actually be doing everything? Probably not, but I think that it's good for people to be exploring what's possible. And then like I mentioned earlier, the benefit of having product and tech teams together, is that if the business problem is clear, I think it's okay and it's healthy for there to be some fluidity in the roles that people play. Because instead of having to wait for the engineering team to be ready, to be able to prototype something, product and design can move faster on it, but they should still work with their engineering partner to think through, how should we productize this? How do we scale it? What are the guardrails for it? So, I don't think it makes the functional expertise obsolete. I think it means that teams have to be more comfortable with maybe this helps us move faster in a certain direction. From an organizational perspective, things I think about to make this more coherent or less frustrating, are some of the things that have to be in place for us to get the benefits rather than the cost. So, that includes clarity on source of truth data, guardrails on shipping code to production or testing, before we make large changes. Thinking about opportunities where we can trust the output of AI versus we should have a process or review that helps us check that we're getting high quality outcomes. And the importance of reiterating that humans are still responsible for what happens.
00:06:34 · Elizabeth Stone
So, it can be that an agent wrote the code or I helped to do an analysis when that's not really my background, but it doesn't make people not have the responsibility that comes with what they've created. So, I think investing in some of those core infrastructure and practices, and reiterating the accountability and responsibility for the outcomes, helps to balance some of what's possible with what we should actually be doing. So, you've mentioned some of the things, so I'll reiterate them and then maybe build. So, I have found that PMs, designers, data scientists, are able to get farther in the product development lifecycle before engineering really needs to be front of the line in unlocking things than was true a couple years ago. I say that with some caution, because like we were talking about, I don't think it's great to all of a sudden have thousands of prototypes, if they're not aimed at this is an important problem to solve for the business. And the engineering partners are aware that we're solving that problem. And that designers and product managers are going to take the lead in starting to shape the idea, but it's not working in a vacuum and it's not throwing a bunch of spaghetti at the wall to see what sticks. But when it's the right problem, approach in a thoughtful way with some alignment on that. I've seen product design data science move faster in the direction of let's get to something that's testable on this hypothesis. So, that's prototyping, that's writing code. The other thing I've seen, has been very valuable is we have a lot of information running around in the virtual walls of Netflix. We have experiments we've run over decades. We have insights from consumers. We have input from stakeholders across the business. And that was a problem that really presented a challenge of, how do we get the most out of that long history of knowledge and learnings, to say, let's apply that to the problem we've got now to move faster in this is a promising path or this is something that we've learned something about and we could leverage here? And AI is very powerful at distilling information, looking across a broad set of things, doing an analysis around it, getting to the core of here's some insights to start with. I would hesitate to rely on that exclusively, but I think it's a head start.
00:09:30 · Elizabeth Stone
And I find even in my own work day-to-day, instead of sending an email that disrupts someone of like, remind me what research did we do in what year and what was the question and what was the test we ran? I can find that almost instantly. Then I can form my own, here's what I find interesting about this. And I've now skipped a couple steps towards, is there something actionable here? So, that's data analysis, it's modeling, it's distillation of information. And I'm seeing more people do that, to your original question. So, instead of that needing to be only the experts who were here for 20 years and saw every experiment or know where to find it, we're now able to do that faster within product and tech across all functions. And a big unlock for us is our business stakeholders, sitting in finance and content and advertising, can do that as well. And then bring back an initial hypothesis where they want to work more deeply with the data scientist and engineer and so on. So, there's something there about the hypothesis generation, prototyping, thinking deeply about problems, that feels like it's accelerating and that functions are able to do that in a more fluid way. But I still see comparative strengths. So, data scientists are still going to be experts at, can we trust this data? Are we interpreting it the right way? What's the data versus judgment that we should be applying here? A product manager is still going to be exceptional at saying, have we really framed the what of this, the problem we're solving in the right way? An engineer still has a craft around the how. How does this scale? What does high quality look like? What problems is this going to create for us based on how we build and deploy something? So, I still see the nuggets of that comparative advantage. It's just that we're able to move more fluidly in a lot of steps that normally we would have blockers on.
00:11:55 · Lenny Rachitsky
[00:11:55] S0: There's so much interesting stuff here. One is this last point you made, something I've been thinking about. If we all become builders, will we still need separate functions? There's this member of technical staff trend that is happening across AI, where it's like, all right, we don't have a title. You could be anything. You don't have to be in a bucket. What you're saying here is you believe we will continue to have specialties, product person, engineer, data science, designer. While they do more of other functions, there's still a lot of value, and tell me if I'm hearing you correct, in having this specific discipline and skill and background.
00:12:26 · Elizabeth Stone, Lenny Rachitsky
[00:12:26] S1: I still see a craft excellence that's really important in the disciplines that I don't think is going away anytime soon, even if there's fluidity or blurring of the work across the functional lines. It goes back to what I mentioned earlier of you still have humans who have to make sure that what we're doing makes sense. We're solving the right problems in a way that is best for Netflix members or business stakeholders. And that if I talk to an engineer, a data scientist, a designer, yes, they speak more languages now than they used to because they have the benefit of these AI tools, but there's still something that is not replaceable when I think about the craft and how they think about what good looks like. And that feels true across all levels. And I still find great engineering to be scarce, great data science to be scarce, great creativity to be scarce. So, yes, some things are easier, but that hasn't dissolved in my mind. [00:13:27] S0: Are there functions that you are finding you are hiring more of, like the pie chart pie expanding, say for engineering or PM or design or something, and then functions you're need less of with AI tooling and LLMs rising?