Lenny's Podcast: Product | Career | Growth
00:28:24 · Lenny Rachitsky, Elizabeth Stone
[00:28:24] S0: That is awesome tactical advice. Making your manager's life easier is always a good tactic career-wise. [00:28:31] S1: Several reasons. Yeah. [00:28:34] S0: Following the thread a little bit, I know you all added career ladders and levels. Recently, it was a new thing. You used to not have these things. So, on that thread, what have you added to the career ladders within this AI world, if anything, that you find you want people to lean into more, you're looking to more or not? Did you not change your career ladders and performance criteria?
00:28:58 · Elizabeth Stone
[00:28:58] S1: So, the way we've approached this so far is, instead of trying to articulate at each level exactly how AI changes those expectations, to instead put an overlay across all of the talent at Netflix, people on the team and those who are hiring, to talk about an aspiration for AI fluency. And what that looks like is going to vary by function. It's going to vary based on where you are in your career. That could be what level you're in or what type of role or persona work you're doing. But the aspiration for AI fluency, which is a tough thing to define. So, does it mean that I have an experimentation mindset? Does it mean that I know where AI is useful and not useful? Does it mean that I've actually built things using AI? I feel like the way that has shown up in career ladders and how we talk about it evolves almost by the quarter, if not month or day, because the tech itself is advancing so much. So, the most useful thing is not to make it level specific or role specific, but to encourage everyone towards the expectation on AI fluency, which doesn't mean use it as a tech for the sake of tech. It's tech where it's useful, to have good judgment about that and to have the mindset to be open-minded to explore and try new things, that's the non-negotiable for all roles. And that's true at the senior most levels of Netflix, where we talk about we too need to have deep fluency in AI, even if we're not writing code as part of our day jobs. So, that's changed. And then that's showing up in our hiring practices as well, getting comfortable within interviews, exploring how are people thinking about AI or technology. What are they using in their day-to-day or their current job? How comfortable are they with change and exploration? And even for things like coding interviews, allowing candidates, of course, to use AI tools, because that's going to be part of what the work requires now. So, those have been shifts that we've made, but I doubt it's a shift that's done versus we're right in the middle of it.
00:30:58 · Lenny Rachitsky
[00:30:58] S0: I'm just going to keep following this thread. Obviously, AI is transformative for coding. It's a big unlock for prototyping. Are there other use cases of AI at Netflix that have been really impactful, that people may not think about or not realize?
00:31:16 · Elizabeth Stone
[00:31:16] S1: So, there's two that come to mind. So, the first is data analysis, distillation of information, modeling, which is using the tools to get our arms around all the insights we have, similar to what I mentioned before. What experiments have we run? What are the metrics that I should be looking at for a certain problem? What's the consumer research that we've done? And that is much higher velocity and much higher quality, contingent on you check that the results are valid, you work with your local data scientist on, am I using the source of truth data on this? But that's been a great one. And that's one personally that I would say I most use some of these tools for. So, that goes beyond prototyping and coding to general analytical thinking, and translating data to action and insight. The other one is on the content production creation part of the business, which has lots of applications. This was true before GenAI. So, ML and AI were deeply used in a lot of the production tools. We've used them to think about how to create promotional assets at scale, how to localize in subtitles and dubs. So, GenAI is a big step function in where the impact can be. In creative ideation, we call those things pre-visualization or basically bringing a creator's vision to life before you even get into the... You bring people to a set and start to actually go through the production itself. There's lots of use cases in post-production. So, we recently acquired a company, InterPositive, that was started by Ben Affleck, that built a set of models and capabilities. That allow you after you've shot something to relight, reframe, reshoot, change dialogue in ways that are very impactful to get higher quality content. Are still led by the filmmaker, creator saying, "You know what? I would like to try something else to bring this vision to life." But that impact is extremely promising and we're seeing lots of productions leverage different tools, some of them built in-house, some of them that we enable through other vendors for those content creation use cases. And then as we think about how content comes to the product, I mentioned localization, subtitles and dubs, but also how we create high quality trailers, images, artwork at scale that then we can use to help make sure that titles find their audiences around the world.
00:34:50 · Elizabeth Stone, Lenny Rachitsky
Those all are huge levers when we think about the AI impact. So, that again, goes well beyond prototyping or coding to some of the creative use cases. And you can imagine that, just like they work for studio productions for film and TV, they work for advertising, they work for marketing, off-service campaigns. And so, those are all areas that we're exploring. The Netflix prize. [00:35:26] S0: Yeah, the Netflix prize. Just show an example of how early you were to AI and ML. I think it was a million dollar prize to optimize the Netflix ranking algorithm a little bit, whoever could optimize it the most. And I think the winner optimized it by a few percentage points, something like that. And it was a huge deal. All these super smart people got around the world and it happened a few times, right?
00:35:49 · Elizabeth Stone
[00:35:49] S1: I mean, you said it on my behalf. Often, when there's questions about how is Netflix thinking about AI, it's great to remind people of exactly that point, that this is not new to us. That especially for personalization, it's been central to delivering a great experience to members. It's impossible to take the breadth of content that we have. There's ever more content, that's one of the challenges we face. And make discovery easier and easier and easier, which is one of the challenges that Netflix has. And using AI and ML has been a way to do that. You want to personalize right title for the right person at the right moment. That problem gets harder. The more exciting our catalog gets, the greater breadth of content we have, not just film and TV, but games and live and podcasts. Personalization becomes even more important and what that experience is. So, we can take a lot of that history and say, okay, well, now how do we solve this problem? Because the tech is even more powerful, but it gives us a running head start in being clear about the problem to solve, how important it is that Netflix solve that for our members. And then the same is true, as I was mentioning on the creative side of the house, AI and ML have been in things like visual effects or in localizing language for a long time. Now we say, what's the next era of that when the tech is more powerful? And in both cases, it ends up taking a strength that Netflix has, which is marrying entertainment, and technology, and making sure we stay ahead of the game to deliver things that are even better. So, I love that it's part of our history. It still continues to be a strength and it's going to have to be a strength, given the size of the challenges we're facing around the breadth of entertainment, while keeping a great experience.