Lenny's Podcast: Product | Career | Growth
00:09:24 · Anish Acharya, Lenny Rachitsky
And that means burning the buildings down and starting from scratch versus taking what was previously coal and simply swapping it with electricity. So, I do think that the most ambitious companies are rethinking everything around the models, and those that are a little less ambitious or perhaps a little earlier are thinking more about how do we give people in existing orgs, existing job functions, access to the technology. [00:10:01] S0: So, kind of a theme I'm hearing so far is we can be a little less stressed about where things are going and the future of your job, your careers.
00:10:01 · Lenny Rachitsky, Anish Acharya
[00:10:01] S0: So, kind of a theme I'm hearing so far is we can be a little less stressed about where things are going and the future of your job, your careers. [00:10:10] S1: I think so, man. I mean, I think if you even just think of the kind of incentives for the CEO and executives, Sundar running Google, he doesn't want to run a more efficient $4 trillion company, he wants to build a $40 trillion company. So anytime you have an economically productive unit, it's rational to, especially if it gets more productive, to maintain that sort of presence in your organization. And then, I don't know what you hear, but anecdotally, I talked to a good friend who's an executive at Google and I said, "Hey, have you laid anyone off?" And he said, "No, we didn't. What we instead do is now rip through our roadmap." So, two years of roadmap happens in three months, and we're actually our hardest problem is knowing what to add to the roadmap, which by the way is every PM's fantasy. How much emotion has been drained on prioritization conversations between you and I? So yeah, I actually think that people shouldn't be as stressed and I think they should feel really empowered. And by the way, the best way to do it is just to ship stuff. I mean, Claire is my muse. She's so awesome because she's- [00:11:07] S0: Claire Vo?
00:11:07 · Lenny Rachitsky, Anish Acharya
[00:11:07] S0: Claire Vo? [00:11:07] S1: ... always shipping. Yes, Claire Vo, she's shipping, she's trying things, she is not afraid to be a little embarrassed by it. And if you just see her whole kind of affect, she feels like the best version of herself that she's ever been and I think we all have an opportunity to be that. [00:11:23] S0: I love that. I want to be Claire Vo when I grew up. [00:11:25] S1: Totally. [00:11:26] S0: Kind of along those lines, you have this interesting take that company building more and more is going to become this kind of series of creating loops and creating series of loops. Talk about that.
00:11:36 · Anish Acharya
[00:11:36] S1: Yeah, yeah. Well, I think the broad concept and the loops concept kind of gets teased a little on X because at sometimes it feels like maybe we're big braining it. I know there was a big meme around graphs, like the next stage of loops is graphs, but let me make the kind of steel man for it, which is we had prompts and then we invented agents. Agents of course are just models in a loop with tools and memory and skill files. And then we had sort of loops, which are sets of agents that are doing tasks. If you look at a lot of work in coding, coding is such a great domain because you have the best models and you have the most sort of technically apt customer, plus you have these established loops, like bug fix, or a bug report comes in, repro gets generated, bug fix gets created, bug fix gets reviewed. If high risk, then humans should confirm that it's okay to ship to prod. And if low risk, it just gets shipped. And maybe you even email the customer and say, "Hey, we fixed the bug you reported." That happens in five minutes. There are many loops like that in engineering, everything from bug fixes to customer feedback, to sales demos, to new feature development. So, coding sets itself up well. So, you have a coding loop and the kind of change it makes is to the code base. My question is what are the business loops? So if you're the GM of a business, you're looking across many job functions and you've got loops running in coding, and marketing, and sales, and support, and legal, the output of all of those loops is something that is itself a loop that you should be able to optimize for. And I think the strong form of this is that it sends a message to the CEO saying, "Hey, we need to actually make a change to one of the physical aspects of the business, or to our business model, or to our strategy." So, I think we're going to see this sort of cascading set of everything from a loop per person, loop per job function, loop across entire business units to loops that can run large parts of the company. With that said, I think humans are a critical ingredient. I just don't think that most work in the organization can be done fully autonomously. When you think of what a human will do in this AI native company, sales, support, strategy and exceptions. And all those things are super critical.
00:13:42 · Anish Acharya, Lenny Rachitsky
We've seen one thing, Lenny, it's that the ability for models to do new thinking out of distribution thinking is still really limited. And I don't actually take the point that some of the new thinking in math is actually representative of new thinking in domains like business. So, you're still going to need a person to say, "Hey, here's the thing I think we should make," and have them be right about it. [00:14:05] S0: Let me just kind of make sure this point is really clear, because it's so interesting. What you're saying here is engineering and building more and more is becoming this loop of input, feedback, support ticket, whatever, input of just what to build. And then AI more and more is taking that, deciding, "Here's a PR. Here, is this ready?" And then shipping it. And you're saying that you expect that to spread to say go to market, legal, growth, support. So, maybe describe what that loop looks like or may look like within a company. [00:14:38] S1: I mean, a great example is a growth team. You worked on the growth team at Airbnb, right? [00:14:41] S0: Yeah, yeah. [00:14:41] S1: Supply growth. [00:14:42] S0: Yeah, that's right. [00:14:42] S1: Awesome. [00:14:42] S0: That's right. [00:14:45] S1: I don't know how you ran your team, but I'm guessing it was something like you got everyone together, you built a list of possible experiments, you prioritized them, you built them, you ship them, you measure them- [00:14:53] S0: Brainstorming. Yeah. A lot of brainstorming- [00:14:54] S1: Correct, right? [00:14:55] S0: ... a lot of spreadsheets.
00:14:55 · Lenny Rachitsky, Anish Acharya
[00:14:55] S0: ... a lot of spreadsheets. [00:14:56] S1: So, the loop version of that should be that every variant gets generated, every variant gets measured. Once you get to stat sig with a high enough P-value, you converge and ship that variant. You then have a long-term holdout and you start working on the next experiment. And then you're going to hit some local maxima, and I think this is really important. The loop will help you climb to the local maxima, but then it plateaus and you need some sort of out of distribution thinking, you need human intuition, you need somebody to actually help you land at the base of the next hill. [00:15:26] S0: Yeah, you have this chart. I don't know, maybe we'll show it over later as we talk about this, which is such an interesting way of thinking about it. This idea that agents will help you hill climb and reach some new plateau, and then you need a human there to think about a bigger idea, kind of unlock it, and then it keeps going and going and there's kind of this agent to human kind of back and forth. [00:15:47] S1: Yes. Yeah. And you know what really illustrates that? If you've ever tried to have an agent come up with a business idea for you, like, "Hey Claude, make me a million dollars, make no mistakes," why doesn't that work? And it's because you sort of need to set it in the right direction, and nothing in the technology has shown us that that is not needed.