Lenny's Podcast: Product | Career | Growth
00:20:04 · Adam Mosseri, Lenny Rachitsky
[00:20:04] S0: But just think about the cost, think about what you pay for a model now and what the level of intelligence you're getting from that model is. And then at that same price point a year ago, what were you getting? At some point, the incremental value won't matter. We're getting there I think with small projects and programming. I think the models will matter even beyond ... This week you've got Fable and, obviously, Mythos from Anthropic, but I spent a lot of time with that this week. For the first time, I'm like, "Oh, I'm just talking to a much more technical, much smarter engineer than I am." The next version, a year out, of that model, do I need to pay for frontier tokens for whatever Anthropic model 6.0 is? Or is Fable just fine for all of my side projects? Probably just fine. Probably pretty cheap by then too. [00:21:05] S1: Yeah. When Kevin Weil on the podcast when he was CPO at OpenAI, he famously said, "This is the worst the model will ever be." It's still hard to comprehend that. It's only going to get better. So on this point of tokens, spend, ROI, and things like that, Meta is famous for this leaderboard of token spend. [00:21:25] S0: It's a terrible idea. No leaderboards for token spend. [00:21:28] S1: Okay. Okay. Talk about that, and just how do you think about just budgets for engineers and product teams at this point? Just spend as much as you want? Is it like there's a cap we have? Is there any sort of thing you've figured out that works well?
00:21:39 · Adam Mosseri
[00:21:39] S0: Right now, we've managed to get the costs reigned in a little bit by shutting down the silly things that we were doing. It's not that hard to build a token incinerator and that doesn't create a lot of value. And as soon as you actually look at the dollars in and value out, you might just be like, "Oh, that's just a bad idea." And so right now, we don't have token limits for our engineers. Actually, I think for anybody, really. I think that'll eventually have to happen, particularly if costs go up before they go down. I think they'll eventually go down for the reasons that we just talked about. But I think of it as any other resource, like I have to decide how to deploy capacity to my different teams, because I have a limited number of GPUs and CPUs and storage and RAM, et cetera. I have to decide how to deploy OPEX for labeling budgets across my teams. I have to decide how to deploy payroll for headcount across my teams. I think that you can imagine, at least in a year or two coming, that the burn rate of a strong engineer might be the same as their salary or their cost of employment. And in that world, you're going to probably need to put in some caps. The caps should probably be portional to the company's trust in your ability to use them in an ROA positive way. But I can imagine caps being healthy. Right now, we're not there. I think costs will go up because we'll just be using more tokens, not because prices will necessarily go up. But then I think prices will come down because all of these frontier models are going to be in a bit of a pricing war. So we'll see. I think it'll be a bit of a rollercoaster.
00:23:23 · Lenny Rachitsky
[00:23:23] S1: So coming back to this idea that, as you said, we've evolved from, we used to write all our code, to now we're approaching all code will be written by AI. And it feels like now the transition is it's not just written by AI, but it's one-shotted by AI. Coding now is steering AI, and it's like how often you have to correct it is coding now. So it's like the software development lifecycle slowly being eaten by AI. It'll start helping us come up with ideas, I imagine, more and more. The question I like to ask people is, where do you think human brains will continue to be most valuable as AI continues to eat more and more of that product development lifecycle?
00:24:05 · Adam Mosseri
[00:24:05] S0: Taste, like we talked about. Judgment, particularly around strategy. You might get feedback from an AI on a strategy, but you're not asking an AI to come up with a strategy anytime soon. Or if you are, then it's within the context of bounds you set. So here's my goal, here's my vision, here are my constraints, here's my job, here's my budget. I think that it looks more like management. You are trying to define what success looks like, decide how prescriptive you want to be about the path to success, and then giving feedback along the way. And that is its own craft. And it'll be interesting to see how if some of the same dynamics come up. I believe that if you are too prescriptive, as a leader with a team, you end up stifling good ideas. But if you're too open-ended, sometimes teams just waste time going in the wrong direction. And so that level of autonomy you give a team, maybe that applies to agents in the future, particularly when we're talking not just about building something, but deciding what you build in the first place. But I think of vision as an articulation of the world or the state of the product you want to get to. And I think of strategy as an opinionated path to achieve that vision. Strategy can't be like be the best or be amazing. It has to be controversial, that you have to be ... A reasonable person should be able to disagree with it, because otherwise you're probably just trying to compete on raw execution. And I think both vision and strategy I think are going to be where our brains are spreading more and more of our cycles, and I think less on execution.
00:25:56 · Lenny Rachitsky
[00:25:56] S1: Something I have always though is AI should be incredibly good at strategy because you would think, here's the market, here's all the information on the market, our competitors, our metrics, our numbers, our growth, all these things. Help me figure out how to win. You think AI, knowing all that, would be really good at this.
00:26:15 · Adam Mosseri
[00:26:15] S0: I think it could be. I have found it's not unless you steer it pretty aggressively. And I don't mean towards an answer, I mean based on the constraints. It turns out when you're trying to come up with a strategy, there's a lot of things to consider. You need to consider the state of the technology, personnel on the team and what's motivating them and what you can get. Sometimes coming up with an idea that is on the bubble, you know it's going to actually attract some of the best talent. And so the push then goes to the idea. Obviously, the competitive landscape, the regulatory landscape for companies as large as ours and the compliance landscape, the identity and reason to exist for the brand, you have to consider all of these things. I think if you ask an AI just for a strategy lazily, you're not going to get something great. You're going to get something pretty predictable that probably the competition would expect you to do. I think if you want a really more effective one, you need to think long and hard about what are all of the different inputs that need to be considered, make sure you steer the AI in a way that it's considering those as well. And then it needs to be a conversation in the back and forth. But I think if you're willing to put in the work and the time, it can definitely be helpful and definitely be clarifying, particularly if you tell it to be critical. Different models have very different vibes, though, on how willing they are to be pushed back, so I recommend picking one that likes pushing back.