Lenny's Podcast: Product | Career | Growth
00:26:44 · Tara Seshan
[00:26:44] S0: Yeah, I'm trying to think if there's other good cultural memes. Certainly a really important one is feeling the AGI or just being conscious of AGI coming. There are so many outcomes for what it could look like or how one thinks about it, but a huge part of what puts most people at this company is believing in that mission of AGI being beneficial and trying to do whatever it takes to make that possible, both realization of AGI and ensuring that it is beneficial for humanity. And in building products, another just constant refrain I have to keep in the back of my mind is, are we building for where the models are going to be in two to three months? You fail if you build for where the models are now, you fail if you build for where you think the models will be in a year. Both outcomes are equally wrong, and I'm sure many people have talked about this, but both outcomes are really equally wrong. If you're too early, you're wrong. If you build something that was overly focused on a past model's capabilities, you're entirely wrong. The only way to build is two to three months. And having this meme of models are going to get way better. I need to think about the model capability as the center of this product. I need to get out of the way of the model in terms of the product constructs that I create. How do I ensure that this is right for the model in two to three months' time?
00:28:05 · Lenny Rachitsky, Tara Seshan
[00:28:05] S1: How do you know what two or three months is like? It's a challenging understanding, especially while we're on this exponential. Is it just a gut feeling? Is there anything the researchers give you a sense? How does that work? [00:28:17] S0: Yeah, certainly communicating really tightly with research on where they think things are going is incredibly important. These things aren't entirely a black box in that you know, "Hey, we're focused on these particular things. We would like models to be better at coding in these specific ways," or, "better at writing in these specific ways." So we certainly have focused efforts on making the model better at specific capabilities. And so knowing where that is and ensuring that product development is as tied as possible to what research has as its agenda and its roadmap is really important. [00:28:54] S1: A quote that I'll never forget is when Kevin Weil was on the podcast, he was chief product officer at that time. He said that this is the worst the models will ever be. And it sounds so simple, but it's hard to just wrap your head around that, that this is the worst there will ever be. It's such a cliche almost now to say that, but it's true. It's absurd. This is- [00:29:16] S0: Yeah, it's absurd. It's truly absurd. [00:29:21] S1: Oh man. Okay. I want to talk about ChatGPT, the app, briefly. Okay, so I have it open right now. [00:29:28] S0: Yes.
00:29:28 · Tara Seshan, Lenny Rachitsky
[00:29:28] S0: Yes. [00:29:30] S1: Okay. So here's what I see in it. ChatGPT, and then there's a dropdown and there's ChatGPT and Codex, and then there's this toggle, Chat and Work. Tara, what is going on? What are all these things? Help us understand what each of these things are for. And where do you think this goes? Is it going to stay like this? Is there a next step that you're imagining already? [00:29:50] S0: Our north star here is that users do not need to make decisions between picking between all these different options. Ideally, there is no toggle here. That you go to the box, you type in your task, like, I would like to build a really awesome app that, I don't know, helps my podcast guests do research before episodes or something like that. [00:29:50] S1: Yes. Do that.
00:30:10 · Tara Seshan
[00:30:10] S0: And it will just pick the right harness, it'll pick the right model for you to be able to get that thing done. Ideally, the choice here is not on our users to have to pick between all these different concepts and understand not only what are they trying to do, but understand the limitations and capabilities of our products. So that is certainly where we want to go. In the near term, picking between ChatGPT and Codex is really a choice for, do you want to stay in more development oriented UI or do you want to have the same power and capabilities in the ChatGPT mode? And so if you're a Codex user, keep using Codex, you're not missing out on anything, continue using it as much as possible. But if you're a ChatGPT user who is like, what are these new agentic capabilities? You should probably be in ChatGPT mode. And then when you're in ChatGPT, if you want to have conversations, if you want to search, that's where chat mode is the right thing. It's the same chat mode you know and love with better and better models and newer and newer capabilities every time. But in Work mode, that's where under the covers, this is Codex. We've removed some of the coding UI. You're not going to see a work tree pop up all of a sudden in Work mode, but it is the same power to get things done, to, for example, generate a really complex financial model. That's all possible in Work mode. And we see people, especially I mentioned our corporate finance team, use Work mode to do incredible, incredible things that were previously either manual or required deep expertise from one person on the team, become things that the whole team can be able to execute or just elevate the ambitions of everyone on the team in terms of timeline or capabilities or frontier of what they can get done.
00:31:58 · Lenny Rachitsky
[00:31:58] S1: Okay, that's really helpful. So there's these three modes currently. There's the engineering mode, the chat mode, and then the do knowledge Work mode. And the knowledge Work mode, it's actually Codex doing all that work, but people may not know what Codex is, may be afraid of it. Is there anything in that Work mode that's not just Codex? Because that's actually really interesting. Is there additional harness tweaks to make it feel a little different or is it just the same thing with a little different UI?
00:32:23 · Tara Seshan
[00:32:23] S0: It's really at the UI level. So Work mode and Codex mode, if you go to Codex and ask it to generate an amazing financial model to price your product or something like that or predict my revenue for the next six months or something like that, Codex will do as good a job as Work mode. It's really about whilst it's doing so, what kind of UI do you want to see in the chain of thought? What kind of technical detail do you want exposed to you? It's incredible. It's similarly powerful. And so Codex users aren't missing out on anything by not switching modes. In fact, we do not want them to. Stay in Codex and do all the stuff you want to do in Codex, and we will show you the appropriate UI based on the things you asked for. Truly our north star is to merge all these things so that users don't have to make any of these decisions. The separation is really more about how can we meet people where they are as much as possible in terms of the products that they use, in terms of their familiarity with concepts and make sure that we are enabling everyone to take advantage of working with agents, which has transformed entirely the way every single developer works. We should do the same thing with knowledge work.