Lenny's Podcast: Product | Career | Growth
00:33:28 · Ian Silber
[00:33:28] S1: I think that 100% we do work that way. It depends. I think for us, it depends on what stage you're at in the process or maybe how far along the product is. There are certain features for ChatGPT where we really obsess and we really spend a lot of time. We try a hundred things and we throw out 99 and we finally ship one. Or the composer is changing every single day and we try many things and we throw them out. And you could take that for many different things within ChatGPT. And then there are other things like Codex where, I don't know, I mean a super app is a good example where we definitely are just trying stuff really quick. I think that we really embrace building in public, which is a different way of working. But I think that when you get comfortable with it is actually really gratifying or satisfying because you can try stuff and get instant feedback both internally and externally. But yeah, I mean it is moving really fast. And I think it just depends. Like I said, certain things we really sweat the details and we try and we do test, we A/B test, we'll do user research and we kind of iterate and evolve a lot. And then other things, we take big swings and then we learn quickly. But yeah, I don't know. I think the biggest thing is embracing that the technology might change in a month and maybe a certain interaction or setting or whatever that you needed doesn't need to exist or now a new one does need to exist or a new capability is there or the latency profile has changed such that you can rethink this or whatever. All these capabilities are changing or the intelligence has changed. And so you have to be just really quick on your feet and maybe not be so precious about it until it's at the point where, okay, this is kind of hardened. We don't think that this is necessarily going to change tomorrow because of the technology, so let's make this the best it can possibly be. We're thinking a lot about how we should evolve. Really think about ChatGPT becoming deeply embedded in your personal life and how should the app feel? That's something that we're not going to necessarily design and ship overnight and just put it out there. We're really thinking about that and we're trying a bunch of different ideas and we're putting it up on a whiteboard or we're doing some design in Figma. We're prototyping it.
00:36:23 · Ian Silber, Lenny Rachitsky
We're probably going to dog food a bunch of different ideas long before we ever ship something. But that's a bigger project. And then there's other things where we're like, idea to ship in four hours. [00:36:38] S0: This all makes sense. Essentially what I'm hearing is you pick your battles and you focus the design effort on the things that will be durable and not change.
00:36:47 · Ian Silber
[00:36:47] S1: Exactly. Yeah, yeah. And it's hard to know what that is. Yeah. That's a great question, and it's a really fun part of the way we work. I think that we have vastly different audiences. Really, really different. I've never worked in this kind of quite a spread. At Instagram, I spent eight years working at Instagram. We had a billion users or billions of users maybe. I don't actually know what they're at now, but it was a lot of people. But the product was pretty simple and it was sort of a couple ways that you could use it. We did think about creators and larger accounts that had special needs, but this is vastly different where you have someone who's using this to help them come up with a recipe to eat tonight or brainstorm their next haircut or something like that. And then there's somebody who's transforming the way they're doing farming for... helping to automate their farm in Japan or something. And it's just the broadest spectrum ever. Generally what we've seen is that we believe in this idea of this is sort of capability overhang where the product, vast majority of the people are getting a sliver of the true value that they could out of the model. That's not necessarily a bad thing. They're still getting tons of value. But we found that we make sure to focus on creating an extremely simple, streamlined experience for many, many people. And then we put all of the cutting edge stuff. We also make sure to give that and make sure that's really accessible for the people that care. The challenge of course is not necessarily over-indexing and putting something that people might not understand in front of a billion users. But generally what we've seen is we can experiment early with the people that really care. And that's where we have our desktop app and we have Codex. And now ChatGPT Work is stepping into a much broader audience, which we're really excited about. And eventually what we know, and we think about this a lot in design is things will truly become for billions of users when they don't have to think about a switch or a mode or anything like that. And so we know that eventually it has to kind of get distilled into that main experience. But it is a really fun thing to balance. And we have to think about people that are using this for free every day to people that are paying us hundreds of dollars and lots of different needs.
00:41:15 · Ian Silber, Lenny Rachitsky
It's just a matter of balancing. And as a designer, it does add a lot of complexity for sure to think... You really have to have empathy for every sort of different type of user, but it makes it fun for sure. [00:41:39] S0: That is such a mind-blowing point you're making about the spectrum of usage of ChatGPT and Codex. I didn't think about it that way. Has there been a product that has such a wide spectrum from just, "What's this rash on my finger," to building a Salesforce?
00:41:39 · Lenny Rachitsky, Ian Silber
[00:41:39] S0: That is such a mind-blowing point you're making about the spectrum of usage of ChatGPT and Codex. I didn't think about it that way. Has there been a product that has such a wide spectrum from just, "What's this rash on my finger," to building a Salesforce? [00:41:57] S1: Yeah. And it makes it really hard too. It's so funny, one of the big challenges we're always thinking about is this blank box problem. And people talk about this all the time. It's like, is chat really the final frontier? It's a glorified terminal. It is some critique that I think these chatbots get. And then it's like, okay, so how do you design something that can kind of shape-shift into anything? It's like maybe someone is coming here for image generation for fun or they're trying to generate a PowerPoint. So having this adaptable interface is a really interesting design challenge. And it's sort of uncharted territory because it can really shape-shift into many different tools. And it's just incredible at so many different things and trying to figure out how to present that. And then thinking about, we think a lot about a novice user, someone who's sort of brand new to this experience, that we want to help them understand how this can add value to their lives or to their professional work or whatever capacity they're using this in. And then we also want to get out of the way of people that use this every single day and they use this as a real workhorse tool. And so designing for that spectrum is something we have to think about a lot. But yeah, it's all a world of spectrums, I guess.
00:43:31 · Lenny Rachitsky
[00:43:31] S0: There's a comment that Kevin Weil, your former chief product officer, made on the podcast that has really stuck with me, which is to your point, people always diss this idea of just like, it's just like a chatbot. Why hasn't it ever changed from that? Even though it has, but still it's kind of this chatbot. And he made this really good point that what is an interface that is very good at spanning a wide spectrum of intelligence? It's actually just talking. If you think about humans, we talk to people of low IQ to very high IQ. We can all just talk to each other and it's actually a really good interface for packaging something that is increasingly smarter and smarter and smarter.