Lenny's Podcast: Product | Career | Growth
00:38:43 · S1, Andrew Ambrosino
[00:38:43] S1: Micro-optimize. [00:38:44] S0: And they should. And people on Twitter like to remind us of this and I thank them for that. People should be focused on the features that exist and make them more reliable and better. But this is why we also have a culture of bottoms up exploration here. Because sometimes in the same way that Codex app came and disrupted ChatGPT in some way, this thing will get disrupted by a future effort. And that's part of the design is that. You can't always as one team be good at both the disruptive piece instead of maintaining a product and its quality piece. At some point, you have to design a process that allows for both. [00:39:18] S1: Kind of zooming out a little bit. If you think about the progression that we've been on of AI impacting how we build product, it's insane how far we've come from. As you said, we used to write all our code by hand, artisanally created human code. To AI writing 100% of our code, to you actually put it this way that now coding is steering the AI. And when you think about what percentage of my code is written by AI, it's almost like how many times did I have to steer it in the right direction as the AU version of coding? And now that there's agents and loops and all these things, what's kind of the latest frontier from what you've seen of how people are building? Is it loops? Is there something else of just the most AI-pilled, AI forward teams, here's how they operate now that people may not be aware of.
00:40:07 · Andrew Ambrosino
[00:40:07] S0: Yeah. I mean, loops are so last week, man. I mean, we talked about this. One of the big questions is always, "Well, how much of the product is AI written?" And it's always hard to answer that question because if you're using the goalposts from last year, it's like, "Well, 100% of our product right now is AI written code." So the question is more like, "Well, okay, fine. Is the code written supervised versus unsupervised?" And that's a totally different thing. I welcome the moving of goalposts because that means we're making product progress. There have been a lot of explorations here around autonomously developed software. A lot of harness engineering stuff, a lot of different explorations on like, "Okay, well, what if you came in overnight and did garbage collection of the code base to clean it up?" One thing that I think all models suffer with right now is just they usually increase complexity. If research is listening at any company, please make the models better at deleting code. But that becomes a problem right now when you try to put development completely on autopilot and it's both on the human side and the code base side. So feature requests. How do you teach a model which features to build, which ones to ignore, which ones to group together and reframe a little bit? How do you teach to model how to build the right abstractions? All of this is getting better. I don't think we're at place yet where we're like, "We're just going to set up a loop that's like improve the app." And listens to Twitter and listens to Slack and listens to email. We're not there yet, but we're trying to make it happen.
00:41:49 · S1, Andrew Ambrosino
[00:41:49] S1: Do you think we'll get there? Do you think we'll get to a place where it's just like grow, win? [00:41:54] S0: Slash goal, make money, make me a billion dollars. [00:41:58] S1: Win the market. [00:41:58] S0: I don't know, man. I am not in the business of saying never or always or whatever. [00:42:05] S1: Yeah. How are you using AI in your work as product leader, eng leader? [00:42:10] S0: Yeah. [00:42:11] S1: What are some ways that you use it that maybe people may not be aware they can use AI for?
00:42:15 · Andrew Ambrosino
[00:42:15] S0: Yeah, I think I have the best job in the world right now. But one of the things that makes it very fun is that when we were developing the original Codex app, the goal for me personally was to make it the thing that I wrote the code with. I was like, "I need to make this so good at development that I can build the Codex app with this." And the Codex app at that time was a development tool. And we did that super quick dog fooding loop because you've got your personal dog fooding loop. Where you're like, "Oh, I can't do this thing. I should fix that so that I can do the thing. Now I can do the thing, now I can do more things." We released that. And then the next challenge was, "Hey, people are starting to do some different shaped things with this. And now I need to grow this and so I need to hire a few people and help." So then my role changed at the same time that the role of the app needed to change. I'm like, "Okay, I need to do more product discovery here. I need to figure out the right loops for seeing what everybody's working on and steering things that are off track." And so all of a sudden that's what I started using the Codex app for. I did still write code. I've tried to align my own usage of it with the problem that we're trying to solve. And now I'm like, "I need to build a spreadsheet that models this out." I need to do internal deep research on all of the efforts that have gone into this area of research for the next version of this. There was a release or a series of releases in May-ish that introduced the in-app browser, computer use, and artifact creation to the Codex app. That was, I think our Codex is for almost everyone release. And everybody knows the term vibe coding. I think that was our first vibe coordinated release. Where I had a Notion Doc somewhere with everything that needed to happen. And I was automating, going out to gather updates from pull requests, from Slack channels, and updating the status tracker. And now this is pretty commonplace, but at the time I felt like I was at the bleeding edge of how to manage a product release. In short, the way that I use the Codex app is basically like, what has my job grown into and how do I make this thing able to do everything I need to do? I will get up in the morning, I will see the daily brief that I have from everything from the 3,000 Slack channels that I'm in.
00:44:39 · Andrew Ambrosino, S1
Which things need my attention. I can kind of message back and be like, "All right, give me five questions and I'll answer them." And I can do that. [00:44:49] S1: How do you set that up? What's the workflow for somebody to set that up? Yes, that sounds amazing. [00:44:53] S0: Again, I think we're still in the discovery phase on a lot of this. And so right now it's like I'm making an automation that says, or scheduled task that says like, "Go through my Slack channels, these are the things that I care about and think are most important." So I'm still kind of defining that. These are things to watch out for different categories, here's some context. And that'll get set up as an automated task and then first few times it runs it might need some steering. And luckily with this app, I don't have to find out how to edit the instructions. I can just be like, "Hey, next time this runs, can you please worry about this instead? Or can you de-emphasize this work stream? Or, hey, this thing happened and it didn't come up in a brief, can you make sure that stuff in this shape?" So I can kind of coach it along the way. It'll update the way that it notifies me, stuff like that. [00:45:44] S1: Amazing. [00:45:45] S0: I think in the future though, this has been a core problem with the chatbot shape, is that I know how to set this up. I have time to set this up because for me, it's product discovery to set it up. But if you're not working at OpenAI, not developing this, you don't want to have to figure out all this stuff. We need to figure out that shape of things.
00:45:45 · Andrew Ambrosino, S1
[00:45:45] S0: I think in the future though, this has been a core problem with the chatbot shape, is that I know how to set this up. I have time to set this up because for me, it's product discovery to set it up. But if you're not working at OpenAI, not developing this, you don't want to have to figure out all this stuff. We need to figure out that shape of things. [00:46:06] S1: Yeah. What I'm hearing here is I don't think people realize that your app can act a lot like OpenClaw, people were so excited about. You just talk to it, set up this thing, check on this thing for me every day, and then tell me what's going on. [00:46:19] S0: Exactly, yeah. [00:46:20] S1: It's starting to become a part of all the products, which is amazing. So the way somebody would set this up is they just talk within the app and say, "I want to set up an automation to do this. Look at my Slack and these are the things I want to do." [00:46:20] S0: Yeah. [00:46:32] S1: Great. [00:46:32] S0: Yeah. And the app will say it'll set it up for you. If it doesn't have a Slack connector, it'll say, "Can I add the Slack connector? Yes or no?" You can hit yes to that. The least that we can do is make it so that if you don't know how to do something in the app, that you can just ask it. I don't think that's enough, but I think that's the least we can do.