Lenny's Podcast: Product | Career | Growth
00:25:32 · Lenny Rachitsky, Boris Cherny
[00:25:32] S1: This idea of under-funding, it's so interesting, because, in general, there's this feeling like AI is going to allow you to not have as many employees, not have as many engineers. And so, it's not only you can be more productive. What you're saying is that you will actually do better, if you under-fund. It's not just that AI can make you faster, it's you will get more out of the AI tooling if you have fewer people working on something. [00:25:54] S0: Yeah. If you hire great engineers, they'll figure out how to do it. And, especially, if you empower them to do it. This is something I actually talk a lot about with CTOs and all sorts of companies. My advice generally is, "Don't try to optimize. Don't try to cost cut at the beginning. Start by just giving engineers as many tokens as possible." And now you're starting to see companies ... Like, at Anthropic, we have ... Everyone can use a lot of tokens. We're starting to see this come up as a perk at some companies where if you join you get unlimited tokens. This is a thing I very much encourage, because it makes people free to try these ideas that would have been too crazy, and then if there's an idea that works, then you can figure out how to scale it, and that's the point to optimize, and to cost-cut, figure out ... Maybe you can do it with a haiku, or with Sonnet instead of Opus, or whatever. But at the beginning, you just want to throw a lot of tokens at it, and see if the idea works, and give engineers the freedom to do that.
00:25:54 · Boris Cherny, Lenny Rachitsky
[00:25:54] S0: Yeah. If you hire great engineers, they'll figure out how to do it. And, especially, if you empower them to do it. This is something I actually talk a lot about with CTOs and all sorts of companies. My advice generally is, "Don't try to optimize. Don't try to cost cut at the beginning. Start by just giving engineers as many tokens as possible." And now you're starting to see companies ... Like, at Anthropic, we have ... Everyone can use a lot of tokens. We're starting to see this come up as a perk at some companies where if you join you get unlimited tokens. This is a thing I very much encourage, because it makes people free to try these ideas that would have been too crazy, and then if there's an idea that works, then you can figure out how to scale it, and that's the point to optimize, and to cost-cut, figure out ... Maybe you can do it with a haiku, or with Sonnet instead of Opus, or whatever. But at the beginning, you just want to throw a lot of tokens at it, and see if the idea works, and give engineers the freedom to do that. [00:26:49] S1: So, the advice here is just be loose with your tokens, with the cost on using these models. People hearing this may be like, "Of course. He works at Anthropic. You want us to use as many tokens as possible." But what you're saying here is the most interesting innovative ideas will come out of someone just taking it to the max and seeing what's possible.
00:26:49 · Lenny Rachitsky, Boris Cherny
[00:26:49] S1: So, the advice here is just be loose with your tokens, with the cost on using these models. People hearing this may be like, "Of course. He works at Anthropic. You want us to use as many tokens as possible." But what you're saying here is the most interesting innovative ideas will come out of someone just taking it to the max and seeing what's possible. [00:27:08] S0: Yeah. And I think the reality is at small-scale, you're not going to get a giant bill for anything like this. If it's an individual engineer experimenting, the token cost is still probably relatively low relative to [inaudible 00:27:21] other costs of running the business. So, it's actually not a huge cost. As the thing scales up, so, let's say they build something awesome, and then it takes a huge amount of tokens, and then the cost becomes pretty big, that's the point at which you want to optimize it. But don't do that too early. [00:27:37] S1: Have you seen companies where their token cost is higher than their salary? Is that a trend you think we're going to find and see? [00:27:44] S0: At Anthropic, we're starting to see some engineers that are spending hundreds of thousands a month in tokens. So, we're starting to see this a little bit. There's some companies that we're starting to see similar things. Yeah. [00:27:58] S1: Going back to coding, do you miss writing code? Is it something you're sad about that this is no longer a thing you'll do as a software engineer?
00:27:58 · Lenny Rachitsky, Boris Cherny
[00:27:58] S1: Going back to coding, do you miss writing code? Is it something you're sad about that this is no longer a thing you'll do as a software engineer? [00:28:06] S0: It's funny. For me, when I learned engineering, for me, it was very practical. I learned engineering, so, I could build stuff. And, for me, I was self-taught. I studied economics in school, but I didn't study CS. But I taught myself engineering. Early on, I was programming in middle school. And from the very beginning, it was very practical. So, I learned to code, so, that I can cheat on a math test. That was the first thing- [00:28:33] S1: Nice. [00:28:33] S0: ... we had these graphing calculators, and I just programmed- [00:28:36] S1: The T83. TI83? [00:28:39] S0: Yeah. T83+. Yeah. Yeah. Exactly. [00:28:40] S1: Plus. [00:28:41] S0: Plus. Yeah. I programmed the answers in, and then the next math test, or whatever, the next year, it was just too hard. I couldn't program all the answers in, because I didn't know what the questions were. And so, I had to write a little solver, so, that it was a program that would just solve these algebra questions or whatever. And then I figured out you can get a little cable, you can give the program to the rest of the class, and then the whole class gets As, but then we all got caught, and the teacher told us to knock it off. But from the very beginning- [00:29:08] S1: Wow.
00:29:08 · Lenny Rachitsky, Boris Cherny
[00:29:08] S1: Wow. [00:29:08] S0: ... it's always just been very practical for me where programming is a way to build the thing. It's not the end in itself. At some point, I personally fell into the rabbit hole of the beauty of programming. So, I wrote a book about typescript. Actually, at the time, it was the world's biggest typescript [inaudible 00:29:29] just because I fell in love with the language itself. And I got deep into functional programming and all the stuff. I think a lot of coders they get distracted by this. For me, it was always ... There is a beauty to programming, and, especially, to functional programming. There's a beauty to type systems. There's a certain, like, this buzz that you get when you solve a really [inaudible 00:29:54] math problem. It's similar when you balance the types or the program is just really beautiful. But it's really not the end of it. I think, for me, coding is very much a tool, and it's a way to do things. That said, not everyone feels this way. So, for example, there's one engineer on the team, Lena, who was still writing C++ on the weekends by hand, because for her she just really enjoys writing C++ by hand. And so, everyone is different. And I think even as this field changes, even as everything changes, there is always space to do this. There is always space to enjoy the art, and to do things by hand, if you want. [00:30:34] S1: Do you worry about your skills atrophying as an engineer? Is that something you worry about or is it just like, "This is just the way it's going to go"?
00:30:41 · Boris Cherny
[00:30:41] S0: I think it's just the way that it happens. I don't worry about it too much personally. I think, for me, programming is on a continuum. And way back in the day, software actually is relatively new. Right? If you look at the way programs are written today using software that's running on a virtual machine or something, this has been the way that we've been writing programs since probably the 1960s. So, it's been 60 years or something like that. Before that it was punch cards, before that it was switches. Before that, it was hardware. And before that, it was just, like, literally, pen and paper. It was a room full of people that were doing math on paper. And so, programming has always changed in this way. In some ways, you still want to understand the layer under the layer, because it helps you be a better engineer. And I think this will be the case maybe for the next year or so, but I think pretty soon it just won't really matter. It's just going to be the assembly code running under the program or something like this. At an emotional level, I feel like I've always had to learn new things. And as a programmer, it doesn't feel that new, because there's always new frameworks. There's always new languages. It's just something that we're quite comfortable with in the field. But at the same time, this isn't true for everyone. And I think for some people they're going to feel a greater sense of, I don't know, maybe, like, loss or nostalgia, or atrophy, or something like this.