Venture with Grace
00:07:46 · Vitaly Gordon
[00:07:46] S1: Yeah. So look. This is something that obviously has been the journey since I started this company seven years ago where, like, it was pre GitHub Copilot and Codex and, you know, Claude Code and all that stuff. So improving engineering productivity was kind of like a relatively slow and laborious process that required a lot of change management. And now AI changed all of it. Like, honestly, there is no engineering productivity today that does not include AI. It's just like it's primarily, kind of how to use AI. And... But the good thing is, like, the results are now, like, massively, massively different, and we truly see, you know, like, these, you know, kind of proverbial 10x engineers that are, like, you know, orchestrating agents and outputting, like, so much more. Now, you know, we can talk about the quality of that. But, like... But that aside is, like, there's no doubt about that this technology is probably the most revolutionary technology that we ever had for in kind of engineering. And now having said that, I I think that some people are still, like, you know, just not, kind of thinking about it the right way because in the vast, vast majority of the companies, their AI spend is still less than 5% their labor spend, like the salary cost and kind of all of that stuff. I think, you know, Anthropic and S1 just reported that since, you know, like the beginning of the year, actually, their cloud code spend is more than their salary spend. So if you take a page from Anthropic, like, you know, like, again, as a percentage of, you know, engineering spend, maybe your AI spend should be over a 100%. Right? That's gonna look at least... That's what Anthropic can report. And so, like, there's a lot of talk about, like, you know, token maximum, which is also, you know, incorrect, because just like, you know, spending any kind of tokens in AI really does not yield a result. So the thing that we are kind of advocating for is outcome axing, which is really the thing that probably most organizations should be optimizing for, which AI is amazing at, is really lowering the price per outcome. Right? And by lowering the price per outcome, more often than not actually involves using more AI and not less AI.
00:09:57 · Grace Gong
[00:09:57] S0: You guys have... Like, so basically, you fund the engineering team with, like, the best pre AI practices, strong dura metrics. So I guess for a startup, how do you evaluate if your token spend or if you are spending the things into the right things. Then for the engineers, so I guess, like, how do you measure each person's like productivity? Because some of them are like defense, some are office slash, like, you know, people are, some are like writing code, some are like kind of evaluating the stuff. And like, nowadays, like people are also using different toolings on like different things. Like how do you picture where does your company sit in this kind of like, or like how do you guys evaluate like if something is more effective if you just bought in versus like buying the support?
00:10:48 · Vitaly Gordon
[00:10:48] S1: Yeah, look, so it's a, you know, it's a really hard time right now. Because here's the thing, the baseline just have moved so far up, you know, compared to before. So while, you know, on one hand, we can, you know, sit here and argue and, like, you know, taste is, you know, more important and, like, you... Which is true, by the way, but you still need to execute to kind of, like, at a level that is just will make, you know, like, what engineering team could produce five years ago seems laugh... Laughable. Right? So there is, like, if you're not producing, at a certain level, now we can, you know, debate and kind of industry by industry dependent, and we can, you know, share some of these numbers. But your competitors are probably, like, out executing you. That's really kind of, like, at least the way we go about thinking. And so if you're not also executing, so, you know, there's a thinking about ways, like, what used to take a year of development should now take two months. Right? Like, that could be, like, one kind of rule of thumb if, like, if you wanna be, you know, five times five times faster. Now it doesn't mean that, like, you should just, like, create, you know, massive, massive amounts of SLAB because it's actually pretty easy. You still need a, you know, you you still need a product that people will buy and solve, you know, customer problem and, you know, all of that stuff. However, you also need to kind of, you know, move just much faster. And so if you're not not feeling that, like... So I'll just give you kind of a rule of thumb. I think that if you still have, let's say, a ratio for, like, one one PM to, eight engineers, like... And the PM is not running out of ideas and because engineering is executing so fast, you're probably doing something wrong.
00:12:34 · Grace Gong
[00:12:34] S0: There's a term called code churn. Basically, it's code that's being written and deleted. In So your data, it's up to like 861%. And how is that kind of come together? How do you evaluate on like, how do people basically prevent this from happening or out of
00:12:59 · Vitaly Gordon
[00:12:59] S1: this? Yeah. This is kind of, you know, something that we also used to show that... Look, the... Just looking at how much code is generated is just a stupid thing because most of it ends up going nowhere. And so, you know, one analogy I use is when, you know, if you put your car in neutral, you can, you know, press the gas pedal and, you know, the RPM, know, will go bananas, right? Like, it's very easy to increase the RPM. What you actually care about mostly when you're driving a car is just, you know, is the distance traveled. Right? Like, that's really what here. And so kind of this, like, you know, lines of code, and this is what we show is, like, even though you produce a lot of it, at the end of the day, you care about is features in production. Right? It's something that the customer can enjoy. If like, if your work does not reach production, then, you know, it's basically... These are just wasted tokens. Right? And so or, you know... And unfortunately, what we do see with our data is, like, like, a huge, huge percentage of the code that is being generated is being generated and, you know, is basically being kept on people's laptop, whatever happens to it, but it actually... It does not reach, kind of production, as much. And this is really where, people should not just, like, pat themselves on the back, look, you know, how I orchestrate five agents that are doing a whole bunch of things. But if the customer doesn't see see it, it basically doesn't count.
00:14:31 · Grace Gong, Vitaly Gordon
[00:14:31] S0: You guys have the newest report called The Speed Trap. Can you explain to, let's say, the non technical founder, what is a trap and who is caught in it? [00:14:43] S1: Yeah. So the speed trap, it just goes to say that as you kind of increase, you know, velocity and throughput for the system, you like, you kind of push the can down the road, right? It's not like like this whole thing does not come free because and again, to use, you know, one of the analogy we use about from the Industrial Revolution is like, you know, putting a steam engine in an, you know, arson, like, know, woods kind of crafting shop is just like not gonna make things, you know, run faster. It's just gonna, you know, cause a lot of chips to fly. And so the thing is, like, you have to also work on the rest on the rest of your pipeline. You have to think about the code reviews. You have to think about your CICD. You have to, like, again, Anthropic just, you know, recently released that they were, you know, pushing code, you know, so fast that, like, they broke GitHub pretty much. Right? And so they they have to kinda go and build another system. Like, you have to really think about the end to end process. And, like, it's great that we, you know, sold one bottleneck, but, you know, as ball... Bottlenecks don't disappear, they just move. Right? And so what we talk about in the report is where kind of the bottleneck shifted to and what can you do about it?