Venture with Grace
00:09:57 · Grace Gong
[00:09:57] S0: You guys have... Like, so basically, you fund the engineering team with, like, the best pre AI practices, strong dura metrics. So I guess for a startup, how do you evaluate if your token spend or if you are spending the things into the right things. Then for the engineers, so I guess, like, how do you measure each person's like productivity? Because some of them are like defense, some are office slash, like, you know, people are, some are like writing code, some are like kind of evaluating the stuff. And like, nowadays, like people are also using different toolings on like different things. Like how do you picture where does your company sit in this kind of like, or like how do you guys evaluate like if something is more effective if you just bought in versus like buying the support?
00:10:48 · Vitaly Gordon
[00:10:48] S1: Yeah, look, so it's a, you know, it's a really hard time right now. Because here's the thing, the baseline just have moved so far up, you know, compared to before. So while, you know, on one hand, we can, you know, sit here and argue and, like, you know, taste is, you know, more important and, like, you... Which is true, by the way, but you still need to execute to kind of, like, at a level that is just will make, you know, like, what engineering team could produce five years ago seems laugh... Laughable. Right? So there is, like, if you're not producing, at a certain level, now we can, you know, debate and kind of industry by industry dependent, and we can, you know, share some of these numbers. But your competitors are probably, like, out executing you. That's really kind of, like, at least the way we go about thinking. And so if you're not also executing, so, you know, there's a thinking about ways, like, what used to take a year of development should now take two months. Right? Like, that could be, like, one kind of rule of thumb if, like, if you wanna be, you know, five times five times faster. Now it doesn't mean that, like, you should just, like, create, you know, massive, massive amounts of SLAB because it's actually pretty easy. You still need a, you know, you you still need a product that people will buy and solve, you know, customer problem and, you know, all of that stuff. However, you also need to kind of, you know, move just much faster. And so if you're not not feeling that, like... So I'll just give you kind of a rule of thumb. I think that if you still have, let's say, a ratio for, like, one one PM to, eight engineers, like... And the PM is not running out of ideas and because engineering is executing so fast, you're probably doing something wrong.
00:12:34 · Grace Gong
[00:12:34] S0: There's a term called code churn. Basically, it's code that's being written and deleted. In So your data, it's up to like 861%. And how is that kind of come together? How do you evaluate on like, how do people basically prevent this from happening or out of
00:12:59 · Vitaly Gordon
[00:12:59] S1: this? Yeah. This is kind of, you know, something that we also used to show that... Look, the... Just looking at how much code is generated is just a stupid thing because most of it ends up going nowhere. And so, you know, one analogy I use is when, you know, if you put your car in neutral, you can, you know, press the gas pedal and, you know, the RPM, know, will go bananas, right? Like, it's very easy to increase the RPM. What you actually care about mostly when you're driving a car is just, you know, is the distance traveled. Right? Like, that's really what here. And so kind of this, like, you know, lines of code, and this is what we show is, like, even though you produce a lot of it, at the end of the day, you care about is features in production. Right? It's something that the customer can enjoy. If like, if your work does not reach production, then, you know, it's basically... These are just wasted tokens. Right? And so or, you know... And unfortunately, what we do see with our data is, like, like, a huge, huge percentage of the code that is being generated is being generated and, you know, is basically being kept on people's laptop, whatever happens to it, but it actually... It does not reach, kind of production, as much. And this is really where, people should not just, like, pat themselves on the back, look, you know, how I orchestrate five agents that are doing a whole bunch of things. But if the customer doesn't see see it, it basically doesn't count.
00:14:31 · Grace Gong, Vitaly Gordon
[00:14:31] S0: You guys have the newest report called The Speed Trap. Can you explain to, let's say, the non technical founder, what is a trap and who is caught in it? [00:14:43] S1: Yeah. So the speed trap, it just goes to say that as you kind of increase, you know, velocity and throughput for the system, you like, you kind of push the can down the road, right? It's not like like this whole thing does not come free because and again, to use, you know, one of the analogy we use about from the Industrial Revolution is like, you know, putting a steam engine in an, you know, arson, like, know, woods kind of crafting shop is just like not gonna make things, you know, run faster. It's just gonna, you know, cause a lot of chips to fly. And so the thing is, like, you have to also work on the rest on the rest of your pipeline. You have to think about the code reviews. You have to think about your CICD. You have to, like, again, Anthropic just, you know, recently released that they were, you know, pushing code, you know, so fast that, like, they broke GitHub pretty much. Right? And so they they have to kinda go and build another system. Like, you have to really think about the end to end process. And, like, it's great that we, you know, sold one bottleneck, but, you know, as ball... Bottlenecks don't disappear, they just move. Right? And so what we talk about in the report is where kind of the bottleneck shifted to and what can you do about it?
00:14:43 · Vitaly Gordon, Grace Gong
[00:14:43] S1: Yeah. So the speed trap, it just goes to say that as you kind of increase, you know, velocity and throughput for the system, you like, you kind of push the can down the road, right? It's not like like this whole thing does not come free because and again, to use, you know, one of the analogy we use about from the Industrial Revolution is like, you know, putting a steam engine in an, you know, arson, like, know, woods kind of crafting shop is just like not gonna make things, you know, run faster. It's just gonna, you know, cause a lot of chips to fly. And so the thing is, like, you have to also work on the rest on the rest of your pipeline. You have to think about the code reviews. You have to think about your CICD. You have to, like, again, Anthropic just, you know, recently released that they were, you know, pushing code, you know, so fast that, like, they broke GitHub pretty much. Right? And so they they have to kinda go and build another system. Like, you have to really think about the end to end process. And, like, it's great that we, you know, sold one bottleneck, but, you know, as ball... Bottlenecks don't disappear, they just move. Right? And so what we talk about in the report is where kind of the bottleneck shifted to and what can you do about it? [00:15:58] S0: The time in QA is up to 300%. And it is like, quote unquote, the worst number in your whole dataset. So what is the bottleneck that AI can't write code or human can verify it fast enough?