The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:25:15 · Mike Cannon-Brookes
[00:25:15] S0: I guess that depends if you continue to fund them and believe they'll be funded forever. It seems unlikely given the history of technology, but maybe. I think cost is really important, but it's not just because of that. I think if you look at a lot of examples out there, it's very easy to use a very expensive model to get pretty good results pretty fast. I think using a cheaper model or an older model and getting good at applying that to problem sets is a real advantage for most technologies and prices and costs. I think customers that that we spoke to, so large enterprises around the world, they're incredibly sensitive to this type of stuff nowadays. They really, really care about the price performance. They are increasingly doing so. We have a whole raft of products now that will show you your token spend either across the board in terms of your talent, either in in DX, in your engineering organization, right, understanding where you use which model, where you're getting value, which features you're using it for, where you should change to optimize your spending because this is a bigger and bigger budget item, and they see it's only gonna be a bigger budget item, and so they have to care about the spending. Now that I don't think is any different for them buying a generic model or a foundational model of any of the big three or four or five as it is buying a product where their costs are gonna be baked into whatever that product they're buying is. So I think that cost discipline is going to arrive continually more and more. And I think it's a strength of ours, for example, in how we are able to use our FinOps strength to continue to reduce the price of features that are used heavily. I think that is a core strength and attribute of a great technology company will be its ability to manage technological cost.
00:26:58 · Harry Stebbings, Mike Cannon-Brookes
[00:26:58] S1: And that'll be rooted in intelligent model routing on a per task basis to optimize cost. [00:27:03] S0: Model routing, model blending, applying the right model at the right time, using the right harness that, again, picks and adjusts itself. It's a real burden area of exploration right now. Right? And we we... You have this. The more you can get good at it, the more, again, you can apply AI to smaller functional areas. If you look at the so called decision models, JEV and the 15 clones that have emerged over the last three weeks, which is insane and amazing and scary all at the same time, they are often used for tasks that previously would have been too expensive to use even like a GPT three or GPT four level model for. And that's great. Right? Because we're now applying probabilistic thinking to more and more feature sets. I think we're all just gonna see better products as a result. [00:27:47] S1: How do you think about the model routing layer? Is this an internal capability that every company will just build out commoditized? Is there particular prowess in it? We obviously saw OpenRouter, as the dear Americans call it, bought for for a lot of money. Maybe it's brilliantly strategic. Maybe it's not. I I don't actually know. How do you think about it if it is a core FinOps strength, as you mentioned?
00:28:10 · Mike Cannon-Brookes, Harry Stebbings
[00:28:10] S0: I think for companies of our scale, you have to be very good at doing that yourself. There is a constant challenge with scale. I think any large technology company is probably gonna do that internally, and that may involve using OpenRouter plus many other things, plus your internal router and everything else. Right? Like, these things can be continually laid upwards. For smaller businesses or businesses where technology is maybe not their expertise, those are just very practical solutions. Right? There are many, many model routers and gateways with lots of claims about what they'll do. I think, generically, it's very hard to just route automatically, I think, is what they're showing. You need to give enough steer or intent, and I think harnesses will also be a part of this. So this is... You're sort of talking for generic LLM calls. The more using harnesses, they're doing their own sort of routing. The more you're using SAS or the... You know, you come to the Atlassian platform and you use Rovo or any of our AI features, you will get model routing happening for you automatically, and it's our job to blend those costs and give them back to the customer for the the best quality answer for the cheapest quality price at the best speed. This triangle trade off, we're always looking at an any given feature. That's part of our USP to the customer. [00:29:18] S1: And you won't use third parties. That's internal routing.
00:29:18 · Harry Stebbings, Mike Cannon-Brookes
[00:29:18] S1: And you won't use third parties. That's internal routing. [00:29:21] S0: That is all internal routing. Yes. Yeah. We have an an an entire AI gateway team. It's not a small team that is doing that, but we have to do it at, firstly, at massive scale. And secondly, we want to continue to reduce that cost where it makes sense. So I say to product teams all the time, don't over optimize for cost in the short term. Build amazing magical features. If they're taken up by customers and what we thought was magical, the customer thinks is magical and it scales, now let's start working about the cost. Right? So think a little bit in advance. Can we possibly get this cost down? But then worry about that later. Right? If five people using a feature, I don't care how much it costs. If 5,000,000 people use a feature, I'm going to care a lot about cost. And in the middle, I usually have a fair bit of time to keep optimizing. [00:30:07] S1: What costs a lot today that you don't think will cost a lot in five years' time?
00:30:07 · Harry Stebbings, Mike Cannon-Brookes
[00:30:07] S1: What costs a lot today that you don't think will cost a lot in five years' time? [00:30:11] S0: Model inference. Maybe slightly spicy take is that the amount of energy startups, capital, time, smarts, brains, people who would rather have gone to work for McKinsey or some NASA or something else working on some version of the price per watt performance of inference is crazy. And I suspect a lot of those take years to come through from new chips to better data center design to ways of distilling and getting things out of foundational model. Like, there's lots of... The overall progress of that, I can't believe doesn't bring our price per performance unit of an LLM, we don't have a good term for this, down at a precipitous rate just because the volume of capital being applied, the volume of brain and talent being applied is so large. [00:30:57] S1: Is anyone arguing that that's the case? So I think they're just arguing that Chevron's paradox means that we will use so much more inference. You know? I I have instinct running about 50 different searches continuously in real time every every second of the day. We just use more.
00:30:57 · Harry Stebbings, Mike Cannon-Brookes
[00:30:57] S1: Is anyone arguing that that's the case? So I think they're just arguing that Chevron's paradox means that we will use so much more inference. You know? I I have instinct running about 50 different searches continuously in real time every every second of the day. We just use more. [00:31:12] S0: We will, but it'll still mean the cost comes down. So where the cost is near free, yes, it will explode. If something's free, use an unlimited amount of it. Right? And you don't care. You throw it away, whatever. Like, if it's not got a a waste component to it, which is ostensibly a cost, then, yes, that will be the case. I think there's always gonna be an economic curve to that, though. Right now, the production of technology is faster than the consumption of technology. It is ahead. We've reached a tipping point. We can build features faster than customers can consume them in a realistic way that changes their business. There's gotta be a point where that just... It doesn't stack up anymore. Right? There's not gonna be an infinite usage of all of this stuff. Right? Your phone has amazing superpowers. It's a supercomputer in your pocket. Do you burn the CPU a 100% all the time? No. You don't. You're not even close. Right? [00:32:03] S1: So so what happens to this inference layer of value and talent cohort where you mentioned the votericarization of chips, the different providers? What happens to it, Mike, in the next few years?