The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:33:43 · Harry Stebbings, Harrison Chase
[00:33:43] S1: How does the infrastructure change when agents are the fundamental interface with sites? And what I mean by that is, I wanted tickets to a specific concert, and I said check every five minutes and buy any seat that comes available, da da da da. And for three days, it checked every single five minutes. The volume of queries was a thousand times what it would have been with humans. How does the infrastructure require change in a world of interaction or human interaction? [00:34:13] S0: When agents are interacting, it makes the number of requests go way up. I think I think you saw a story about how... I think it was Resi that blocked a user's account because they were using Instinct to to pull it, I don't know how frequently, but pretty frequently. And so I think there's just way more strain on the infrastructure. How how does that percolate down to lower layers? I I don't know if I have an opinion on that yet. I think you have to build systems that can be much more bursty, that can scale way way broader. I don't I don't know if I have more specifics on Do you [00:34:40] S1: think providers will allow agent usage or not? [00:34:44] S0: I think the future is providers exposing their own agent and having people either interact with that agent or having people's horizontal agents interact with that agent. [00:34:53] S1: So my horizontal agents, sorry, just to understand this, would interact with Resy's vertical agent. [00:34:59] S0: I think that's exactly right.
00:34:59 · Harrison Chase, Harry Stebbings
[00:34:59] S0: I think that's exactly right. [00:35:00] S1: Why would that work? Because that still doesn't solve the two core problems of advertising for a lot of these businesses. They lose their ads model and then upsell. They lose the, like, anything else you might like also on the side. You get reduced basket size. Why why would agent to agent work? [00:35:14] S0: I think you could include some of those things in the agent response. So you could do advertising for specific things on on Resi or Etsy or whatever and include that in the agent's response. Like, hey, I found these three things. You might also want to consider this other thing. That's now a natural language response that goes back to kind of like the the main agent. I think also to the point around the strain on the systems, if you have a vertical agent, it's gonna be better and more efficient at interacting with the underlying systems than a third party agent will. So I think it will make it a lot more efficient for for them to do that. [00:35:44] S1: When we look at the amount of queries and the volume that changes with agents, It takes a lot of tokens to do all of this work. How does the cost of tokens and cost of that intelligence change over time? Because we always thought it would get cheaper. Hasn't gotten much cheaper. How do we actually think about that?
00:36:00 · Harrison Chase
[00:36:00] S0: Cost is one of the big things that we talk with people about right now. I think there's a few different dimensions to this. One, I think like the frontier models are relatively getting cheaper over time. Astra is, like price per performance, fantastic compared to a lot of things that existed a year ago. Two, there's just way more demand. When you start talking about agents talking to agents or multi agent systems, there are just so many more tokens that are getting burned, and so the costs are going up. We see this in coding primarily. Coding costs have skyrocketed, and I think a lot of organizations, including ours, are now thinking about ways to kind of like pair pair those costs back. How does this end up getting resolved? I mean, if you look at how we're tackling coding agents, I think there's a few hints there. So so when we're tackling coding agents, think of it in three steps. One, we think of basically just, visibility showing kind of, like, how much things will cost, how things have cost to admins. Two is cost control. So you could say something like, hey, you know, go get me a reservation, but don't spend more than $5, like, doing it. And then third is actually impacting the harness itself. So I think primarily a lot of harness engineering has been done around making the agent more efficient because that's the... Or sorry. A lot of the harness engineering has been done to make the agent just like better, more accurate because that's the biggest blocker. I think as cost becomes more concerned, we're seeing people care more about things like prompt caching, which are really good for kind of like reducing costs. We're seeing model providers actually make this easier by... And this getting a little bit in the weeds, but they introduce things like where where you can change the tools that are available to an agent mid flight. Previously, this would have busted the cache and caused like a massive increase in in cost. Now you can just add those dynamically and keep the cost the same. So I think we're seeing more attention to both the model APIs and and harnesses in terms of, like, controlling costs.
00:37:38 · Harry Stebbings, Harrison Chase
[00:37:38] S1: When you talk to large enterprises today, what matters most to them, is it accuracy? Is it cost? Or is it security? [00:37:46] S0: Another thing would be latency, by the way. So so we so we run developer surveys every year where where we ask developers what they care about. For the last two years, the number one response, basically double the next two, has been accuracy. So that's still the the primary thing. The next two are latency latency and and cost, cost, and security is, like, a a little bit after that. Pretty close. When we talk to large enterprises recently, I'd say, like, cost has has risen in terms of concern. I'd still probably say it's below accuracy, but it's probably higher than latency. And then security is... For large enterprises, security is higher, but still probably below accuracy and cost. [00:38:21] S1: What percent of your engineering salary, give or take, not specifically, but do you spend on coding tools?
00:38:29 · Harrison Chase
[00:38:29] S0: I think we spend probably roughly 30% of our engineering salary on coding tools. It's actually gone down the last two months, while tokens, the number of tokens that we've used, has actually gone up. And so this has... We've done this by three steps. First is basically just visibility. We tracked all of our coding agent usage to Langsmith so we could tell who was spending how much, what they were spending it on. We could do analytics over that. Second is just basic cost controls. We set kind of like a budget. And and if you if you hit the budget, you can talk to our VP of engineering and and he'll almost always, like, increase it, but it's just a safeguard against kind of like agents being used incorrectly or going in a in a in a bad loop. And then third would be we control the harness. So one of the things we've done is we actually built our own in house cloud coding agent. So not not a IDE, not a not a terminal based agent. We we let people use whatever they want for that, but we built our own cloud based agent, Oop... OpenSUI is what we call it. It's actually the only way I code anymore because I'm in Slack all the time, and so I just message it from Slack and it kicks off. But in that harness, we've done a bunch of work to control costs. So things like prompt caching, but also things like routing, we've built into this harness. And I actually think that's like a pretty repeatable three step process for kind of, like, getting coding agent spend under control. Just first, visibility. Second, like, basic controls. Third, like, try to improve the harness itself.
00:39:42 · Harry Stebbings, Harrison Chase
[00:39:42] S1: Totally get that. I do just wanna unpack that. You mentioned that visibility, see who's spending what. Were the people spending the most the most effective? [00:39:51] S0: Sometimes the people spending most are the most effective, and other times they're not. So what I mean by that is we have a few senior engineers who spend a lot, and I a 100% trust that what they're doing is well worth it. No questions there. We've also just had some cases do do more to, like, kind of, like, silly mistakes or kind of, like, improper setup than anything. We've had, like, massive spikes in usage. So so an example of this, we we had one or two tasks where some plug in was being used that basically, like, would would screenshot or take a snapshot of of the DOM of a web page and send that back every turn of the agent because they were doing some front end engineering, and they wanted to take what it looked like and just send it back. It would just run this massive loop. And so I think we spent 40 k in a week on on that one thing. You know, when it gets to that, like, again, like, I think we're pretty kind of, like, liberal as a company and trust people, but I I know that's probably not worth it. So And that's just like an example of like... Again, it's... It wasn't malicious. It wasn't even intended usage really. It was kind of just a mistake. So I just think just like catching that is well worth it. So it depends a bit on the side of the spectrum. [00:40:52] S1: 30% of salary, as you mentioned. [00:40:55] S0: That's guesstimate. Yeah.