The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:10:29 · Harrison Chase
[00:10:29] S0: Yeah. You wanna basically not be locked in to models, and if memory, if context starts building up in a way that is specific to how one model wants to interact with memory or wants to interact with context, that's very bad for portability, and then that puts you at a disadvantage building your your business going forward. We talk with a lot of people about owning their intelligence and what that means, and there's a lot of different versions of that. The version that that that we think, like, owning your intelligence means, we we think there's a few different components of that. But one key component is having kind of, like, this open and controllable agent. Like, you wanna be able to adjust all the different pieces. You wanna be able to adjust the model. You wanna be able to adjust the harness as well because sometimes these model labs are training harnesses to be really good at the things that are in distribution for them, which oftentimes is coding, and coding is great, and there's a ton of use cases around coding, but there are a ton of use cases that aren't coding. And maybe they'll write code as part of it, but they're using that as as a way to do sales, as a way to do research, as a way to do these... The law, as a way to do these other vertical specific things. And so when you get out of distribution of what the the model labs are training on, you wanna have control over, like, what the harness does. And then you also want control over the context as well. So model, the harness itself, and the context make up kind of like this open and controllable agent architecture, and we think that's a big part of what it means to own one intelligence.
00:11:42 · Harry Stebbings, Harrison Chase
[00:11:42] S1: I just wanna back up a little bit. When we think about, like, agent value, so to speak, some people say that the value is actually in the verification, like whether it did well or not, because that unlocks the next step, which [00:11:55] S0: is [00:11:55] S1: automation. If value is derived through verification, who wins? The harness?
00:11:55 · Harry Stebbings, Harrison Chase
[00:11:55] S1: automation. If value is derived through verification, who wins? The harness? [00:12:02] S0: If value is derived through verification, I think the people who win are the domain experts, the people who are able to kind of, like, quantify what that value means, which is very hard. Like, building evals is a very hard job. And so I think a lot of the value will accrue to whoever can kind of write those verifiers, capture that domain knowledge. I'm massively bullish on vertical agents for different domains for this reason, because I think the domain experts who are best suited to write those verifiers or build those evals, as we kind of talk about it, They will be able to build harnesses and models. And it it won't be just the harness. It won't be just the model. It will be some combination of them. So so very practically, if you're getting started building a vertical agent, I would highly suggest you start at the harness layer. You do context engineering. You kind of, like, tweak the harness. And then when you've got users, then you start tweaking the model in post training. I think it's it's much higher ROI to start at the context or harness layer first. And so I think we'll have all these different verticals. They'll be able to quantify what good looks to them in a very domain specific way, and that'll let them build not only, like, vertical agents, but also vertical agent experiences. Different agents might have different experiences. And so I think, like, a lot of the value massively bullish on vertical agents, and so we want to enable a lot of that.
00:13:13 · Harry Stebbings
[00:13:13] S1: You said if I was building that, I would start at the harness layer. We're seeing a lot, whether it's Harvey or I think it's Ramp, build their own models. To what extent are they actually building their own models? Mike Cannonbrooks at Alasdians, don't know, show we really say that it's largely kind of, respectively, BS, and they're fine tuning open source Chinese models. Is that true? And and how do you think about that?
00:13:35 · Harrison Chase
[00:13:35] S0: I think a lot of it is they are post training open open weight models. Absolutely. I still think that's a valuable thing to do. I don't think there's kind of like anything wrong with that. I think that... I think there's different, like, dimensions that you can go in terms of owning your intelligence. I think the cheapest way is you just control the context. You use an off the shelf harness, off the shelf model. The next level down is you start tweaking the harness, and and that gives you more control than just the context, but it's a it's a little bit more expensive to do. The next level down is is you post train an open weight model. And so you might take an an existing open weight Chinese model and post train it for your domain. And then the very kind of, like, bottom layer is they actually train a model from scratch. I think many folks are training kind of like a model from scratch. I think most are post training. I think if you're training a model from scratch, you're probably at NeoLab. Think those are one of the few that are doing But, yeah, the ramps and the Harveys of the world, they have a ton of talent. They have a ton of data. They have a bunch of usage and are trying to think about really optimizing it, and so they've gotten all the juice they can out of the context, all the juice they can out of the harness, and now they've arrived at basically post training of these models.
00:14:35 · Harry Stebbings
[00:14:35] S1: Can I ask you, the jury's kind of out on the efficacy of these specialized models in the post training realm like we mentioned there? Some reports I've seen say they're massively more efficient and better, and some less so. How do you think this will play out?