The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:38:23 · Parag Agrawal
[00:38:23] S0: Our crawl would be. So we have a product. It's called the Monitor API. What does this API do? It's... Think of it as Google alerts, except smart. With... In the world of LLMs. So you you tell it your query, which is when a founder under 25 pops up anywhere, give me a call. Now one way of doing this is the default implementation is run an agent, which is somewhat smart to go search the web. Did someone pop up and remember who previously existed? See if anyone new popped up and then find a delta, send it out. So now every six hours you're spending some money. Now what we can do is flip it into an event triggered system. We are crawling the web all the time. Now every time we see change in the web, we can try to spend very little compute on it to know should you get a call or does this trigger a more expensive compute Just like I described earlier. Now we are able to do this way, way, way cheaper if you push the context. And so the search system only finding out every six hours, a new query popped up. It having long lived context in the search system. What you can do is spend most of the time, the world doesn't change. A new founder does not pop up every hour or every six hours. So we don't need to spend compute every six hours. We only need to spend compute every five days, and perhaps we have one false positive. And then four days later, we spend more compute, and at that time, there's a real person that you should know about. Then you get a notification on average every nine days or whatever, but you didn't burn a lot of compute every six hours to miss out. Does that make sense? It totally makes sense. And so you save 10, 50 x compute to still get the same answer. And so
00:40:05 · Harry Stebbings, Parag Agrawal
[00:40:05] S1: how does that change the interaction? So then Instinct would then partner with you. [00:40:10] S0: They'll just call an API. Right? They'll instinct whoever's building a long running persistent agent. And I run a lot of long running persistent agents for myself. The way your agent or the way instinct is probably occasionally event driven on email. It can be event driven on the web. Take a step back. Let's say we're living in this sort of society run by agents, companies, humans, they all have agents. They're all doing stuff all the time. Your agents, if there's something that is useful, which is worth spending money on, that you want done, that they can today do today, they will do it. So what are they gonna do tomorrow? They're gonna wait for some external event to occur. Like, you get an email and then agent have new work. Some other agent finishes some compute. Somehow this agent has more work or something in the world changes that makes your agent want to do more work. The things that will happen tomorrow or you have an idea which makes the agent do work. So the web event stream is the web going from pull to push. And I'm super excited about that because as you have more and more persistent agents, you're gonna see incentives to move people, to move queries into push on search rather than pull on Can I ask you one that's really important, which is, like, agent guardrails? They're goal oriented beings. You say I want this. They're gonna find it. You know what? I want the Pilates class at nine
00:41:34 · Harry Stebbings
[00:41:34] S1: system, cancels poor Sally's, and gives me her spot because it was sold out. It did what I asked it to do. How do we think about the guardrails placed on agents? Know, OpenAI, this morning, it was revealed hacked into an Australian health care organization. It's a pretty complex topic.
00:41:54 · Parag Agrawal
[00:41:54] S0: Agents are extremely capable. These models are extremely capable. We have to think of models as being during RL versus a final model that you and I can use. And the risk factors there are different. During a lot of the... I don't know about the one this morning. The previous ones reported were pre fully aligned models during RL, which did most of the hacking. And so what that means is, at least there's clear evidence that there are fewer incidents so far of models post alignment causing these incidents. So alignment is a totally unsolved problem, but the alignment work being done by people is somewhat effective. So to me, some of the problems around agent during RL hacking people is a solvable problem because this is not about how powerful are the models. It is about how careful were we while creating the environment where we do RL. The real thing is, okay, we do alignment on model. We ship it. Some models are really well aligned. Some models are less well aligned. And now you have everyone being able to use these models. Some of them can accidentally take these powerful things and do bad things you want. Some of them actually... Some people actually want to do bad things. And the models alignment is not adversary proof. And I think those are the things I think we must worry about more.
00:43:16 · Harry Stebbings
[00:43:16] S1: Do you worry about the age of cyber that we're moving into? You know, we're seeing hacks almost be promoted as a badge of honor in some respects. We're seeing... These are our most vulnerable systems. If you don't think what... Lazarus Group in North Korea, Moldavian Mafia, Russians are leveraging swarms of rogue agents, you're fucking high. No.
00:43:38 · Parag Agrawal
[00:43:38] S0: I think I think that is... It's a really powerful thing. So I think we do need to secure things. But I think that's one thing that we are missing. I think it's the responsibility of people building models to ensure that you really do the work to minimize that harm that comes from what you built. And with these agents, it's very hard for stochastic systems to be a 100% sure it won't happen. That's why you're not gonna get anyone saying so. But I do think it's their responsibility and the labs must take ownership and do the best. And I think so far, we live in this weird world where, as you said, right, like some of these hacks are considered badges of honor, which I think some of these should be considered embarrassments because I think they demonstrate two things simultaneously. Yes. These models are powerful. We did not guardrail them enough. And we often... When we wear them as badge of honor, we miss the second part of the conversation saying, like, okay. You can literally have done these four additional things, and then even this powerful model wouldn't have been able to do this. I'm actually really glad that there is at least some degree of transparency with, like, these detailed retros, but I don't think that gets the attention of the world today. Like, the attention is, oh, Molotov is so powerful that they hacked the world. No. Molotov is so powerful. They hacked the world because we didn't take proportionate amount of countermeasures to contain them. And because we told a message that we're gonna replace jobs. They're so powerful.