No Priors: Artificial Intelligence | Technology | Startups
00:27:27 · Sarah Guo, Jake Stauch
[00:27:27] S2: You guys built an agent that helps you build automations. Can you explain how that works? [00:27:30] S0: Yep. So one of the things, I think every company discovered this as they're building kind of the first generation of these these AI tools, is that you could build such amazing complicated systems, but then you had to train people to understand all of your product primitives and how these systems fit together. And so the first generation of this is we deployed all these FDs to explain to customers, okay. Here's here's how to build in the platform. Actually, let me just build it for you. And then we realized it would be really nice if customers could just talk to an agent that understood their requirements and then translated that into the product primitives. And so we built this agent called Catalyst that does exactly that. It's an agent for managing the platform. So it's an agent for admins to build out all those underlying automations, but also configure the rules around approvals, permission checks, build out different policies, update all these different systems, understand the the entire deployment. And so because Serval is all cogen under the hood, having the agent on top of that to help you manage the system just made a ton of sense and is so powerful. Because now I don't need you to train yourself on how to use Serval. You can just talk to an agent that actually understands how to use Serval and how to build on it, and and that is your primary interface into the product.
00:28:39 · Sarah Guo, Jake Stauch, Elad Gil
[00:28:39] S2: I I think one thing that's underappreciated and I think is cool is that the experiences for agents in different products and actually in, you know, cogeneration agents as well, being able to bring a user along as they are building a product or an implementation or something is actually, like, raising the technical competence and ability of a huge number of users and and turning people into more, you know, engineers, actually. [00:29:04] S0: Absolutely. You have this blank slate problem in the old generation of automations of where where do I even begin? I know I wanna automate all these parts of my job, but where do I begin? Products like ours say, where do you wanna begin? And just can start making suggestions, and you can iterate with them. You can say, I actually don't understand that. What are you trying to say? Can you explain this to me? Okay. I'm trying to do this. How would I even go about doing this? And they can give you that information. They can fetch the information. They can explain concepts to you. And so it does become a build partner, and you can achieve so much more than just prompting an agent to go and build something. It can actually walk you through the process and help you explore new opportunities that you wouldn't even have even thought of. [00:29:39] S1: So one thing that seems a little bit under discussed right now is background agents. Could you tell us more how you think about that and what you think is coming in terms of this next wave?
00:29:39 · Elad Gil, Jake Stauch
[00:29:39] S1: So one thing that seems a little bit under discussed right now is background agents. Could you tell us more how you think about that and what you think is coming in terms of this next wave? [00:29:46] S0: Yeah. I always describe background agents to the companies like, this is the ultimate moonshot in enterprise software, just these agents that go off and do work for you and magically discover all your problems and solve all your problems. And it would be amazing if we could get that deployed, and it could work, and it could follow our permissions. And I think no one's really figured out how to do this well. Because on the one hand, you want these to be very autonomous, to go off and figure stuff out, and then solve problems without somebody babysitting them. On the other hand, autonomy is very scary in the enterprise, and so you do wanna see what exactly can they read. [00:30:18] S1: This is kind of the OpenAI hugging face incidents or other things where agents are given a task, and then they do something that's unexpected Yes. In order to accomplish that task. And so what are the guardrails we can put in place?
00:30:18 · Elad Gil, Jake Stauch
[00:30:18] S1: This is kind of the OpenAI hugging face incidents or other things where agents are given a task, and then they do something that's unexpected Yes. In order to accomplish that task. And so what are the guardrails we can put in place? [00:30:27] S0: Exactly. Because my best employees, what do you do with them? You say, here's the general high level mission. Go off. Don't talk to me, and go do it. And you'll come back, and you'll have done amazing things. I want to be able to trust agents to do the same thing for me, where I can give them this high level task, and they can come back and solve this problem and come back to me. I don't think we figured out how to do that in a way that it feels safe in the enterprise. What do [00:30:46] S1: you think is a way to make it safe? Is that... I mean, people have experimented with different things over time, everything from AI to certain forms of RL that... I'm just sort of curious if you have a solution in mind, or or is it just
00:30:46 · Elad Gil, Jake Stauch
[00:30:46] S1: you think is a way to make it safe? Is that... I mean, people have experimented with different things over time, everything from AI to certain forms of RL that... I'm just sort of curious if you have a solution in mind, or or is it just [00:30:57] S0: We think of it in in the large enterprise. It's not perfect, but it's it's kind of an incremental escalation of capabilities. So you go off and you do these things, and it comes back and says, hey. I would like to be able to do all this stuff. This would help me do my job, and you approve them. And all this stuff are deterministic workflows. I wanna call this tool. I wanna run this script that does x, y, and z. Can I do that? And you just kind of approve these with certain permissions and guardrails around them, you say, okay. Yeah. That's okay. That's okay. That's okay. And over time, the agent becomes pretty competent because it has a a big enough toolkit that it can solve most problems. And it's not perfect because it's annoying to have to babysit that process, but I think it's what you have to do because otherwise, it could theoretically just start writing code that could take action on all your systems, and that is an unbounded security risk that I don't think anyone's gonna be able to take anytime soon.
00:31:46 · Sarah Guo, Jake Stauch, Elad Gil
[00:31:46] S2: Do do you think to your point of being excited or you and Alex being excited about future capability, like, we're we're still missing some model capability here? The reason I ask is, know, the ability to be proactive and then predict what an individual, Jake as CEO, me as an investor, or me as a human being, you know, and and get it right is still very hard. Right? You need a lot of preference data. Think about an employee working for you. Like, okay. What are Jake's actual priorities? When he gives me this high level mission, what does he mean? Do you think that's a model problem, or it's entirely like a a guardrails and harness problem today? [00:32:22] S0: I think it's it's both, honestly. I think memory obviously has to be better and better because you wanna understand the preferences. You want the context to be insanely large, because I don't wanna have a bunch of different chats with you. I wanna have kind of one single chat that goes back all the way to our very first conversation, and you remember the key points of that that are relevant. And then I think the permissions problem will always remain the biggest thing, because even if I feel like you are a very, very valuable, smart agent, and I... You can do all these things for me, how much do I trust you to just run wild and not have to babysit [00:32:56] S1: your [00:32:56] S0: actions? [00:32:57] S1: You have to do that with people too. [00:32:58] S0: Right? Mean But you also have anybody who