Latent Space: The AI Engineer Podcast
00:09:32 · Aman Singh
[00:09:32] S0: Yeah. So all of our safety restrictions are are fully deterministic, and they they take the decision making off the the agent's hands. So for example, certain tools let the let the agent kind of execute the tool itself. So if it's like a simple record keeping kind of tool, it will execute it. We also... With every time a tool is called that writes data, we have LLM based verification running on on that action. These, like, fast LLM calls that that analyze every single write to our database. So that's another layer of verification we're running. And then we also ingest a lot of, like, external information. So there's emails coming in, documents coming in, phone calls being made, which are targets for prompt injection. So we run sort of verifications at the at at the edges that detect presence of certain, know, flavors of prompt injection. And then anytime there is a there is a case where data leaves our system, for example, in, like, a a web search in in the most obvious case, but also if we're sending a calendar invite to Richard. Right? That's an external kind of data where we have both deterministic checks on presence of certain information that we know is PII, but also a consensus of small models that that ensure that there isn't any kind of data exfil being attempted. So we have tons of different different kind of guardrails built into the harness with the principle that we want to kind of contain them at the edges of capability and not earlier than that.
00:11:02 · Richard MacManus, Aman Singh
[00:11:02] S1: Yes. And and because you're in the financial, you know, industry, are there certain kind of regulations that are required that you have to sort of keep to in terms of the technology that you're using or is that is it just more about outcomes and the way you sort of deal with clients? [00:11:18] S0: Yeah, the industry cares about outcomes but compliance cares about auditability. Right. Right? And so our system is such that auditability is by default. Mhmm. We have every single action that that an agent takes produces a change log of what changed, who changed it, from which part of the system, and when. And this applies not just to agents, by the way. It applies to human actors as well. Mhmm. So if you change something in the UI that affects the client, we'll log that action. So it lets us both get auditability by default, but also power some of our agentic capabilities where the same thing that that retracts like a UI change Mhmm. Also could be used to, like, inform the agent that that happened. [00:12:05] S1: Right. [00:12:06] S0: So auditability has been a big part of our our story, both for obviously compliance and regulation, but [00:12:12] S1: also for our agentic loop. Yes. In your presentation, you had a nice line agents prompt people more than people prompt agents. So how do the agents decide who to involve and you know, what sort of how much of the context that they need to provide that person in order to do what they need to do?
00:12:12 · Richard MacManus, Aman Singh
[00:12:12] S1: also for our agentic loop. Yes. In your presentation, you had a nice line agents prompt people more than people prompt agents. So how do the agents decide who to involve and you know, what sort of how much of the context that they need to provide that person in order to do what they need to do? [00:12:32] S0: Yeah. Another interesting one. So this is similar to how humans work in in in in our industry is there is a process that we're tracking over ten days. There's multiple actors involved. The the actor at step seven may not need to know about this just yet. Right? They're pulled in at the right time. And so our agents with our kind of, like, long memory persistence plus the daily reflection on what is happening reflect on where the the state of the world is at the moment. And they're able to pull in the right person at that time when it is... When when there's a signal that that implies that that person must be pulled in. And all of this is work that we have done to operationalize the context of how this industry works. But not just that, but how does Arca want it to work and what are our principles of of how work happens in this industry. And that is another advantage of us being sort of a a a... Not just a SaaS company. We're actually not a SaaS company. We are we are the firm ourselves. So we can standardize the way this work even happens so that our agents can participate in standard ways that that work must happen.
00:12:32 · Aman Singh, Richard MacManus
[00:12:32] S0: Yeah. Another interesting one. So this is similar to how humans work in in in in our industry is there is a process that we're tracking over ten days. There's multiple actors involved. The the actor at step seven may not need to know about this just yet. Right? They're pulled in at the right time. And so our agents with our kind of, like, long memory persistence plus the daily reflection on what is happening reflect on where the the state of the world is at the moment. And they're able to pull in the right person at that time when it is... When when there's a signal that that implies that that person must be pulled in. And all of this is work that we have done to operationalize the context of how this industry works. But not just that, but how does Arca want it to work and what are our principles of of how work happens in this industry. And that is another advantage of us being sort of a a a... Not just a SaaS company. We're actually not a SaaS company. We are we are the firm ourselves. So we can standardize the way this work even happens so that our agents can participate in standard ways that that work must happen. [00:13:45] S1: So earlier in the conference, Anthropic did a workshop about their managed agents infrastructure and a lot of it, even the orchestration is becoming more automated now. So do you foresee kind of using that type of functionality? Because it obviously presents a danger and that the human is not necessarily always in the loop anymore. So have you had any thoughts about using that technology to automate even more of the process?
00:13:45 · Richard MacManus, Aman Singh
[00:13:45] S1: So earlier in the conference, Anthropic did a workshop about their managed agents infrastructure and a lot of it, even the orchestration is becoming more automated now. So do you foresee kind of using that type of functionality? Because it obviously presents a danger and that the human is not necessarily always in the loop anymore. So have you had any thoughts about using that technology to automate even more of the process? [00:14:11] S0: I think we are very bullish on the models from from Anthropic and and labs like OpenAI, but less so on on taking a dependency on a product like managed agents. Because managed agents, one of their their key things is how the models kind of persist and and contribute memory as to what is happening in the world and all of the operational knowledge that they need to be... Need to use to to be successful. Yes. And so letting them own that part of the business is kind of like, a a risk because, owning that is... Owning your intelligence is kind of, gives you the ability to impact the business. If you if you hand off the intelligence to to another provider, you lose some of that control that you need to kind of, like, build something that is unique and tailored to your business to such a great degree that it can actually impact your business. [00:15:07] S1: And just on Anthropic, they've they've also recently released financial services, so they're partnering with a whole bunch of different firms. Mhmm. Do you think what they're doing is kind of competitive to what you're trying to do, or do you see them different things?
00:15:07 · Richard MacManus, Aman Singh
[00:15:07] S1: And just on Anthropic, they've they've also recently released financial services, so they're partnering with a whole bunch of different firms. Mhmm. Do you think what they're doing is kind of competitive to what you're trying to do, or do you see them different things? [00:15:20] S0: Yeah. I think, I think Anthropic getting into our kind of industry is is very validating for for our industry. But the way I think about it is that it is very different to be an, a a services firm that has access to AI tools versus a firm that was built to be AI native. Mhmm. The firm with AI tools requires their users to prompt the use... The agents kind of to to produce work or to do work. But here, like I said, in... At Arca, agents prompt people more than people prompt agents, and it is a proactive kind of, service that is running for them twenty four seven, and it's kind of like running the business in the most standard way that we that we expect at the level of service that we that we that we aspire to, which looks very different if you're if you're a... An AI AI... Sorry, a company with AI tools. [00:16:09] S1: Yes. Yes. Okay. So a little bit more about on the security side. So can you explain, like, what in your processes runs inside a sandbox and how do you verify like a change to a financial plan, you know, before anyone acts on it? Mhmm.