Zero-Shot Learning
00:25:22 · Adarsh Hiremath
[00:25:22] S2: Maybe just a broader point is that people often view these investments in evals or whatever your infrastructure is as a one time investment. I think that that is extremely far from the truth. These are live investments. They're continuous investments. A good analogy is if you are shipping a feature, you would wanna write unit tests for that feature, assuming it's a net new feature. In the same way, if you are doing something with an agent, you would want an eval set that covers the scope of that agent and evolves over time. A concrete example of this in the customer support context, just because it's one that we're all familiar with, is let's say you have an agent that is trained to respond to inquiries about the product. Right? But then a new feature is released. Right? Like, there needs to be some steering and updating of the skills file to make sure that at the first time the agent fails on a query like that, there's automatically feedback that is inserted into the agent to address that new use case. And that is not something that's just true for customer support. It's also true for agents in in all contexts, their context. Right? Like, companies are idiosyncratic. They evolve quickly. They, add a bunch of new context. So that mechanism, that that you use to distill the company context into the agent and do it at really, really high velocity, keep the feedback loop short, is is a very, very, important one to solve.
00:26:35 · Dev Tagare, Adarsh Hiremath, Nancy Wang
[00:26:35] S1: So along the same, acts as other, Shah, if you think about, you know, side effects that an agent would have in interactions with the model, or the model is instructing something malicious or funny back to the agent, either directly or with some other feedback loops from other agents. How do you think about adding a layer of security in the middle, in which you can enforce some policies, you can, you know, evolve the intelligence of the agent alongside the model. What's your take on that? [00:27:03] S2: Yeah. So, you know, the first thing I wanna call out is that it's really, really hard to roll out agents to production in in part because of this issue. Right? And it's supported by the fact that the initial adoption of these agents has largely been for personal assistant use cases where the human is always steering the agent, and if the agent fails, it's on the order of magnitude of failing for one person as opposed to maybe a million people [00:27:24] S3: Yeah.
00:27:24 · Nancy Wang, Adarsh Hiremath, Dev Tagare
[00:27:24] S3: Yeah. [00:27:24] S2: That would be relevant to to a company. So just wanna say that that that's a really, really hard problem that a lot of people are trying to solve. In terms of how we actually steer the agents for a production context that has a larger blast radius, there's several components. I would say that there's just a natural progression of what you wanna do as you're in agent development life cycle. Right? The first one is you just try the agent in like a personal assistant context. Does it feel like the agent can do the task? Right? Then the next thing that you wanna do is then connect it up to some tools in a sandbox environment without the real data, validate that it can do the task again. Then you wanna roll out a set of evals, you know, live or not live evals to more methodically see if it could do the task. Then you wanna connect it all together in a sandbox environment that has real users, which means live eval connected to to real tools, but not real data. The agent is actually usable. And then you wanna do a phased rollout in a production context that has all of this infrastructure as well. So the key is not jumping the gun and following the whole process. [00:28:20] S1: Perfect. And like you throw an HITL somewhere in the loop with and without production. Right. That way, you have some control over what could happen, what should happen, so on and so forth.
00:28:20 · Dev Tagare, Nancy Wang, Adarsh Hiremath
[00:28:20] S1: Perfect. And like you throw an HITL somewhere in the loop with and without production. Right. That way, you have some control over what could happen, what should happen, so on and so forth. [00:28:31] S3: That's right. And as a quick aside, don't want to take us down, like, a huge rabbit hole. But, you know, I was on a call with a large multinational bank this morning, where they're saying that for more of their sensitive workloads, they're actually running it not just in sandboxes, but in remote dev environments. Are you seeing that request from some of your more compliance minded customers? [00:28:50] S2: Yes. [00:28:50] S3: Yes. Okay. And they're all kind of workloads. 100%. [00:28:53] S2: Yeah. Mean, I the typical ones are are are workflows in highly regulated industries. Right? Like, financial services related use cases being a, being a big one. The answer is yes. Right? Like, people... Like, the the general idea of you do not wanna run the agent in production for the first time. You wanna run it on sandbox environments, locally, remote just to every... The whole thing under the sun, people are are experimenting with that. [00:29:17] S1: You know, sandboxing and all of those is gonna be a big business very soon. [00:29:21] S3: I see actually the worlds of sandboxing and remote dev environments, Coder, for example, and others, converging over time.
00:29:21 · Nancy Wang, Adarsh Hiremath
[00:29:21] S3: I see actually the worlds of sandboxing and remote dev environments, Coder, for example, and others, converging over time. [00:29:27] S2: Totally. Right? And and we, course, do a lot of that work with the labs, right, which is, these RL environments, right? Instead of if you don't have access to the real production tools and context, or you don't wanna give agents that access when you're rolling something out, it can be helpful to create a mock up. Right? Which is select the agent, select the set of tasks you wanna test on, create mock ups of everything, mock ups of all the tools that that the agent would need to have access to, use those mock outs to... Mock ups to roll out a trajectory, eval the trajectory, and then eval the output. Is is what people are doing in in context like these. Totally. [00:30:04] S3: So speaking of outputs, I wanna maybe take us to actually the recent benchmark that y'all published, Apex Suite. Yeah. Yeah. Super cool. And just for listeners here, Apex suite tries to measure, you know, whether frontier models can do economically valuable work over time. So I'm curious, you know, what led you all to create this benchmark, and what was missing from existing benchmarks?
00:30:26 · Adarsh Hiremath
[00:30:26] S2: The thing that inspired the whole Apex series of benchmarks is that we felt that there are benchmarks that are really good at maybe evaluating how well the model can do an academic task. Mhmm. Can it can it do IMO math? Yeah. Maybe modeling how well, you know, these models can do tasks that are kind of like an approximation of the economy, right, like GDP val. But there's no benchmark that can really evaluate how well can the model do enterprise work output, right? Or or knowledge work, or or just like real work. That's when we started with Apex, the first version, which was largely for professional services. We made that benchmark agentic, and then we also rolled out Apex Suite. In the software engineering context, the biggest gap that I see is that a lot of these benchmarks are isolated to small patch tests. Right? Okay. This whole repository is messed up. Just like patch these kind of functions. Right? So it focuses on these patches as opposed to the whole system. And secondly, as you see the failures in coding models, just based on these benchmarks. Right? The failure of the benchmarks to create this distribution of real software engineering work is also a failure that we see in the models in production. So APEC SWE is our attempt at addressing that. And there are two types of tasks in APEC SWE. The first one is observability, right? Which is in a real enterprise context, if system fails, right, you would need the agent to triage across all these different things, go look at the logs and say, hey, this is not right, and then raise that to the user in whatever area that they're doing work. And the second is integration, right, which is just not a patch test or repository. It's a set of systems and how they integrate with one another. The last thing about Apex is, as we know, just correctness, whether the code compiles or not, is, like, not the the only thing. Yeah. It's also these other things that are super relevant. Right? Is this a piece of code that you would be excited about, being in your code base six months from now? Does it follow style requirements? Does it follow all these other things? So the way that we actually come up with the benchmark scores in Apex Suite is a rubric that is not just correctness based, but it also has other criteria, to evaluate how well the models are actually performing in a real context.