"The Cognitive Revolution"
00:14:09 · Nathan Labenz
[00:14:09] S0: One big kind of narrative violation that we've heard from other sources, namely Andon Labs, who, as I'm sure you're aware, does these, like, Gonzo, you know, put a AI in charge of a business kind of experiments, is that OpenAI models, they have found to be more aligned, less inclined to do sort of, you know, sneaky, bad behavior in the course of running their businesses. I see basically a a similar signal on the Arena alignment index where Correct. GPT is holding the top five slots.
00:14:46 · Anastasios Angelopoulos
[00:14:46] S1: And what we've seen from this is that if you annotate all the traces that go through Arena, about, you know, single digit percentage of that... Of them, even from OpenAI and Anthropic models exhibit some form of misalignment. So perhaps the high level takeaway would be, OpenAI and Anthropic are the two most aligned labs. Alignment tends to follow capabilities. So more capable models also tend to be more aligned perhaps because these companies have, you know, really made an effort to make their models more aligned. But also the rates of misalignment are perhaps higher than people initially would think. Single digit percentage misalignment is a ton. It's a ton. It means that anybody that's had a thousand chats has experienced 10 at least, you know, 10 to 50, let's say, examples of a misaligned behavior like a lie or deception. And so it's pretty interesting. OpenAI is number one on our leaderboards right now. It is overanthropic, I believe, in all three of our signals. So there's actually not one of our signals in which anthropic is leading. It really speaks to the efforts that the OpenAI team has made to make the models more aligned in production. And, yeah, it is a narrative violation, but the reality of the matter is that these two labs are clearly the front runners in this area. Both are working very hard, to avoid misalignment. And then the third runner-up is, SpaceX AI, you know, just behind Anthropic and then Google. And then there's a big gap, between these and the and the front... And the, open source labs. You know, Tencent is number five. The deceptive completion rate of the Tencent model is 13%. So 13% of the time that it's making these claims, it's it's making them deceptively.
00:16:42 · Nathan Labenz, Anastasios Angelopoulos, Sagar Vaze
[00:16:42] S0: The day before, artificial analysis had published version five of its index with most of its benchmarks now private. The open weight models fell to the back, which some read as a sign of benchmark maxing. Prakash asked whether Arena could detect benchmark maxing on its own platform. [00:16:58] S1: Well, let me just say, first of all, zooming out, I'm actually pretty concerned about the way that the industry is heading with respect to benchmarks. I love the artificial analysis guys. They're great people. I don't think that it's healthy for our industry to be relying on privately held benchmarks that are just like a dataset that's collected by some companies. You know, a lot of them, you know, in cahoots with the labs to sort of like, you know, sell data that's very similar to these benchmarks to have people improve, and then they're not auditable by the community. I don't really think that that's a sustainable way for us to be evaluating models. I have a serious problem with it because it's not actually solving the fundamental issue of quote unquote benchmarking. You can benchmarks on a dataset that's private. All you need to do is find data that's similar to that dataset and then... Or, you know, test on it enough times. And because that set of questions is fixed and and you have the, you know, the issue of multiple comparisons, you're you're gonna be able to, like, you know, improve on it [00:17:54] S4: more and more and more and more until you get to number one.
00:17:54 · Sagar Vaze, Anastasios Angelopoulos
[00:17:54] S4: more and more and more and more until you get to number one. [00:17:56] S1: I don't really get the narrative about the benchmarking of the open models and so on and so forth simply because we don't really know whether the frontier closed source models in The US weren't benchmark maxing against the closed datasets that are now going into this index. You know, these companies have very close relationships with, like, Cognition and, you know, some of the other companies and artificial analysis that are building these closed datasets. What we really need is benchmarks that speak to the utility of AI to real people and real workflows. And there should be a consistent and scalable methodology to do this that doesn't overfit. When you have live fresh data coming in all of the time from, like, real production use cases, it can't be overfit. It cannot be overfit, at least statistically. You can argue that you're doing better on that distribution than you are on other distributions, but at the end of the day, for that customer set, you're doing better as you go more and more because every question is fresh. It's like you're constantly taking new tests. It's not like you have the same test and you're taking it many times. And so I worry if that's the direction that the ecosystem is going, that we're going to set ourselves up for failure in the long run.
00:17:56 · Anastasios Angelopoulos, Prakash Narayanan
[00:17:56] S1: I don't really get the narrative about the benchmarking of the open models and so on and so forth simply because we don't really know whether the frontier closed source models in The US weren't benchmark maxing against the closed datasets that are now going into this index. You know, these companies have very close relationships with, like, Cognition and, you know, some of the other companies and artificial analysis that are building these closed datasets. What we really need is benchmarks that speak to the utility of AI to real people and real workflows. And there should be a consistent and scalable methodology to do this that doesn't overfit. When you have live fresh data coming in all of the time from, like, real production use cases, it can't be overfit. It cannot be overfit, at least statistically. You can argue that you're doing better on that distribution than you are on other distributions, but at the end of the day, for that customer set, you're doing better as you go more and more because every question is fresh. It's like you're constantly taking new tests. It's not like you have the same test and you're taking it many times. And so I worry if that's the direction that the ecosystem is going, that we're going to set ourselves up for failure in the long run. [00:19:10] S6: So let's talk a little bit about model routing. So I believe you guys have just introduced a model router. Can you explain business reasons or the user reasons for introducing a router? Let me tell you what we
00:19:28 · Anastasios Angelopoulos
[00:19:28] S1: saw internally at Arena. At some point, we were looking at all our traces that are being done on our coding agents internally. By the way, we're in the top you know, less than 1% of business in terms of AI spend. We're spending thousands of dollars a month per employee on AI just to help us accelerate our workflow. That's why our sort of revenue to employee ratio is crazy off the charts because we're using a lot of agents internally to do our work. Now what happened is that we were looking through our traces, and we saw these traces, tons and tons of traces that were like, thank you, and then that costs you $40. What? Why is this happening? It's because somebody did a long running task, they let their agent go, it created some artifacts, blah blah blah, very long conversation. They left for a day, they came back, the cash was invalidated, and they're happy with the results. So they say thank you and enter, and that... That's a $40 charge. Terrible. Terrible. Terrible. We had... We're burning money. We're burning VC money. Right? We shouldn't do that. What we should do is we should figure out which tokens are best spent on frontier models, you know, which tokens are best spent on a cheaper model that can do the job, where we should be caching intelligently and, you know, avoiding some of these other types of cache hits, more intelligent caching behavior. And that's why we've been building the Arena router, we call MAX, short for argmax, Is it maximizing capabilities subject to cost constraints?