The Frontier with Brendan Foody
00:27:44 · Brendan Foody, Jonathan Frankle
[00:27:44] S1: What are the public benchmarks that you find most useful today? [00:27:46] S0: That's a good question. I think when like increasingly at Databricks, we're not really working off of any public benchmarks. It's all in-house annotated benchmarks and usually on our own data because our data is real and messy. And we can at least, it's data we have full access to. We don't have to ask anybody's permission. We don't have to create a synthetic data set. Honestly, I mean, things like Browse Comp Plus have been really useful. Things like Stark have been really useful. Like a lot of retrieval benchmarks are good. They're contrived. Like you can read the questions in Browse Comp Plus, and you're not going to say this is a real search anybody would do, but they do challenge some capability of a model in a meaningful way. [00:28:19] S1: Totally. [00:28:20] S0: We've released our own benchmarks at this point. Like the Office QA benchmark that we've worked really hard on is a parsing and reasoning benchmark. Because it turns out that PDF parsing is still an unsolved problem, even in this day and age. Shocking. And you know, the the fun part, we actually just released it this week, was that it's a benchmark on treasury documents. And we actually worked with the Treasury and got, you know, Treasury documents going back to 1789 to try to parse. And good frickin' luck parsing benchmarks in 1789. We have a copy of like Alexander Hamilton's paycheck, which is just hilarious. [00:28:50] S1: Jonathan, is that actually in the user distribution?
00:28:50 · Brendan Foody, Jonathan Frankle
[00:28:50] S1: Jonathan, is that actually in the user distribution? [00:28:52] S0: Um, some Databricks customers are actually hundreds of years old and actually have some data like this. Okay. It's really cool. But like I was chatting with a customer that has jet engine schematics going back to before there were jet engines because they developed some of the first ones. And that data is there. And people want to ask questions about it. So, again, not a realistic benchmark in a sense that nobody, at least I don't think any Databricks customers are trying to figure out how much Alexander Hamilton was getting paid in 1790. But it's a really challenging benchmark that pushes on some important capabilities. And it's one we could package cleanly enough that like now OpenAI includes it in every new model release as a benchmark that they use. And we just released a new version of it and we're continuing to iterate. Even that is tiny compared to the stuff we're doing internally, where we do have tens of petabytes of workspaces and the environments are dynamic and the answer may actually change day to day. [00:29:38] S1: Another part of benchmarking that we've talked about is governance, which is obviously becoming extremely important to enterprises as they trust the models with more tools, more data, more access. How are you seeing those evals evolve? Do most enterprises also have evals for governance yet alongside these evals that they're curating for accuracy? Or uh is that still an earlier process that many of them are just beginning to embark on?
00:30:06 · Jonathan Frankle
[00:30:06] S0: I don't think anyone's even figured out how to measure this yet. I think it's worth defining governance because it's a like if you want to talk about an enterprise-pilled word that the average AI person doesn't even think about. What access should a model have? What access should anyone have? And this feels like a really simple question initially. Like, you should have access to this code, but not that code. You should have access to this information, but not that financial information, or these Google Docs you have permission to, but not those Google Docs. And then when you get to a company with a thousand people, let alone 10,000 people, the permissions get so complicated. And then also think about I think for anyone out there, the number of times you've made a Google Doc world readable to everybody at your company and have just sent it to a couple of people, but you didn't want to have to keep updating the permissions list, or you've made a document, worst of all, world editable because you didn't want to have to keep updating the list. This is the kind of thing that like you probably have access to hundreds of Google Docs that you have never looked at and never thought about. Agents tend to find that stuff. And so the question of what permissions an agent should have, agents do very creative, very strange things when you give them permissions. Things that can make a ton of sense when you really think about it, but are kind of inexplicable in the moment that you wouldn't have predicted. And so when you think about an enterprise, there are probably hundreds of thousands or millions of individual permissions that you need to keep track of. Even just doing that for humans is complicated. And there are all sorts of solutions. We have our own at Databricks, but lots of governance solutions just to track that globally across all of your different kinds of data, both within Databricks and beyond. Now apply that to agents. You're going to give an agent permission to go and pursue a task. A lot of people say, like, oh, just give the agents your permissions and you're responsible for the agent's actions. Um, the agent is gonna find that doc that you would never have found. Agents work at scale in ways that humans don't. You don't have time to go looking through all the Google Docs you technically have permission to.
00:31:54 · Jonathan Frankle
An agent does and will find something it should not have found, HR-related, legal-related, what have you, that may actually cause some trouble for your business. Like the permissions today are probably not very rigorous as it stands. And then you as a human need to understand what permissions did I give my agent and what could an agent do with those permissions that I would never have thought of. I had a fun example at Databricks where you know I gave an agent access to Jira and a Google Doc. It's like, oh, what could go wrong there? And actually narrowly avoided a really complicated scenario because I didn't think about what data was in both places and what would happen if you combine those two permissions together. Now multiply that across Slack, you know, glean, what have you. And that's just services, not individual Google Docs that you might or might not have access to. Things get messy really quickly. And I think the evolution for evals is not just about did the task get completed, but what was the process? And was that process a good and reasonable process? What permissions did the agent need? What did it access? Should it have accessed that? Does it need internet access? I gave it internet access. There's no like you just have permission to view this one website. There's no read and write permissions for the internet. There's just get requests and post requests. You can write data to the internet really easily without having permission to do that and weird stuff happens.
00:33:04 · Brendan Foody
[00:33:04] S1: It feels like so much of this has come from the fact that agents have improved so much over the last 12 months that enterprises trust them to do more or at least want to. So as we give agents more autonomy to interact with access context, to use tools, that also comes with the risk associated with them and how we build the right guardrails. I'm curious, how has progress of models and agents over the last 12 months compared to your expectations?
00:33:31 · Jonathan Frankle
[00:33:31] S0: I think it's been ahead of my expectations in general, but I tend to always be a little more pessimistic. You ask my friends three years ago. I have a friend who's now at Anthropic where I think he told me two years ago that he thought that by 2026 or 2027, AI was going to be able to address like 50% of all GitHub issues out there. It's like, you're an idiot. It's going to be like 5%. These are really hard tasks. He's totally right. I was a little pessimistic. The way I look at it is more of, you know, this is going to sound dumb, but like roll back the clock to December of 2025. I know it's a very long time ago at this point, and ask yourself, what are the like, what did all the VCs say the big trends in AI were going to be in 2026? And it was things like we're going to deploy agents widely for work, memory is going to be a really important topic, open claw was a hot topic. Like, we're really going to delegate our whole lives to agents. Coding agents were going to be really important. It was just this unbridled optimism about everything exciting that was going to happen. And I was sitting there, and you were sitting there saying the same thing of like quality is going to matter and measuring quality is going to matter, like evals, evals, evals, and evals were going to be the big trend. When I talked to my customers today, I would say the three big trends I hear most or the questions they ask me first are cost. We were in a three-month period of token maxing and leaderboards. And now we're, God, I can't believe I used this word in front of 1,500 people. But I said the era of token maxing is over and the era of value mogging is here. And I'm I'm going to cringe, but I said it anyway. Again, recorded again. Um, the second one is governance. What should an agent have access to? What shouldn't it? And how do you avoid an agent doing really stupid, inexplicable things that actually cause you a lot of risk? And the third is security. I mean, the fact that now we don't even know what code is being written and an agent can pip install anything. And we've seen the number of compromises in pip installations, like that's actually really scaring enterprises. And I'm seeing it cause them to shy away from even using software from startups because you don't even know what to do.