This Week in Startups
00:43:46 · Alex Wilhelm, Jonathan Siddarth
[00:43:46] S1: I'm very impressed. Also, we're both in our home offices today. And I think this just goes to show that remote work, not entirely dead, even though everyone seems to claim that it is. I'm glad that you're here. So sorry for folks who are less aware of what LLM evaluation is. Donathan, can you just give us the working definition from your side of the fence? [00:44:04] S0: Yeah. So, it's really important to benchmark LLMs to figure out where they are good at, where are gaps in the models so that you can generate data that could help the LLMs get better at those specific tasks. One big challenge, Alex, that I see today in the overall evaluation and benchmarking market is that a lot of the evaluations are somewhat academic and somewhat synthetic and don't connect to real world applications or real world use. Ideally, you want AGI to progress in a way where the models get better at useful tasks. So when we evaluate models, the three dimensions to look at are complexity. Are you evaluating them on really hard tasks? Real world use. Are you evaluating them on something that a human would actually care about? And third is diversity. You want a wide breadth of test cases that you're evaluating the models on. It's such an exciting space. And I think of evaluation as step zero of data generation. That's why we do both evaluation and data generation.
00:45:14 · Alex Wilhelm
[00:45:14] S1: Okay. So let's talk about the benchmarks because there's been some commentary. I think Apple had a paper that came out, and they said that, you know, one of the problems we have with a lot of the benchmarks that everyone likes to shot out their new LLM and put them up against and have the charts is that there's data contamination issues. There's overfitting. So based on what you just said about trying to solve real world problems and also the fact that we know that these benchmarks are getting a little bit, I don't know, dicey to use, is the standard way that companies announce how their LLMs perform an effective form of evaluation, or is that mostly window dressing to get more social media hits because you're one point higher on one test or another?
00:45:51 · Jonathan Siddarth
[00:45:51] S0: So, I'd say it's, the labs care about the benchmarks because it's good for bragging rights. It's good for recruiting. Who has the best model for coding? Who has the best model for STEM, etcetera? So benchmarks serve some useful purpose. And you can argue that in today's $100,000,000 sort of talent wars, maybe it's closer to 300,000,000 $400,000,000 talent wars, those bragging rights help because the best researchers ideally wanna join the winning team, who's already kind of close to being number one. But the labs care about two things, public benchmarks and private evals. And I would argue that private evals are actually more important because then you're evaluating how well is your model doing relative to the competition on the prompt distribution that you care about. Meaning if you're Google or if you're Beta or if you're OpenAI or Anthropic, the type of queries you might get inside in a coding context might look different from what you might get on the phone in like a general chat assistant context or what a human might put when you're searching through a desktop. So you want to optimize for your own query distribution that your users are testing your model on. So that's why these private evals are helpful. So you need public benchmarks and private evals. The challenge with public benchmarks, Alex, is if you look at coding, for example, there is this good benchmark called SWE bench, which is created by this lab at Princeton. Last year, we went from 2% to 50% in SWE bench. This year, the best models are already north of 60%. We'll probably saturate SWE bench this year. And then where do you go? Clearly, we haven't automated all of software engineering yet. So the benchmarks are getting saturated.
00:47:40 · Alex Wilhelm, Jonathan Siddarth
[00:47:40] S1: Yeah. Can you double click on what you mean by saturated in that context? I think it's an important point. [00:47:44] S0: Yes. So if a model does 90% plus on a benchmark, you kind of aced the test to some degree. And researchers call that saturating a test because now the test no longer tells you whether your models are improving or not, So the obvious answer is you have to create a harder benchmark where the models would have even more headroom to climb, even more room to improve, which is why at Turing, we are actually creating a benchmark for coding that's even harder than SuiteBench. And it gives us deep satisfaction because we managed to have all the models start at zero. [00:48:24] S1: Okay. So everyone's failing 100% of this new... Ah. Okay. Let's talk about private evals because this is a core thing of what Turing offers to its customers. So I'm curious. Let's say that I'm OpenAI. I have a new model. I want you guys to take a look at it. You find a couple of places where it's not quite up to snuff, not quite where we expected it to be. So then do you turn around and say, Hey, guys, here are the places where there are gaps, and then help them solve those issues? Or do you guys just point out, Here's the spot where, you know, you might wanna do some more work?
00:48:54 · Jonathan Siddarth
[00:48:54] S0: So we do both. We evaluate the model and we generate data to help the models improve. But let me take a step back, Alex. And let's look at this landscape, which is super interesting right now. What's happening is as these models have gotten smarter and smarter, the data that's needed to advance the models has become increasingly harder to generate, right? The models are advancing in-depth. They're improving in coding, stem reasoning, etcetera. They're advancing in breadth in multimodality, multilinguality, multi industry, etcetera. And the models are becoming agentic, meaning the models can now execute complex multi step workflows in a real world business context. Now, with the models advancing like this, what's needed in a platform is you don't just need Iron Man. You need the Avengers. Frontier models need frontier data. Frontier data needs a team of frontier humans, right? So we have like PhDs from physics, chemistry, math, biology, expert Olympiad level coders, people who are literally at the pinnacle of their fields. And sometimes you have to have them working together to break the model. For example, in physics, if an expert is evaluating the model in physics to test some theory in physics, you might want to a PhD in physics might ask the question to test a theory. A software engineer might need to build a simulation to test that theory. And then a data scientist might have to analyze the results of that simulation. So you need to daisy chain really smart humans together to generate data.
00:50:32 · Alex Wilhelm
[00:50:32] S1: Actually, Jonathan, can I talk to you about that? Because what you're describing to me sounds like very interesting tests. Having people with different domain expertises work together to find places where the model might have a weakness or might not be able to answer something. And then you say that they're generating data. Is the data they're generating simply where the LLM in question misses or doesn't meet expectations? Because I thought this was more like, here's new information for the LLN to be trained on versus looking at a model that's already been trained. And then... Does that make sense? I feel like I may have had this backwards.