This Week in Startups
00:45:14 · Alex Wilhelm
[00:45:14] S1: Okay. So let's talk about the benchmarks because there's been some commentary. I think Apple had a paper that came out, and they said that, you know, one of the problems we have with a lot of the benchmarks that everyone likes to shot out their new LLM and put them up against and have the charts is that there's data contamination issues. There's overfitting. So based on what you just said about trying to solve real world problems and also the fact that we know that these benchmarks are getting a little bit, I don't know, dicey to use, is the standard way that companies announce how their LLMs perform an effective form of evaluation, or is that mostly window dressing to get more social media hits because you're one point higher on one test or another?
00:45:51 · Jonathan Siddarth
[00:45:51] S0: So, I'd say it's, the labs care about the benchmarks because it's good for bragging rights. It's good for recruiting. Who has the best model for coding? Who has the best model for STEM, etcetera? So benchmarks serve some useful purpose. And you can argue that in today's $100,000,000 sort of talent wars, maybe it's closer to 300,000,000 $400,000,000 talent wars, those bragging rights help because the best researchers ideally wanna join the winning team, who's already kind of close to being number one. But the labs care about two things, public benchmarks and private evals. And I would argue that private evals are actually more important because then you're evaluating how well is your model doing relative to the competition on the prompt distribution that you care about. Meaning if you're Google or if you're Beta or if you're OpenAI or Anthropic, the type of queries you might get inside in a coding context might look different from what you might get on the phone in like a general chat assistant context or what a human might put when you're searching through a desktop. So you want to optimize for your own query distribution that your users are testing your model on. So that's why these private evals are helpful. So you need public benchmarks and private evals. The challenge with public benchmarks, Alex, is if you look at coding, for example, there is this good benchmark called SWE bench, which is created by this lab at Princeton. Last year, we went from 2% to 50% in SWE bench. This year, the best models are already north of 60%. We'll probably saturate SWE bench this year. And then where do you go? Clearly, we haven't automated all of software engineering yet. So the benchmarks are getting saturated.
00:47:40 · Alex Wilhelm, Jonathan Siddarth
[00:47:40] S1: Yeah. Can you double click on what you mean by saturated in that context? I think it's an important point. [00:47:44] S0: Yes. So if a model does 90% plus on a benchmark, you kind of aced the test to some degree. And researchers call that saturating a test because now the test no longer tells you whether your models are improving or not, So the obvious answer is you have to create a harder benchmark where the models would have even more headroom to climb, even more room to improve, which is why at Turing, we are actually creating a benchmark for coding that's even harder than SuiteBench. And it gives us deep satisfaction because we managed to have all the models start at zero. [00:48:24] S1: Okay. So everyone's failing 100% of this new... Ah. Okay. Let's talk about private evals because this is a core thing of what Turing offers to its customers. So I'm curious. Let's say that I'm OpenAI. I have a new model. I want you guys to take a look at it. You find a couple of places where it's not quite up to snuff, not quite where we expected it to be. So then do you turn around and say, Hey, guys, here are the places where there are gaps, and then help them solve those issues? Or do you guys just point out, Here's the spot where, you know, you might wanna do some more work?
00:48:54 · Jonathan Siddarth
[00:48:54] S0: So we do both. We evaluate the model and we generate data to help the models improve. But let me take a step back, Alex. And let's look at this landscape, which is super interesting right now. What's happening is as these models have gotten smarter and smarter, the data that's needed to advance the models has become increasingly harder to generate, right? The models are advancing in-depth. They're improving in coding, stem reasoning, etcetera. They're advancing in breadth in multimodality, multilinguality, multi industry, etcetera. And the models are becoming agentic, meaning the models can now execute complex multi step workflows in a real world business context. Now, with the models advancing like this, what's needed in a platform is you don't just need Iron Man. You need the Avengers. Frontier models need frontier data. Frontier data needs a team of frontier humans, right? So we have like PhDs from physics, chemistry, math, biology, expert Olympiad level coders, people who are literally at the pinnacle of their fields. And sometimes you have to have them working together to break the model. For example, in physics, if an expert is evaluating the model in physics to test some theory in physics, you might want to a PhD in physics might ask the question to test a theory. A software engineer might need to build a simulation to test that theory. And then a data scientist might have to analyze the results of that simulation. So you need to daisy chain really smart humans together to generate data.
00:50:32 · Alex Wilhelm
[00:50:32] S1: Actually, Jonathan, can I talk to you about that? Because what you're describing to me sounds like very interesting tests. Having people with different domain expertises work together to find places where the model might have a weakness or might not be able to answer something. And then you say that they're generating data. Is the data they're generating simply where the LLM in question misses or doesn't meet expectations? Because I thought this was more like, here's new information for the LLN to be trained on versus looking at a model that's already been trained. And then... Does that make sense? I feel like I may have had this backwards.
00:51:05 · Jonathan Siddarth
[00:51:05] S0: So that's a great question, Alex. So the way these models are trained is there's a step called pre training, where you basically feed the model gobs and gobs of data. The war in pre training is mostly over. Almost all the models are trained on the same subset of the internet. Now the battlefield is post training. And there's a new field called mid training, which is I can get into that later. But with post training, what you do is you have these human experts create question answer pairs where a software engineer might ask a question like, hey, how do you create an app that connects dog walkers to dogs? And can you write it in both Swift and Kotlin for Android, right? And now the system has to generate the app. So that's an example of a supervised fine tuning data set where you give it a prompt and a completion, and the model learns how respond to that prompt, right? So you need experts in different fields. Now, reason this has gotten harder is two years ago, a very low skilled contractor was capable of creating tokens that could have advanced the model. Now, because the floor has gone up, now you need PhDs from Stanford, Berkeley, MIT to figure out where the models break. You first have to break the model. You have to ask a question that literally stumps the model. And then the humans create good question pairs of questions and answers that you then feed into the model for fine tuning, and then the model learns.