This Week in Startups
00:45:51 · Jonathan Siddarth
[00:45:51] S0: So, I'd say it's, the labs care about the benchmarks because it's good for bragging rights. It's good for recruiting. Who has the best model for coding? Who has the best model for STEM, etcetera? So benchmarks serve some useful purpose. And you can argue that in today's $100,000,000 sort of talent wars, maybe it's closer to 300,000,000 $400,000,000 talent wars, those bragging rights help because the best researchers ideally wanna join the winning team, who's already kind of close to being number one. But the labs care about two things, public benchmarks and private evals. And I would argue that private evals are actually more important because then you're evaluating how well is your model doing relative to the competition on the prompt distribution that you care about. Meaning if you're Google or if you're Beta or if you're OpenAI or Anthropic, the type of queries you might get inside in a coding context might look different from what you might get on the phone in like a general chat assistant context or what a human might put when you're searching through a desktop. So you want to optimize for your own query distribution that your users are testing your model on. So that's why these private evals are helpful. So you need public benchmarks and private evals. The challenge with public benchmarks, Alex, is if you look at coding, for example, there is this good benchmark called SWE bench, which is created by this lab at Princeton. Last year, we went from 2% to 50% in SWE bench. This year, the best models are already north of 60%. We'll probably saturate SWE bench this year. And then where do you go? Clearly, we haven't automated all of software engineering yet. So the benchmarks are getting saturated.
00:47:40 · Alex Wilhelm, Jonathan Siddarth
[00:47:40] S1: Yeah. Can you double click on what you mean by saturated in that context? I think it's an important point. [00:47:44] S0: Yes. So if a model does 90% plus on a benchmark, you kind of aced the test to some degree. And researchers call that saturating a test because now the test no longer tells you whether your models are improving or not, So the obvious answer is you have to create a harder benchmark where the models would have even more headroom to climb, even more room to improve, which is why at Turing, we are actually creating a benchmark for coding that's even harder than SuiteBench. And it gives us deep satisfaction because we managed to have all the models start at zero. [00:48:24] S1: Okay. So everyone's failing 100% of this new... Ah. Okay. Let's talk about private evals because this is a core thing of what Turing offers to its customers. So I'm curious. Let's say that I'm OpenAI. I have a new model. I want you guys to take a look at it. You find a couple of places where it's not quite up to snuff, not quite where we expected it to be. So then do you turn around and say, Hey, guys, here are the places where there are gaps, and then help them solve those issues? Or do you guys just point out, Here's the spot where, you know, you might wanna do some more work?
00:48:54 · Jonathan Siddarth
[00:48:54] S0: So we do both. We evaluate the model and we generate data to help the models improve. But let me take a step back, Alex. And let's look at this landscape, which is super interesting right now. What's happening is as these models have gotten smarter and smarter, the data that's needed to advance the models has become increasingly harder to generate, right? The models are advancing in-depth. They're improving in coding, stem reasoning, etcetera. They're advancing in breadth in multimodality, multilinguality, multi industry, etcetera. And the models are becoming agentic, meaning the models can now execute complex multi step workflows in a real world business context. Now, with the models advancing like this, what's needed in a platform is you don't just need Iron Man. You need the Avengers. Frontier models need frontier data. Frontier data needs a team of frontier humans, right? So we have like PhDs from physics, chemistry, math, biology, expert Olympiad level coders, people who are literally at the pinnacle of their fields. And sometimes you have to have them working together to break the model. For example, in physics, if an expert is evaluating the model in physics to test some theory in physics, you might want to a PhD in physics might ask the question to test a theory. A software engineer might need to build a simulation to test that theory. And then a data scientist might have to analyze the results of that simulation. So you need to daisy chain really smart humans together to generate data.
00:50:32 · Alex Wilhelm
[00:50:32] S1: Actually, Jonathan, can I talk to you about that? Because what you're describing to me sounds like very interesting tests. Having people with different domain expertises work together to find places where the model might have a weakness or might not be able to answer something. And then you say that they're generating data. Is the data they're generating simply where the LLM in question misses or doesn't meet expectations? Because I thought this was more like, here's new information for the LLN to be trained on versus looking at a model that's already been trained. And then... Does that make sense? I feel like I may have had this backwards.
00:51:05 · Jonathan Siddarth
[00:51:05] S0: So that's a great question, Alex. So the way these models are trained is there's a step called pre training, where you basically feed the model gobs and gobs of data. The war in pre training is mostly over. Almost all the models are trained on the same subset of the internet. Now the battlefield is post training. And there's a new field called mid training, which is I can get into that later. But with post training, what you do is you have these human experts create question answer pairs where a software engineer might ask a question like, hey, how do you create an app that connects dog walkers to dogs? And can you write it in both Swift and Kotlin for Android, right? And now the system has to generate the app. So that's an example of a supervised fine tuning data set where you give it a prompt and a completion, and the model learns how respond to that prompt, right? So you need experts in different fields. Now, reason this has gotten harder is two years ago, a very low skilled contractor was capable of creating tokens that could have advanced the model. Now, because the floor has gone up, now you need PhDs from Stanford, Berkeley, MIT to figure out where the models break. You first have to break the model. You have to ask a question that literally stumps the model. And then the humans create good question pairs of questions and answers that you then feed into the model for fine tuning, and then the model learns.
00:52:33 · Alex Wilhelm, Jonathan Siddarth
[00:52:33] S1: Ah, it's the question and answer pairs that generates the data for the model. Okay. Now that makes good sense to me. So it sounds though like what you've done is back to your Avengers: Iron Man point. Just got together like a, like a super set of nerds. And basically you're applying the smartest humans to find the flaws in the most advanced LLMs. What happens, Jonathan, when we don't have PhDs that can ask questions that the model can't answer? Because it seems to me like we've raised the bar, but we're raising it towards a ceiling. [00:53:03] S0: Yeah. Yeah. I mean, I've oversimplified it a little bit. So I would say like... So one example I gave is you give question answer pairs. There's another type of data that you generate for reinforcement learning where the humans are generating questions and verifiers. They're not generating the solution. They're generating a way to verify whether the solution is correct or not, which you can do with certain fields like coding, math, hard sciences. You can do that, right? Write a program to sort numbers in Python. You can write test cases to verify whether the program that you wrote is correct or not. [00:53:42] S1: But you're not telling them every single step along the way. You're just saying, I can verify that what you've done, the work you've done, does generate an acceptable answer.