Scaling Europe
00:11:09 · Thore Graepel
[00:11:09] S1: Yeah. So the problems you mentioned, some of them, there are hallucinations. The systems are intrinsically stochastic, so they usually don't actually max out at 100% in... Even in benchmarks. They, you know, they they don't typically get all the questions right all the time. So that's one of the problems. But possibly the bigger problem is the trust problem because they are black boxes, and we cannot see why they came up with the answer that they did. And, you know, you might think, oh, but we have chain of thought reasoning, and we can inspect the reasoning chain. But first of all, that reasoning chain isn't the actual causal trace that led to the answer. Often... And often, it it looks very confused. Have you ever seen a desperate large language model producing tokens in the chain of thought traces where it says, oh my god. The user wants this. I have to be quick and get the answer. You know, it it just isn't what you would call a reasoning trace of a of a proper reasoning process. And the the problem is, of course, in critical applications. You know, when you have a chatbot, these things don't matter too much. But when this is a system that does medical diagnosis or critical engineering or is an air traffic control system or you know controls robots that are very powerful entities and you know can can do harm if they're not controlled in the right way then we need systems in where we can understand why they come to the conclusions that they do, why they make this the decisions that they do. And in the architecture that we're proposing, this is entirely transparent. You can see how the system reasoned about it. You can see what sources of evidence it used, how it judged those sources of evidence, how it synthesized everything into an answer. And I believe that's what's needed if we want to apply AI in all those critical domains where it actually matters.
00:13:20 · Sebastian Johnson
[00:13:20] S0: Yeah. And we... When when you look at the proposal, think there's three core components you're putting forward, which I think reached kind of what you just covered, a record of what's established and what, you know, degree of certainty. I think you you described it as a persistent epistemic state, the judgment system itself, and this sort of like objective. Is that right? Can you kind of, I guess, explain a bit about the the the core components of the proposal?
00:13:45 · Thore Graepel
[00:13:45] S1: Yes. Exactly. So, first of all, the key thing is that the system actually maintains beliefs about what is the case, you know, what what is its current state of knowledge. And that, again, could be a scientist who has some beliefs about the scientific question or it could be a robot a robot that needs to represent what its current situation is and the environment and task in front of it. So it's a very general state notion. And then what is reasoning? Reasoning is the application of moves to this state to advance that state. So if I have... Because have some set of beliefs at the moment, I can now apply a move and say, oh, why don't I take my problem and decompose it into three subproblems? Now I've advanced my state of knowledge, and now I can tackle these subproblems. I can pull it in a bit of evidence and maybe resolve some uncertainty. And the key thing, of course, is that we need to understand what is the goal of this process, what is the objective that drives it, and the objective that drives it is the reduction in uncertainty. So we need to maintain our beliefs using the language of probability theory and we track if through the actions that we execute the reasoning actions our uncertainty is in fact reduced. And if that's the case then we're reasoning in the right direction and that is our measure of success. And then as the system improves over time that also serves as the reward signal because what is good reasoning? Good reasoning is when you can use very few and clever moves to reduce your uncertainty. You know there's often many different strategies that can lead to a solution and a good reasoner would find the shortest path or the least expensive path. You can even think about it in terms of bits per bug, know, you try to get as reason with in the cheapest possible way to get the maximum, reduction in uncertainty at the minimal price.
00:16:02 · Sebastian Johnson
[00:16:02] S0: Yes. It's... Yeah. It's really interesting. There's a bit where you... I think in the paper, you you talk about where the analogy with AlphaGo sort of ends, I think. It's almost like the the the biggest challenge that you're facing is applying this sort of like very, very scientific approach. And this... I think it's, I guess, very scientific reasoning process, which works perfectly for a game like AlphaGo where you always know where you stand. So you have all the information on the board, you know your position. There are very fixed rules about what you can and can't do. And the value as well is, like, very... Is very clear. Right? They they... You know, you wanna progress, you wanna win. Right? So for for the reasoning in that game, you can kind of see a really clear structure. How do we try and rebuild that for, like, a much more general intelligence, do you think?
00:16:48 · Thore Graepel
[00:16:48] S1: Yeah. Yeah. You're quite right. So, of course, a game like Go or chess is a very constrained space. It's not stochastic. It's not an open world. And, clearly, the the general reasoning problem is harder. But we do have tools to tackle it. For example, in the game of Go, there is no uncertainty about the board state. Or if I make a move, I know exactly what's gonna happen as a consequence of making that move in in the next step. And these things are not the case in the real world, but we can use the calculus of probability to quantify the uncertainty that arises. That's number one. And number two is that in fact we can use large language models for various tasks that arise within the reasoning. So we're not at all saying that you know current AI is useless or something like that. No, no, it is super useful. We just want to direct it at more specific tasks which are usually related to some form of semantic interpretation. So how how would we use a large language model in this context? I'll give you an example. Our system... If you ask a large language model, what is the capital of France? It'll just tell you Paris, and it'll usually be right, but there will be no trace and, you know, where does the answer come from? From the weights? Where do the weights come from? From the training corpus. So there's not much more you get than the answer. Metis reasoning, in contrast, would go about it as follows. It would first ask the question, oh, does France even have a capital? Is France the kind of thing that can have a capital? Right? It's kind of questioning the assumptions. And then it doesn't jump to a conclusion, but it puts forward hypotheses. And we would... Might very well ask a large language model to help us formulate the hypotheses because nothing can go wrong here. Even if it proposes a nonsensical thing like Mickey Mouse as the answer, we can always root it out later by looking at evidence. And so now we have a number of hypotheses what the capital of France might be, and now we pull in external evidence from Wikipedia or from some other source and test these hypotheses against the evidence. And then, of course, our system would probably get get you Paris if that's the framing that... Where where that is the correct question. But it wouldn't have relied at all on on that information being within an LLM.
00:19:26 · Thore Graepel, Sebastian Johnson
It's just using the LLM to judge the hypotheses against the evidence, and that makes it so powerful. But then, of course, next time, it will save time because it will have stored that fact together with the evidence, and it will give you an instant answer the next time you ask it. That's why it doesn't hallucinate, you see, because it doesn't rely on the LLM fantasizing about the answer. It just uses the LLM to interpret the evidence. [00:19:57] S0: And I'm cautious. We're out of time. Wanna ask one more question, which is that you've you've kind of come out of stealth. You've released this white paper. At the moment, it is... You know, you're making an argument and you have a strong theory. What's next? You know, are building some sort of reasoning model itself? Are you going to start trying to benchmark or prove that it has no hallucinations? Like, what's next in terms of, you know, what comes after the paper?