00:02:44 · Thore Graepel
[00:02:44] S1: Yeah. That's exactly right. We call this actually the original sin of the transformer architecture, that everything is enmeshed in the weights, you just can't get it out. Even if I gave you a trillion parameters of a model now, the only way to ask it is to prompt it. And so, fundamentally, our architectural decision is to separate these things, the knowledge on which the reasoning operates and the reasoning process. And that unlocks a number of desirable properties for for AI. Because if you explicitly represent your knowledge, then you can inspect that knowledge. You can see what's actually there. You can edit the knowledge. You can remove harmful knowledge or knowledge you're not supposed to have, or you can add knowledge, and you can access control it. You know, you can give access to different people or system depending on on the situation. And these, of course, are the advantages that we knew all along with databases. We had just lost them because they're now... You know, the knowledge is now locked up in the transformer. But on the other side, we have the reasoning process. And now if that process acts on the knowledge, now we can improve that reasoning process separately. In reinforcement learning terms, we can learn a good reasoning policy that becomes better and better at reasoning as it reasons about more problems. And that means we have these two new scaling axes. One, growing the knowledge explicitly and the other one growing the reasoning capability through that learning process.
00:04:21 · Sebastian Johnson, Thore Graepel
[00:04:21] S0: And so at Metis reasoning is is is the approach to build both separately, to build the reasoning capability and the sort of the the knowledge base, the database there, and to try and then kind of get value out of, you know, applying one to the other? Or are you focusing, you know, primarily on the reasoning process itself? [00:04:39] S1: Yeah. No, it's in fact both. And they interact in very rich ways. So you could imagine, suppose the system is confronted with a new question for the first time, and then it reasons through that question and gives the answer together with the reasons for why it gave that answer. That is crucial. And now, of course, we don't throw that away. We can store the answer together with its provenance and all the associated evidence on the knowledge side in in the knowledge graph, if you like. And that means that if that answer is required another time because the user asks another time or it comes up in some other reasoning process as a sub problem, then the system can now just access the cache knowledge in that knowledge store. But that knowledge store isn't just, you know, a bunch of tokens. That knowledge store now is certified knowledge that has already been verified by the system. So through that mechanism, it can become orders of magnitude faster.
00:05:44 · Sebastian Johnson
[00:05:44] S0: And in the paper, one of the things that really... I I thought was really interesting is that you you were talking about AlphaGo. You you were on the AlphaGo team, what, just over ten years ago, you know, with the famous Move 37. Now in the paper, you're kind of arguing that Move 37 was more reasoning, right? It wasn't the similar sort of architecture that we have today in AI. It was a different thing. Can you explain a bit about that? How it's different to modern LLMs? And then I guess like where we went wrong potentially, like why did we move from AlphaGo in this amazing moment at Move 37 to the LLM architecture where we kind of lost reasoning?
00:06:23 · Thore Graepel
[00:06:23] S1: Yes. Yeah. It's a great point. So, going back, ten years in time, so to speak, what what does the AlphaGo architecture actually look like? It has these two sides to it. Yes. There are neural networks. They are very important. There were two of them. The policy network decides what are promising moves in a given position, and the value network is able to judge if a position is good for black or for white. But those were not the only components. They were combined with a tree search, you know, a game tree search. And that is really the the knowledge structure that I've been talking about. The structure that holds the current knowledge of the system about the game or about the game state. And this this tree structure is searched and expanded by the reasoning process and uses the neural networks to interface with the world, you know, to understand, you know, things about about the the go position. And it's really these two systems together that produce the playing strength of AlphaGo. And, you know, in the words of the... Of psychologist Kahneman, these are system one and system two. System one is the quick intuitive pattern matching system, and system two is the more elaborate reasoning systems that thinks forward into the future. And move 37, coming back to that question, was this very surprising move that no human master would have made. You know, the one where Michael Redmond, the commentator said, oh, this must be a mistake. Right? It's how bizarre it looked to good Go players. And we know... Although people think that was the... Like, the gut feeling of the of the computer, It really wasn't. We know that the policy net assigned a very low probability of one in 10,000 to that move so that the policy net wouldn't have played it. It was the reasoning process, the process that looked at the consequences of playing that move, which eventually dared to play that unusual move. And I think there's a really interesting lesson here that if we want to break out of the current patterns of knowledge to generate truly new knowledge, then we need, those reasoning processes, those explicit reasoning processes.
00:08:51 · Sebastian Johnson, Thore Graepel
[00:08:51] S0: And why do you think, you know, given how impactful that AlphaGo moment was in Move 37, We then had LLM, the LLM architecture come up and be this next big wave of AI. Why do you think, you know, researchers went in that direction and built the sort of all in one knowledge and reasoning as opposed to, you know, to your point, system one and system two? [00:09:13] S1: Yeah. So I think if you now... If you go forward from AlphaGo and say to the JetGPT moment, you will see a similarity. You know, our policy net already predicted the next move, and large language models predict the next token. So that part of of the AlphaGo system was in fact further developed and has taken us where we are. But the other side, you know, the reasoning side was clearly much much less developed. There were some attempts to do some kind of search, of course, in the in the space of tokens. But I think there's this additional idea that's needed here, which is to move from token to proposition. You know, a token, the the basic unit of intelligence, as they say nowadays in large language models, is just some little semantic bit that doesn't say anything about the world. A token is never right or wrong. Right? It's just a token. But when we talk about knowledge and reasoning, we need to speak about propositions. A proposition is a statement about the world that can be true or false or about which we can have some kind of belief. And that is the natural unit in which reasoning happens. And that is where we can get to reasoning that can advance our knowledge about the real world.
00:10:39 · Sebastian Johnson
[00:10:39] S0: And why is that important? You know, why is reasoning really important? And like, what are some of the I guess there's two things I want to talk about. The issues that we have with LLMs being a black box and whether that's, hallucination and kind of lack of auditability. And I think the point that was really interesting was actually the economic point that is then made. So let's just focus on like what are the issues with the current end the architecture and what are the issues that we're seeing from that today? And then let's talk about, well, okay, if we get reasoning right, what are the economic impacts that can be unlocked?