Scaling Europe
00:06:23 · Thore Graepel
[00:06:23] S1: Yes. Yeah. It's a great point. So, going back, ten years in time, so to speak, what what does the AlphaGo architecture actually look like? It has these two sides to it. Yes. There are neural networks. They are very important. There were two of them. The policy network decides what are promising moves in a given position, and the value network is able to judge if a position is good for black or for white. But those were not the only components. They were combined with a tree search, you know, a game tree search. And that is really the the knowledge structure that I've been talking about. The structure that holds the current knowledge of the system about the game or about the game state. And this this tree structure is searched and expanded by the reasoning process and uses the neural networks to interface with the world, you know, to understand, you know, things about about the the go position. And it's really these two systems together that produce the playing strength of AlphaGo. And, you know, in the words of the... Of psychologist Kahneman, these are system one and system two. System one is the quick intuitive pattern matching system, and system two is the more elaborate reasoning systems that thinks forward into the future. And move 37, coming back to that question, was this very surprising move that no human master would have made. You know, the one where Michael Redmond, the commentator said, oh, this must be a mistake. Right? It's how bizarre it looked to good Go players. And we know... Although people think that was the... Like, the gut feeling of the of the computer, It really wasn't. We know that the policy net assigned a very low probability of one in 10,000 to that move so that the policy net wouldn't have played it. It was the reasoning process, the process that looked at the consequences of playing that move, which eventually dared to play that unusual move. And I think there's a really interesting lesson here that if we want to break out of the current patterns of knowledge to generate truly new knowledge, then we need, those reasoning processes, those explicit reasoning processes.
00:08:51 · Sebastian Johnson, Thore Graepel
[00:08:51] S0: And why do you think, you know, given how impactful that AlphaGo moment was in Move 37, We then had LLM, the LLM architecture come up and be this next big wave of AI. Why do you think, you know, researchers went in that direction and built the sort of all in one knowledge and reasoning as opposed to, you know, to your point, system one and system two? [00:09:13] S1: Yeah. So I think if you now... If you go forward from AlphaGo and say to the JetGPT moment, you will see a similarity. You know, our policy net already predicted the next move, and large language models predict the next token. So that part of of the AlphaGo system was in fact further developed and has taken us where we are. But the other side, you know, the reasoning side was clearly much much less developed. There were some attempts to do some kind of search, of course, in the in the space of tokens. But I think there's this additional idea that's needed here, which is to move from token to proposition. You know, a token, the the basic unit of intelligence, as they say nowadays in large language models, is just some little semantic bit that doesn't say anything about the world. A token is never right or wrong. Right? It's just a token. But when we talk about knowledge and reasoning, we need to speak about propositions. A proposition is a statement about the world that can be true or false or about which we can have some kind of belief. And that is the natural unit in which reasoning happens. And that is where we can get to reasoning that can advance our knowledge about the real world.
00:10:39 · Sebastian Johnson
[00:10:39] S0: And why is that important? You know, why is reasoning really important? And like, what are some of the I guess there's two things I want to talk about. The issues that we have with LLMs being a black box and whether that's, hallucination and kind of lack of auditability. And I think the point that was really interesting was actually the economic point that is then made. So let's just focus on like what are the issues with the current end the architecture and what are the issues that we're seeing from that today? And then let's talk about, well, okay, if we get reasoning right, what are the economic impacts that can be unlocked?
00:11:09 · Thore Graepel
[00:11:09] S1: Yeah. So the problems you mentioned, some of them, there are hallucinations. The systems are intrinsically stochastic, so they usually don't actually max out at 100% in... Even in benchmarks. They, you know, they they don't typically get all the questions right all the time. So that's one of the problems. But possibly the bigger problem is the trust problem because they are black boxes, and we cannot see why they came up with the answer that they did. And, you know, you might think, oh, but we have chain of thought reasoning, and we can inspect the reasoning chain. But first of all, that reasoning chain isn't the actual causal trace that led to the answer. Often... And often, it it looks very confused. Have you ever seen a desperate large language model producing tokens in the chain of thought traces where it says, oh my god. The user wants this. I have to be quick and get the answer. You know, it it just isn't what you would call a reasoning trace of a of a proper reasoning process. And the the problem is, of course, in critical applications. You know, when you have a chatbot, these things don't matter too much. But when this is a system that does medical diagnosis or critical engineering or is an air traffic control system or you know controls robots that are very powerful entities and you know can can do harm if they're not controlled in the right way then we need systems in where we can understand why they come to the conclusions that they do, why they make this the decisions that they do. And in the architecture that we're proposing, this is entirely transparent. You can see how the system reasoned about it. You can see what sources of evidence it used, how it judged those sources of evidence, how it synthesized everything into an answer. And I believe that's what's needed if we want to apply AI in all those critical domains where it actually matters.
00:13:20 · Sebastian Johnson
[00:13:20] S0: Yeah. And we... When when you look at the proposal, think there's three core components you're putting forward, which I think reached kind of what you just covered, a record of what's established and what, you know, degree of certainty. I think you you described it as a persistent epistemic state, the judgment system itself, and this sort of like objective. Is that right? Can you kind of, I guess, explain a bit about the the the core components of the proposal?
00:13:45 · Thore Graepel
[00:13:45] S1: Yes. Exactly. So, first of all, the key thing is that the system actually maintains beliefs about what is the case, you know, what what is its current state of knowledge. And that, again, could be a scientist who has some beliefs about the scientific question or it could be a robot a robot that needs to represent what its current situation is and the environment and task in front of it. So it's a very general state notion. And then what is reasoning? Reasoning is the application of moves to this state to advance that state. So if I have... Because have some set of beliefs at the moment, I can now apply a move and say, oh, why don't I take my problem and decompose it into three subproblems? Now I've advanced my state of knowledge, and now I can tackle these subproblems. I can pull it in a bit of evidence and maybe resolve some uncertainty. And the key thing, of course, is that we need to understand what is the goal of this process, what is the objective that drives it, and the objective that drives it is the reduction in uncertainty. So we need to maintain our beliefs using the language of probability theory and we track if through the actions that we execute the reasoning actions our uncertainty is in fact reduced. And if that's the case then we're reasoning in the right direction and that is our measure of success. And then as the system improves over time that also serves as the reward signal because what is good reasoning? Good reasoning is when you can use very few and clever moves to reduce your uncertainty. You know there's often many different strategies that can lead to a solution and a good reasoner would find the shortest path or the least expensive path. You can even think about it in terms of bits per bug, know, you try to get as reason with in the cheapest possible way to get the maximum, reduction in uncertainty at the minimal price.