AI and the Future of Work: Artificial Intelligence in the Workplace, Business, Ethics, HR, and IT for AI Enthusiasts, Leaders and Academics
00:17:12 · Dan Turchin, George Sivulka
[00:17:12] S1: But a big part of that custom process involves simulating insights from external sources, obviously. So the the the rag search would restrict it to some, you know, known sanctioned content. Presumably to do this research, you want to extend it to content that hasn't been sanctioned. Right? [00:17:29] S0: A 100%. If you if you actually think about how RAG works, it's ultimately, you know, taking a a a group of of sanctioned content or a group of whatever contents you'd like and giving you just the most similar, the most relevant results and feeding that to a large language model to answer a question. What really matters and what matrix does is it'll take in entire worlds of information, that sanctioned content, but also other things as well. And it'll say, okay. Hey. It's it's... We're gonna give you, not just the relevant results, but just everything that's even remotely related. Package it into something that's a bit more kind of unbiased and and level headed in in its analysis of of the, of whatever it's been given and then present that to the user.
00:17:29 · George Sivulka, Dan Turchin
[00:17:29] S0: A 100%. If you if you actually think about how RAG works, it's ultimately, you know, taking a a a group of of sanctioned content or a group of whatever contents you'd like and giving you just the most similar, the most relevant results and feeding that to a large language model to answer a question. What really matters and what matrix does is it'll take in entire worlds of information, that sanctioned content, but also other things as well. And it'll say, okay. Hey. It's it's... We're gonna give you, not just the relevant results, but just everything that's even remotely related. Package it into something that's a bit more kind of unbiased and and level headed in in its analysis of of the, of whatever it's been given and then present that to the user. [00:18:11] S1: Makes sense. So it's not awesome that I get to hang out with computational neuroscientists. So I gotta ask you a question here. So you use the term meta learning and you talk about teaching machines... Teaching them how to learn. I love the concept. I often though bristle at these analogies where we liken machine learning models to human brains. You know, the concept of learning, it warrants discussion. But really, we're teaching machines to pattern match. And, you know, maybe the philosophical side of that computational neuroscience background, what does it mean to learn? And are we in fact, based on that definition, are we in fact teaching machines to learn?
00:18:54 · George Sivulka
[00:18:54] S0: I love philosophical questions. Whenever I think about... I I mean, even our name, Hebbian Learning, relates to the kind of a way that you can train neural networks, but also a way that the biological brain learns, which is called Hebbian learning. Right? And if you think about what it means... What that actually means in practice, when you're when you're training a model, you are giving it access to the world's information, and and it's predicting the most likely next sequence. And you see learning as an emergent property from that. So forget about learning during training, but learning during inference is is more of what this is all about, wherein you could give, you know, any LLM a variety of examples, and it can then take those examples and generalize them to new examples that's never before seen or that it's never... It's that aren't in its training dataset. And if you think about kind of what what humans do or or what animals do or biological systems, you know, we'll learn very early on, hey. If I pick something up and I let it go midair, it will fall. And I'll I'll do that with maybe, you know, a block, then maybe I'll do that with a spoon. Maybe I'll do that with an entire plate of food. Very common, you know, children, toddlers will will experience this. And then we'll know... Well, you know that, hey. If I pick up the TV remote or if I pick up, you know, mom or dad's computer or phone, if I drop it, it's also going to break and shatter just like that plate or that gravity is is something that we kind of pick up on inherently. And people had a conception of this too far before Isaac Newton. Right? But it but it was it was almost an innate understanding of the world. And you see that these models actually are picking up and understanding not only, things that are in their training data, but that they actually are generalizing pretty well to things that are completely outside of training data. And that that poses the argument, and poses the, really, the conclusion that these models are building what's called a world model. They understand how the world works. They understand logic. And it's quite beautiful that language actually is the only thing that is really needed to understand richly how the world works.
00:20:54 · George Sivulka, Dan Turchin
We see these models generalized to principles, like even physics principles, just based on all of the language that humanity has created. And so I I would definitely call that learning. And I think that, I think that these models are learning actually in a very similar way to how humans learn, even if they're... The way that they compute is very different than how biological brains compute. [00:21:16] S1: It's that last clause I think is really important to capture. Like, anything that involves prediction tasks or computing, computers are really, really efficient at doing that thing over and over again really reliably. But when it comes to, your point, I think you described it really well, those world models or understanding the physical attributes of the world, think that's where the brain is just so incredibly power efficient. And as humans, we learn from so few examples, Like, I guess at some point, can make the analogy hold up, but we just fundamentally learn in ways that are so different from what machines do when they, quote, learn. So I would like to test the, you know, the the boundaries of the analogy.
00:21:59 · George Sivulka
[00:21:59] S0: Sure. I'd say that, the biological brains, especially human brains are fundamentally very random. They're stochastic machines. Right? And, you know, you could give any amount of humans the same problem and we'll probably come up with tons of different ways of thinking through it or predicting. And if you look at large language models, they're still, you know, depending on on, the temperature or the randomness that you inject into their inference. They'll they'll actually create a variety of different possibilities as well. But I think humans have a benefit of being stochastic. Right? If you think about maybe the world's best investors or even the world's best lawyers, this this idea of creativity, or of the randomness of the human brain actually ends up being a superpower. It it... Take what the market thinks and turn it on its head. Right? Do the contrarian thing. These are all fundamental truths, that make great investors great or great knowledge workers great. Don't fit into the mold or fit into the norm. I think that large language models will make humans better at experimenting. They'll make them better, especially really great applications at at doing the stochastic thing. And they'll let us go down different rabbit holes and enable us, in in ways that we can't even imagine yet.
00:23:16 · Dan Turchin
[00:23:16] S1: So you just said the key thing, make humans better. And yet I had Peter Voss on this podcast recently. He's one of the co inventors of the term AGI. And we had a spirited discussion about what the end game of AGI generating artificial general intelligence should be. And he quite proudly, you know, claims the objective is to replicate everything that a human can do, only better because, you know, it's a digital brain. That's not, in my estimation, making humans better. That's aiming to replace or perhaps even in the most cynical sense, confusing humans into thinking that they're interacting with humans when they're interacting with, you know, with digital, you know, likenesses of humans. What's your sense of kind of the concept of AGI and and making better better humans with technology versus potentially replacing them?