Lenny's Podcast: Product | Career | Growth
00:02:55 · Emmanuel Ameisen
[00:02:55] S1: At a really high level, the way a model like Claude works, is you kind of train it in these successive phases. Like the first phase, which people refer to as pre training is you just take a large corpus of data. And over this corpus of data, you sort of feed the model as input, you know, the let's say first half of like a certain document and you ask it to predict the word that comes next. And then you do that again and again and again and again for sort of an ungodly amount of data. And after you do that, there's then this other phase where, and you sort of have this fine tuning phase where maybe instead of arbitrary data, you have a corpus of examples of like how the model should behave and that's what you train it on. So you have examples of dialogue of like helpful assistant telling you, you know, your favorite recipe or solving coding problems. And then there's sort of like further details here that I'm just gonna skip for a second, but then, okay. So what does that mean that you're doing this process? Like, is this thing that you're, that you're sort of training? All models are is, you know, they take this input, which is a bunch of words. They turn these words into numbers and then they do some math on these numbers and they get some intermediate results. Then they do more math and they get some intermediate results. And they do this again and again and again and again many times. And then at the end, they have a final result, which then convert back into a word. And so the math that they do, computations, that's the part that's grown. That's the part where it's just like through learning to do this task better, we sort of like update parameters in one direction or other. And then they learn some formulas. We have no idea what they are. And the goal of my team is to sort of like study, you know, both sort of like what these formulas mean in the end and also what these intermediate results are. Can we decode them and can we see like what the model is doing as it's doing all of this intermediate stuff?
00:04:49 · Lenny Rachitsky
[00:04:49] S0: This episode is brought to you by our season's presenting sponsor WorkOS. What do OpenAI, Anthropic, Cursor, Replit, Sierra, Clay, and hundreds of other winning companies all have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you've felt the pain of integrating single sign on, SCIM, RBAC, audit logs, and other features required by large companies. WorkOS turns those deal blockers into drop in APIs with a modern developer platform built specifically for b to b SaaS. Literally every startup that I'm an investor in that starts to expand upmarket ends up working with Work OS. And that's because they are the best. Whether you are a seed stage startup trying to land your first enterprise customer or a unicorn expanding globally, Work OS is the fastest path to becoming enterprise ready and unblocking growth. It's essentially Stripe for enterprise features. Visit workos.com to get started or just hit up their Slack where they have actual engineers waiting to answer your questions. WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience. Go to workos.com to make your app enterprise ready today. Does it blow your mind as much as it blows mine that we have some code that we wrote that, first of all, we have to now kind of watch and study like like a neuroscientist studies a brain. Like, that's like a new thing that we need to be doing to understand what the hell is going on. And then this idea that as... Like this terminology using is how a lot of people think about this. It's like we've created this new life, artificial life, artificial intelligence. It just is... It's pretty mind boggling if you really think about it and kind of zoom out.
00:06:30 · Emmanuel Ameisen, Lenny Rachitsky
[00:06:30] S1: I think it's really weird. I think that the best way to think about it is sort of like the reason we ended up here. And one of the big reasons is that we don't know how to do it the normal programming way, you know, so there's, there's like for you want to build an app or something you want to sort of do something pretty self contained. You can just have a programmer that's going to write a bunch of rules and say like, oh, well, if somebody clicks on this button, then this happens. You know, if this and that and so on. We don't know how to write that software for classifying, you know, images into categories or for writing, you know, for making a machine that can generally sort of take in arbitrary text and produce texts. It can take in like reports and summarize them, can write emails. And so, know, in a sense, it's weird that this very simple kind of training objective, like gets you all the way there. But in another sense, it's also, you know, I almost feel like there's a sense of like, it's impressive and means something. It means that what you think is a simple objective, this like, oh, we're just feeding data and predicting the next thing is probably deceptively complicated. Like we're probably doing more than what it sounds like. And I think a lot of the ways that maybe like I've updated on how these models work is by realizing how complicated it is to do what they do well. [00:07:56] S0: Say more about that. What do you think about when you say it's this kind of unexpected complication?
00:08:01 · Emmanuel Ameisen
[00:08:01] S1: Okay, so one way to think about next token prediction that I think a lot of people think about is your iPhone autocomplete, you know, where you write some text and then it'll like predict the next word and it's maybe like right, I don't know, 60% of the time and then kind of funny and weird the rest of the time. And I would think of that as sort of, you know, the very, the easiest version of predicting the next word. And unfortunately, I think that's the mental model that people have for these models. But if you get better at that and language models got better at predicting the next word than humans in around 2019, if you keep getting better at at that, you know, you start running out of easy words. So initially you just had to predict sort of like simple sentence structure, you know, later on the model has to predict like, oh, like simple facts about the world. Like if I complete a sentence like, oh, the capital of France is, and that's like, okay, that's Paris. And you know, once it's good at that and it keeps improving, the only thing that's left is sort of predicting, you know, the answer of some complex, physics equation or the sort of final paragraph or some PhD thesis. And so I think that you end up realizing if you think about this is that man, like predicting the hardest words is actually extremely hard to do. And it's a sort of mystery how you would be able to do that. And to me, this is a sort of, this has been a motivation of why you can start studying these models in the first place. So I was like, how can something so simple give rise to something so powerful? And it's like, well, if you think about the model that has to predict sort of, you know, the, one example I always like to use is like, ah, the capital of the states that contains Dallas is, you know, it'll say Austin. Well, to do that well, at least for you and me, you kind of have to have a mental model of like the geography at play, right? You have to be like, oh, where's Dallas? Dallas is in Texas, states have capitals, the capital of Texas is Austin and so on. And so as these models get better at the seemingly simple task, the sort of like best way to solve it, at least for us is to just kind of understand the underlying structure of the world.
00:10:18 · Emmanuel Ameisen, Lenny Rachitsky
And in fact, you know, some of the stuff we look at inside models shows that they do understand that and they do serve manipulate these concepts like we do. [00:10:27] S0: Oh man, there's so much there. One is just this kind of idea, I forget who said this, maybe Ilya, this idea that to predict the next token while you basically have to understand the entire universe. [00:10:36] S1: He has an example that I love to where he he he has the example, like, predict and the murderer was at the end of a mystery novel. Right? It's like, well, to do that, you kind of have to have understood the whole plot. [00:10:46] S0: Yeah. The other the other... I forget who tweeted this idea that, like, humans, if you really boil it down, like we're basically next token predictors ourselves. We're just trying to figure out the next thing to say and if you meditate you start to kind of actually notice your brain wanting to do things and say things. I don't know how much meditation you've done, but it feels like you start to pay attention to your own next token predictor.
00:10:46 · Lenny Rachitsky, Emmanuel Ameisen
[00:10:46] S0: Yeah. The other the other... I forget who tweeted this idea that, like, humans, if you really boil it down, like we're basically next token predictors ourselves. We're just trying to figure out the next thing to say and if you meditate you start to kind of actually notice your brain wanting to do things and say things. I don't know how much meditation you've done, but it feels like you start to pay attention to your own next token predictor. [00:11:08] S1: Oh yeah. Yeah. And this is like, a thing I think about often is that, you know, like right now, as I'm answering you, there's sort of like two things going on in my mind. There's hopefully more than two, but at least two. One of them is, there's sort of like a point I'm leading you to. There's like a broad idea I'm trying to land the sentence on. And the other thing is, at the same time, I'm just predicting the next thing to say. I'm just saying one word at a time and I'm making sentences and I'm trying to make them make sense and I'm pausing when I think it makes sense. But at the end of the day, even though I'm just saying one word at a time, like all along, what I wanted to do is to compare this process to what language models do. [00:11:50] S0: I'm curious what else as you spend time paying attention to how LLMs work and operate. I feel like you've you're like in the top 0.1% of people that have studied and watched LMs operate and think. What else has surprised you about how they operate and think?