Lenny's Podcast: Product | Career | Growth
00:21:09 · Emmanuel Ameisen, Lenny Rachitsky
And then sort of see like, oh, what would happen, you know, if while the model is writing these intermediate results, I went and like secretly changed one of the results. And does that do what I would expect? And in fact, oftentimes it does. [00:21:22] S0: It's so interesting to hear you talk about this and how much of this is us just trying to figure out what the hell is going on versus we designed it this certain way and just how much of this has emerged. I don't know. Is that just like, I guess that's just what makes this such a fun job I imagine for you just like what is going on? How is this all working?
00:21:40 · Emmanuel Ameisen
[00:21:40] S1: Yeah, it's it's a fun and sci fi like job because I have been working in tech for a while. And before this, you know, I was working on more applied machine learning and there are a lot of what you do is about building systems and it's sort of like the practice of software engineering. And here a lot of what we do feels so much closer to biology than to engineering, right? Because it's just the sort of system arrives at you and you're being asked to say like, well, we've noticed that Claude can translate words or it can code in this way, or sometimes it behaves, sometimes there's like a weird behavior, like, and you're just asked to answer, well, like, why is it doing that? But you don't have access to the source code really like you do for a program. Like the source code is just this massive list of mathematical operations, which is just like incomprehensible. And so you end up, you know, in the position of someone kind of similar to someone being asked like, hey, you know, when like a mouse gets hungry, like it sort of like gets aggressive, what's going on with that, right? And you're just like, well, guess I'll like run some experiments to see what the factors are. And so it's a very different kind of study than anything that I think most of us on the team have done before.
00:23:02 · Lenny Rachitsky
[00:23:02] S0: Yeah. I imagine people are starting on this, they didn't expect to create this new kind of almost life life type of being. Do you think about consciousness as a part of this? Just like, do you think AI is conscious? Do you think it'll get to a place where we think it's conscious? Or is that just not even, not even worth our time to think about?
00:23:19 · Emmanuel Ameisen
[00:23:19] S1: I mean, I think it's sort of very hard to say one way or another, you know, some of the most interesting ways to get at this question is interpretability because you're sort of, you're not biased by sort of like what the model says. You can just look at the structure inside and try to determine whether it pushes one way or another on the question. I will say that the thing I believe pretty strongly is that most people underestimate the sort of complexity that emerges and that like, if you look inside, there's just the sort of, you know, the facts that the models, the out learns or these representations for all of the concepts that we have and that we know, that they manipulate them to solve problems, that they do so by like planning many tokens ahead, that they sort of have this mix between this higher order reasoning and lower reasoning. And all of that just kind of emerges something that I think is underappreciated by people. And to me, it gives me, let's say a deeper sense of like, oh, there's a lot to still understand about how they work. I think maybe that's the main thing is, is I find that the more you look inside, the less certain you are about how they work. And then oftentimes people that will confidently say like, ah, language models are just X or just Y are the people that actually haven't spent a while looking inside.
00:24:45 · Lenny Rachitsky
[00:24:45] S0: And there's this, I don't know, thread of history through to how we got here, is a lot of people did not think we would get to artificial intelligence that is useful by just investing in something that was seemingly very simple and just giving it a lot of compute and that was famously something OpenAI bet on and then the Anthropic founders were at OpenAI and then spun off started their own thing. It was all kind of bet on this bet that okay, we could just do this seemingly simple thing and give it a ton of compute and it will become incredibly smart. And a lot of people didn't think that was possible and they were proven wrong. So I don't know, is there anything there that is just like, oh wow, look at this, it worked out.
00:25:24 · Emmanuel Ameisen
[00:25:24] S1: I think it's two things actually. Cause I think it's not just compute. It's also finding an objective where compute keeps giving you returns. So what does that mean? It's actually not that new of an idea. I mean, you're right, like compute versus sort of, I would say expert design architectures is a long debate. But then there's an additional step there, which is compute to do what? You know, so there's a bunch of prior work on like, oh, you can spend a bunch of compute to make a bot that's really good at go, you know, or that can win some video game. And then that's cool. And just the fact that that works is sort of like interesting, but that doesn't give you something that is nearly as useful as LLMs like Claude are. And so I think like one key thing that many in the field, including me underappreciated, you know, a few years ago and that has borne out is the combination of sort of like compute and the richness of text is what gets you there. And to me, like this is kind of what I was alluding to, but you know, if you think about like there's this observation that at OpenAI people made, it was some of the, a lot of the early team at Anthropic on the scaling loss paper. And so this is the observation that if you sort of like pour in more compute and more data to the task, to the task of predicting the next word, you see that like the scale of the model you get at predicting the next word, just like predictably keeps improving. And it's sort of like a trend that just keeps going and it keeps going and keeps going. And that's an empirical law, people sort of, you know, speculated as to why at the time, but it's sort of still just mostly an empirical finding. And that means that it's like, ah, this is a field where you can just keep pouring more compute and you keep improving on that metric. And that's rare. And also it's rare for it to be a metric that then correlates with a bunch of tasks that we actually care about.