Lenny's Podcast: Product | Career | Growth
01:00:43 · Lenny Rachitsky, Emmanuel Ameisen
[01:00:43] S0: next year? Need to think about for a while. [01:00:45] S1: And for some stuff where it's like, you know, is this person who wrote us a review happy or upset? You kind of, you're happy with the sort of gut judgment. So yeah, think this I think this kind of research is cool. And I think it's also kind of different from the, you know, it's like a bit of a novel take on some of the LMs we've seen. So yeah, I'm pretty excited to see where this kind of stuff goes. [01:01:08] S0: So what I'm hearing here is this kind of gonna be a new foundational component, most likely of AI and LMs. And the system one, system two stuff is basically Kahneman's kind of, I have that book up there actually behind me. System one, just to keep it simple, just like kind of your amygdala, limbic system making just like gut level quick decisions like snake, oh no. Yes. Where you don't have to think long about it. It's kind of to protect you from bad things really easily and quickly. And system two is where we really think about stuff and process. Yeah. Exactly. Sweet. Emmanuel, is there anything else that we haven't covered? Anything else you wanna say or mention? Anything you wanna double down on before we get to our very exciting lightning round?
01:01:08 · Lenny Rachitsky, Emmanuel Ameisen
[01:01:08] S0: So what I'm hearing here is this kind of gonna be a new foundational component, most likely of AI and LMs. And the system one, system two stuff is basically Kahneman's kind of, I have that book up there actually behind me. System one, just to keep it simple, just like kind of your amygdala, limbic system making just like gut level quick decisions like snake, oh no. Yes. Where you don't have to think long about it. It's kind of to protect you from bad things really easily and quickly. And system two is where we really think about stuff and process. Yeah. Exactly. Sweet. Emmanuel, is there anything else that we haven't covered? Anything else you wanna say or mention? Anything you wanna double down on before we get to our very exciting lightning round? [01:01:48] S1: I'm definitely curious for folks takes always on some of the stuff that, you know, still an explorer didn't make sense. There's a lot more interpret experiments than we have the time to share, you know, models have a bunch of weird, weird little things about them. Like maybe I'll do a lightning round of weird little things, but that's okay. [01:02:08] S0: Let's do it. Absolutely.
01:02:08 · Lenny Rachitsky, Emmanuel Ameisen
[01:02:08] S0: Let's do it. Absolutely. [01:02:09] S1: Okay. Weird little thing one, we found that, you know, models can like actually answer questions. Obviously they can do math or they can, you know, you can ask them like, what's the color of a banana and they'll actually like, you know, if you look inside, you'll see all these intermediate but they have a separate mechanism, which is kind of like a gut check and which is totally different. And which is just like, as they reason, they'll have like some part of them that will flag like, something's wrong. So if a model like somehow says, you know, four plus five equals 23 or something, There's a part of the model that just like reads that and doesn't do the math, but it's just like that can't be right. Because like four plus five definitely under 10. And so they learn these weird kind of like hunches. They feel like kind of like a hunch. Maybe another one is you can tell the model, hey, do this task and think about this other thing. And then you can use interrupt to see if like there's intermediate representations that map to the other thing. And that works. You sort of see that it does think about the other thing. This is something that's really mysterious to me because there's no kind of training reason why why the model would have to learn that. So it's sort like another one of these weird behaviors. And actually, maybe I'll keep it at two. Think those two are good enough.
01:02:09 · Emmanuel Ameisen, Lenny Rachitsky
[01:02:09] S1: Okay. Weird little thing one, we found that, you know, models can like actually answer questions. Obviously they can do math or they can, you know, you can ask them like, what's the color of a banana and they'll actually like, you know, if you look inside, you'll see all these intermediate but they have a separate mechanism, which is kind of like a gut check and which is totally different. And which is just like, as they reason, they'll have like some part of them that will flag like, something's wrong. So if a model like somehow says, you know, four plus five equals 23 or something, There's a part of the model that just like reads that and doesn't do the math, but it's just like that can't be right. Because like four plus five definitely under 10. And so they learn these weird kind of like hunches. They feel like kind of like a hunch. Maybe another one is you can tell the model, hey, do this task and think about this other thing. And then you can use interrupt to see if like there's intermediate representations that map to the other thing. And that works. You sort of see that it does think about the other thing. This is something that's really mysterious to me because there's no kind of training reason why why the model would have to learn that. So it's sort like another one of these weird behaviors. And actually, maybe I'll keep it at two. Think those two are good enough. [01:03:24] S0: Just what a fun what a fun job this is just studying this thing and trying to understand what the hell is going on. How is it getting this smart? How does it think?
01:03:24 · Lenny Rachitsky, Emmanuel Ameisen
[01:03:24] S0: Just what a fun what a fun job this is just studying this thing and trying to understand what the hell is going on. How is it getting this smart? How does it think? [01:03:32] S1: Yeah. [01:03:33] S0: How many... Like, how many people are even how many people are even in this space? Just like how large is this field? [01:03:39] S1: The field itself is, is, you know, is getting pretty large. Would say maybe hundreds, not thousands. There's, you know, different inter people at the different labs. And then there's, there's in academia, quite a few labs putting out great work. [01:03:54] S0: Is the best, if someone were to get into this space, are they, is it like an ML person usually, or is there like a neuroscientist sort of track to get into this? [01:04:04] S1: Yeah, there's actually a surprising amount of, you know, ex neuroscientists. I do think in order to get in the field, you know, there's a, there's actually, it doesn't require a lot of prior knowledge because it's such a new field. So it's actually, you know, the on ramp is mostly, I would say like self studying and reading some of the key papers. And then there's actually, you know, yeah, maybe I should give a plug. There's actually like websites like nerdpedia.org, I think is the URL, which are sort of like open source interp. So they like host, you know, a bunch of models and let you sort of do interpret experiments on them. And so, you know, people have just been doing that on their own. And I think that there's so much to explore that if you're at all curious, you should just give it a try.
01:04:04 · Emmanuel Ameisen, Lenny Rachitsky
[01:04:04] S1: Yeah, there's actually a surprising amount of, you know, ex neuroscientists. I do think in order to get in the field, you know, there's a, there's actually, it doesn't require a lot of prior knowledge because it's such a new field. So it's actually, you know, the on ramp is mostly, I would say like self studying and reading some of the key papers. And then there's actually, you know, yeah, maybe I should give a plug. There's actually like websites like nerdpedia.org, I think is the URL, which are sort of like open source interp. So they like host, you know, a bunch of models and let you sort of do interpret experiments on them. And so, you know, people have just been doing that on their own. And I think that there's so much to explore that if you're at all curious, you should just give it a try. [01:04:52] S0: That's awesome. We'll link to them in show notes in case that's not the right URL. Cool. Well, Emmanuel, with that, we've reached our very exciting lightning round. I've got five questions for you. Are you ready? [01:05:01] S1: I am as ready as I'll ever be. [01:05:03] S0: Here we go. What are two or three books that you find yourself recommending most to other people?