00:21:42 · Aditya Agarwal, Tom McGrath
[00:21:42] S0: Is it a mistake to analogize these networks to brains? Like like Yeah. Right? Like, I I always sort of, think that, oh, I'm probably... Right. Like, I shouldn't do that. Right? This is this is... No. These are alien intelligences. The architecture is not the same. And yet, it sure seems like it. Yeah. It keeps working. Right? Like... And you hear about how that... You know, one of these networks works. That's like, well, kinda feels like how my own cognition works in [00:22:08] S1: many ways. Right? Like... So, you know, if if if analogizing networks to brains is, like, reliably helpful, whether it's true or not Yeah. You maybe you should keep doing it. [00:22:21] S0: Hey. You know what? If it works, it works. [00:22:23] S1: Yeah.
00:22:23 · Tom McGrath, Aditya Agarwal
[00:22:23] S1: Yeah. [00:22:24] S0: That's not really the approach that you're... You wanna take with the models. You're trying to understand why it works. Alright. So let's talk about how this all sort of comes together as, like, a company and a product. Yeah. Because Goodfire is not like a research organization in, in, like, a university. Right? It's a... It's a company. Yep. It was started as a company to go and and, like, make interpretability available as as a product. So how do you, like, kind of pull together these strands of research that you've been doing over the last few years into something that, you know, if we are right about, oh, suddenly this is going to be a must have, and more and more, companies are gonna want to post train their own models. We're gonna have, like, very capable open weight, open source models to post train on and and probably then more people might wanna pre train their own model. Right? Like, you would expect training to sort of, explode Yes. In in terms of demand. Well, combine that with more capable models, you wanna be able to control them as well. Right? So kind of like a rough outline of would interpretability and steerability, as a result be valuable, makes sense. How how do you actually pull that together as, like, a product? [00:23:31] S1: I mean, this was certainly a bit of a a squiggle. Yeah. But we we had a few occasions. [00:23:39] S0: Cut squiggle in [00:23:41] S1: the... No. The classic SPC squiggle. Cut to [00:23:44] S0: brand brand collateral there. Yeah. Yeah. You for
00:23:48 · Tom McGrath
[00:23:48] S1: the plug. I try. So, yeah, it was definitely, like, a bit of a wiggly path. And what we settled on is this idea of Silico, and Silico is a a way of, like, empowering people to do interpretability and model AI research. You know, if you open up Silico, what you get is essentially, you can talk to an agent and it will kind of go off and use... Either use our libraries of interpretability and training, software that we spent a lot of time on and, you know, the sort of deep skills encoded in there from all the kind of collective wisdom of our researchers. And you can use that to do whatever AI or, you know, research. Like, we're specializing in interpretability, but you can also do training in there. You can also do, like, data research in there, that sort of thing. And the idea here is, like, if we went out and we individually taught everyone to do interpretability, we would just never, like, get enough of it done. Like, as you say, as as increasingly capable of models are coming out and people want to own their own own their own models, they are going to want more complicate... Like, more powerful training services. They won't gonna want want to be able to, like, understand the model they're training, control it during training, like, you know, for instance, like this predictive data debugging stuff, they... They've they've gonna have a load of their own data. Some of it is gonna be great. Some of it is gonna, by surprise, teach the model all sorts of things you didn't want it to teach. And so you're gonna want ways of, like, curating that data, debugging individual, like, rollouts from production back into the data or into the model and saying, like, how can we fix this? That sort of thing.
00:25:27 · Aditya Agarwal, Tom McGrath
[00:25:27] S0: Yeah. I mean, it kinda gets to... It's a useful segue to the the last thing I wanted to talk about, which is kind of research generally. Right? Like, you're almost talking about... Or the way you put it to me, I think, before we... When we were chatting earlier is, you know, it's it's not exactly correct to call it, like, a research platform. But also, you know, like research and product have become so kind of fuzzy, right, like with AI. And you even see this in the way that we sort of talk about certain types of companies. Right? Goodfire is a research lab. Right? The lab alone. Right? Like, the the, foundation model labs or that. Right? Everyone uses that term. But you... You've had kind of an interesting perspective as someone who has experienced research. First in academia, when you got your PhD, you were at ICL. Right? [00:26:14] S1: Yes.
00:26:14 · Tom McGrath, Aditya Agarwal
[00:26:14] S1: Yes. [00:26:15] S0: And then then you were at DeepMind. Mhmm. So you're, you know, kind of the the big tech version of, of an AI lab. And then you cofounded a company. Right? And and you're the chief scientist at a at a, small but growing start up. How has your relationship to the idea of doing AI research changed as you've moved through those? Because I I meet a lot of people who are coming from research backgrounds and are thinking about making similar sorts of transitions. Right? And in fact, very kind of weirdly, I've sort of ended up specializing in working with researcher founders because my background is in, like, storytelling. Mhmm. And it turns out that's, like, a very value... That's a particularly valuable skill when working with researcher founders. [00:26:58] S1: Yes. We we can be very, like, literal. [00:27:00] S0: Yes. Yeah. Right. Exactly. Kinda learns a little bit of the the, like, figurative magic. But also it's just, like, often kind of the most ambitious and sci fi stuff. Right? So, like, I I have an affinity to it. But I, you know, I often find myself talking to people about that that process of making the jump from kind of one version of those to the other and and and... Like, because of where I am at SPC to start ups. But, like, how... What has your experience been like making those transitions, and and how has your... How has research changed for you?
00:27:28 · Tom McGrath
[00:27:28] S1: Yeah. The one thing that hasn't changed is I still love it. Like, I I just think that doing doing science is just, like, one of the... Kind of almost like one of the core human activities. You know? It's sort of... There are not that many things that feel more kind of human than doing... Than trying to understand what's going on. You know, you could imagine that that spent, like, tying your life so closely to something like that would kind of kill your passion for it. But the thing that hasn't changed is, I guess, I just love it. And some of that is to adjust interpretability being so much fun. I suppose I've been going gradually towards more kind of needing to be pragmatic while still trying to maintain this, like, long term vision. And it's remarkable, the effect of, like, having a deadline. Having a deadline is just, like, actually quite quite good. And and sometimes you need to organize things and that kind of thing. I think this may be a personality thing. Obviously, I'm talking about distribution here, but there'll be outliers in either direction. But, like, I think most... More scientists tend towards being kind of relatively egalitarian. They're like, well, we'll just kind of have... We'll kind of organize it all out to ad hoc, which I think is a nice tendency.