No Priors: Artificial Intelligence | Technology | Startups
00:10:35 · Mark Zuckerberg
[00:10:35] S0: I mean, are probably some differences. I mean, you can probably talk more to the specifics around this. But I mean, I think each layer is gonna end up being somewhat qualitatively different. Right? But you need to be able to understand the protein interactions in order to be able to understand how cells work. So you can't just go straight to cells in a way without understanding the protein modeling. And then if you're trying to understand something like the, you know, the way the immune system works or a bunch of cells interact together, then, you know, it's tough to do that without first understanding cells. I mean, you might be able to, at, like, a very high level of abstraction, simulate a system. But if you really wanna, like, understand how it's gonna work, you kind of wanna build the simulations at each level hierarchically. So that's basically the approach that we're going through starting with the building blocks and the and the protein. But yeah. I mean, I think that there's gonna be different types of data that you wanna collect for each. The modeling techniques, think we'll see. I mean, that'll all keep on advancing across the board. But I do think that, like, a big part of the strategy is this view that you need to build it up hierarchically.
00:11:35 · Priscilla Chan, Elad Gil, Mark Zuckerberg
[00:11:35] S1: And, you know, one of the things that's unique about us in this space is we are very intentional that the AI efforts and the wet lab efforts were a single effort. And we've done a lot of work to bring them together. And the really neat thing that we can do is really try to pull and gather data that helps us connect across sort of the hierarchy. You know, you can look at transcriptomics with space within a cell and look at where it's localizing. We can look at translucent zebrafish and look at the development across different cells and when the brain develops. We have sensors that allow us to look at cell cell communication in different molecules. And so we can be strategic about the types of experiments and data we want to collect that helps us bridge across these, that makes it so that there's some connective tissue that helps drive the modeling that you know, the modeling magic that happens. [00:12:35] S4: Yeah. The reason I asked the question, by the way, is I used to be a biologist. I have a PhD in biology, and I worked in wet labs for almost a decade and everything else. [00:12:40] S1: Are looking for a job. [00:12:46] S4: We can talk about that later. [00:12:48] S0: It's not a no. At [00:12:49] S4: this point [00:12:49] S0: in my career, know? [00:12:51] S1: Hearing, I [00:12:52] S0: love my aggressive reaction.
00:12:52 · Mark Zuckerberg, Elad Gil
[00:12:52] S0: love my aggressive reaction. [00:12:53] S4: I'm like Danny Glover, you know, and Lisa Waffen. I'm almost at retirement. But I think one of the things that was always lacking was this integrative nature across the different layers of biology, and the developmental biologists would work on their own, the molecular biologists would be doing different experiments. And so that's what I was curious about. Typically, there's a reductionist view of biology and there's a systems view, and those people didn't really work together deeply. And so one of the exciting things about what you're doing actually is how you're bridging that. And so that was kind of the basis for the question as well.
00:13:20 · Alex Rives
[00:13:20] S2: Yeah. And if I could add something there. You know, it's I think that, you know, we're in the age of this kind of information theory in biology. And so, you know, there there are levels of of complexity and hierarchy in biology, and kind of each level is is made up of and, you know, constituted by the lower levels. And so as you want to have that kind of more complete description and you want to have systems that can really generalize and begin to actually answer, you know, experimental questions digitally that you could ask in the lab, you need to have kind of the right basis for modeling at every level. And so I think what's really unique about what we can do is to, as Priscilla and Mark were saying, really build information at each of these different layers, collect them, collect kind of those connection points, but then also really kind of do it at the scale that will reveal that underlying information architecture. And that's gonna be really critical to actually be able to build digital representations that can answer new experimental questions.
00:14:22 · Sarah Guo
[00:14:22] S3: One of the things that inspires me most about this effort is really what Priscilla said, which is like, well, there's so much we actually understand about biology and what if we could? Which I think is actually very different from lots of other incredibly interesting and useful AI problems we attack where we're like trying to replicate human behavior. I'm like, a lot of that data's, you know, on the internet or captured. And without pretending to understand all human behavior, can predict a lot of it. I thought one of the most interesting things in your release was actually, you know, the mechanistic interpretability stuff you alluded to, which is, can we actually extract new knowledge from, you know, what the model believes is happening? Right? Can you talk a little bit about that?
00:15:03 · Alex Rives
[00:15:03] S2: Yeah, I'm really excited about that. So I think, you know, in mechanistic interpretability, kind of traditionally it's been applied to large language models with the goal of understanding, you know, kind of what is the representation space of a large language model? How does it compute things, and does that really connect to, you know, what we understand about our intuitive understanding of the world? And so there's, I think, this really rich toolkit that has been developed to start to be able to ask those questions. So kind of what does that mean for biology? One of the classes of models that we train are these protein language models. So they're really, you know, trained on the codes of proteins. And so anything they learn about biology is is kind of emergent. And we've seen that they can learn things like biological structure and biological function, and that's just kind of emergent from this, you know, token prediction training task. So, you know, as we think about, like, mechanistic interpretability in those models, you know, we're really seeing the unknown because the models have been trained on billions of protein sequences. They've been trained on, you know, both known and unknown biology. And yet they're developing these representations that start to kind of capture things that we can really see correspond to that reductive picture of biology that's been built up over the centuries. So kind of you can start to connect the dots between proteins where we kind of really don't know anything about them with proteins where we do know something because there's that kind of underlying structure grammar that's linking them in the representation space of the model.