00:14:39 · Danny Fortson, George Sivulka
[00:14:39] S2: a boat. No. I appreciate it. So you actually did it. You pulled it off. I pulled it off. [00:14:44] S4: So So Danny, I built a Neuralink, effectively a Neuralink device that had a thousand different suction electrodes that... And we put different coral samples in there in the Palau Coral Reef Center. And I would... I was the first person ever in the world to record the action potentials, which is the neuron firing in five different coral species. [00:15:03] S2: Of course you did. I mean, because... Of course. Yeah. Okay. [00:15:06] S4: I I went and took all of these action potentials and I trained with an algorithm called Hebbian learning, which is the simplest learning rule that the brain learns with and that an artificial neural network learns with a soulful model of the coral. And I mean that as in it... Like, it did the exact same thing. So if the coral had a soul, you would say that my my running program also had a soul. [00:15:29] S2: Got you. And so Hebbian learning is where Hebbia comes from. [00:15:33] S4: That's right. It's it's the simplest learning rule by which machines and biological brains like yours and mine can can both learn. [00:15:41] S2: So you kind of mapped this activity of these different coral. I'm just curious. What did you learn about coral and how they quote unquote think?
00:15:41 · Danny Fortson, George Sivulka
[00:15:41] S2: So you kind of mapped this activity of these different coral. I'm just curious. What did you learn about coral and how they quote unquote think? [00:15:52] S4: There's a lot of very similar structures in these coral brains. And this is very interesting because coral brains are the first of all neural networks in the animal kingdom. Like, there's the oldest evolutionary quote unquote brains to the same way that our brain works as Homo sapiens today. And so you these recurring patterns where when the sun goes down, they actually have an internal clock that kicks in. There's a circadian rhythm. You see that when they're fighting and corals have a very rich life, believe it or not. You know, when they're fighting other corals or when, you know, it's mating season, you actually see different cycles kick in. [00:16:30] S2: Right. [00:16:30] S4: And just like the human brain has different modes of existing and being, the coral brain would actually shift into completely different dynamics and and modalities of of thinking and engaging with the world. [00:16:41] S2: While that sounds totally fascinating, and I could redo a whole podcast on, like, know, the brain of coral reefs, was there any, like, practical takeaway that you took from that? Like, oh, that's interesting that the way that functions, therefore, it can be applied in a certain way in this completely different place. Or was this just like you're an academic, you're going full academic, and you're studying coral reef brains.
00:16:41 · Danny Fortson, George Sivulka
[00:16:41] S2: While that sounds totally fascinating, and I could redo a whole podcast on, like, know, the brain of coral reefs, was there any, like, practical takeaway that you took from that? Like, oh, that's interesting that the way that functions, therefore, it can be applied in a certain way in this completely different place. Or was this just like you're an academic, you're going full academic, and you're studying coral reef brains. [00:17:10] S4: Yeah. I think there was the super high level goal of if we could create a coral brain in silica, then we could actually apply evolutionary algorithms and get it to do more complex things to mimic an animal brain to then be useful one day. And there is all of this kind of longer term idea. Well, if I could do this for a coral, could I do this for a human? Could we actually start to have cognition biologically motivated? It's actually not too different than the idea of trying to approach machine learning from maybe instead of all the way down to the neural recordings of animals, the words that we use today to train large language models. So it was just a different way to try to get to the same end goal. [00:17:51] S2: When you pitch that, did you did you ever kinda, like, crack a smile? Or were they ever like, you know, okay. Good luck. [00:18:00] S4: I I really committed to the bit, and I showed up with a a actual prepared Neuralink for coral reefs. And I think they were almost incredulous. What... You know, is this kid serious? And I was I was dead serious, not even a crack smile.
00:18:00 · George Sivulka, Danny Fortson
[00:18:00] S4: I I really committed to the bit, and I showed up with a a actual prepared Neuralink for coral reefs. And I think they were almost incredulous. What... You know, is this kid serious? And I was I was dead serious, not even a crack smile. [00:18:14] S2: I... The reason I ask is I I did something far less elaborate but similar. I I got a scholarship to... Through the rotary to go to Spain to study journalism because I figured I needed to study journalism and I need to study study it in Spanish. And so I figured out where I wanted to go in Spain and then kind of reverse engineered my pitch to go there, and it was complete. I mean, God bless the rotary, but they refunded me, and it was, like, the best thing ever. But it was not exactly, you know, wasn't the most practical idea, but it was... For me, it kinda changed my life. But [00:18:50] S4: Hey. You you got what you needed out [00:18:52] S2: of it. Exactly. [00:18:53] S4: And there's an there's an educational element to that way outside of learning Spanish or coral reefs that I think helps in business too.
00:18:53 · George Sivulka, Danny Fortson
[00:18:53] S4: And there's an there's an educational element to that way outside of learning Spanish or coral reefs that I think helps in business too. [00:18:59] S2: Exactly. So I'm curious as a mathematician, are you surprised we have got to where we are in terms of what these LLMs can do or more to the point that it's LLMs that have become the thing? Right? Because I've talked to some people in the field who are like, you know, LLMs, you know, for people are kinda like, there's a thing over there that some people are working on that's not gonna... It's not gonna be productive in any way. It's kind of like this fringe thing, and all of a sudden chat GPT happens and all... And everybody all of a sudden is like, oh, this actually works. When we talk about AI, this is the first kind of thing that a lot of people are, like, grabbing onto and be like, oh, actually, this is different. This is a new thing. [00:19:45] S4: It's a good question, and it's a funny one. And the way I phrase it is is is, like, why this? Why did LLMs work? You know, looking back, it's always actually relatively obvious. In hindsight, it's it's actually the simplest explanation, actually. And, kind of in in math, the most elegant or the the best proof is the one that gets there in the fewest amount of steps. And I actually think LLMs are are... Is is is kind of, if you look at all of the data that we have available and and the way that we can constrain machine learning algorithms, I think it's actually the most obvious answer. [00:20:21] S2: Right.
00:20:21 · Danny Fortson, George Sivulka
[00:20:21] S2: Right. [00:20:21] S4: We have the most text. Language truly is the only way that we encode logic. That's actually what it's for. And if we could map and constrain a a neural network to be very good at understanding language and sequences, then it obviously would be very good at, you know, generalizing to more tasks. It would it would be obviously very good at that. But if you perspective take forty, fifty years ago when some of the first AI research was going on, we just didn't have the scale or we didn't So have the proof from an analytical perspective, as a mathematician, it's like, well, that's the the most obvious play. But all the scientists, right, the people that collect data over the last forty, fifty years had to wait until we had enough compute and enough data and and these models to be... To to really just be large enough and for us to commit enough for that to actually carry through. [00:21:10] S2: When did you start Hevia? [00:21:13] S4: Started Hevia in 2020, 08/14/2020. [00:21:16] S2: August 2020. So we're almost exactly four years on. And what was the initial idea and has it changed? [00:21:26] S4: Yeah. As as I mentioned, never intended to work on any of this. [00:21:31] S2: Was there a moment where you're like, you're kind of an oh shit moment where you're like [00:21:36] S4: Actually, yes. So so the most interesting topic of research for me in grad school was this idea of meta learning or multitask learning. And it's the idea of, even whether it's a coral reef or an LLM, teaching a machine to learn to learn. [00:21:51] S2: Right.