00:10:45 · Danny Fortson, George Sivulka
[00:10:45] S2: Yeah. [00:10:46] S4: And there are certain engines that are are really good for certain tasks. Heavier, instead of just putting a... An engine in the car and then just throwing you down the... Throwing you up at hell or or or letting the the the LLM dictate everything, says that for a really difficult task, maybe we actually need to run way more large language models. And that all of these are effectively good enough at simple single step tasks. But if I go back to that investment question or this really complex work problem, we actually are orchestrating maybe a GPT four model or a thousand GPT four models to answer one question, or we're orchestrating quite a lot of LLM calls with much smaller, simpler tasks. You can think of it as a car that's driving a thousand engines. And even if the engines are terrible, I can still get up a hill because you're just looping in all of that collective power. [00:11:38] S2: I'm gonna use a different analogy, you could tell me whether it's off base or not. Heavy is kind of like an orchestra conductor where you have a whole group of musicians. Some may be good, some are amazing, but together you just make sure that the music that comes out is fantastic. [00:11:55] S4: That's right. [00:11:56] S2: Right. [00:11:56] S4: And the conductor knows exactly which, musician to point at at which point in time for which song. [00:12:02] S2: Got you. Why call it Hebiah?
00:12:02 · Danny Fortson, George Sivulka
[00:12:02] S2: Got you. Why call it Hebiah? [00:12:06] S4: When I was in grad school, I had... I'd been accepted into... It's PhD at Stanford. It was in maybe the most boring, you know, not outside department in the world. And by outside, I literally mean outside. I did my undergrad in math, my master's in applied physics and then my PhD was accepted in electrical engineering with a focus in computational neuroscience. [00:12:27] S2: What was the plan or did you have a plan? [00:12:30] S4: I all along wanted to be a professor. I did not intend at all to be an entrepreneur. I was solely focused on academic. I loved research. I loved discovering new things, and I loved teaching. Teaching was, like, the highest ideal for me. And, honestly, there's a ton of different teachers that I grew up with that that changed my life. And so I was I was mimicking them. That was the path I was going down. But I get into this PhD and I'm like, well, I'm about to go sit in a room for the next ten years, however long. I've already been in in in the math corner for for quite a long time at Stanford. And I wanna do a fun rotation. I wanna do something fun outside. I sat... I thought I thought to myself, was like, what's the most interesting or, you know, rotation or or research project that I could do now that I'm I'm kind of in this break before really deep grad school? And I wanted to go to Micronesia, And I promise this relates to this relates to the name Hepia. [00:13:25] S2: Question. Question. Why Micronesia? [00:13:29] S4: I love the ocean. [00:13:30] S2: Okay.
00:13:30 · Danny Fortson, George Sivulka
[00:13:30] S2: Okay. [00:13:30] S4: And I thought it was... There's this beautiful place, called Palau in in the coral reef triangle of the world. And I said, well, if I can make up a research project that would let me go to Palau, then I could really have a lot of fun for my first stint in grad school. And I wanted this to be relevant to my research. I wanted to be relevant to computational neuroscience. And everyone talks about AGI today, and there's a lot of different... You know, we're gonna make the models larger that become AGI. There was a whole separate subfield of computational neuroscience around mapping a connectome of animal brains. You might see where I'm going and training artificial neural networks based on like animal neural networks. And so I said, well, I'm gonna go and collect coral reef samples in Palau, and I'll get to be on a boat every day and free dive every day. And, you know, it'll it'll be in grad school. And then I'll train, an artificial neural network to go and mimic that that behavior. [00:14:29] S2: The the coral... The the different bits of the coral reef talking to other, etcetera. This sounds like a real stretch. [00:14:37] S4: I I... Hey. I I was on [00:14:39] S2: a boat. No. I appreciate it. So you actually did it. You pulled it off. I pulled it off.
00:14:39 · Danny Fortson, George Sivulka
[00:14:39] S2: a boat. No. I appreciate it. So you actually did it. You pulled it off. I pulled it off. [00:14:44] S4: So So Danny, I built a Neuralink, effectively a Neuralink device that had a thousand different suction electrodes that... And we put different coral samples in there in the Palau Coral Reef Center. And I would... I was the first person ever in the world to record the action potentials, which is the neuron firing in five different coral species. [00:15:03] S2: Of course you did. I mean, because... Of course. Yeah. Okay. [00:15:06] S4: I I went and took all of these action potentials and I trained with an algorithm called Hebbian learning, which is the simplest learning rule that the brain learns with and that an artificial neural network learns with a soulful model of the coral. And I mean that as in it... Like, it did the exact same thing. So if the coral had a soul, you would say that my my running program also had a soul. [00:15:29] S2: Got you. And so Hebbian learning is where Hebbia comes from. [00:15:33] S4: That's right. It's it's the simplest learning rule by which machines and biological brains like yours and mine can can both learn. [00:15:41] S2: So you kind of mapped this activity of these different coral. I'm just curious. What did you learn about coral and how they quote unquote think?
00:15:41 · Danny Fortson, George Sivulka
[00:15:41] S2: So you kind of mapped this activity of these different coral. I'm just curious. What did you learn about coral and how they quote unquote think? [00:15:52] S4: There's a lot of very similar structures in these coral brains. And this is very interesting because coral brains are the first of all neural networks in the animal kingdom. Like, there's the oldest evolutionary quote unquote brains to the same way that our brain works as Homo sapiens today. And so you these recurring patterns where when the sun goes down, they actually have an internal clock that kicks in. There's a circadian rhythm. You see that when they're fighting and corals have a very rich life, believe it or not. You know, when they're fighting other corals or when, you know, it's mating season, you actually see different cycles kick in. [00:16:30] S2: Right. [00:16:30] S4: And just like the human brain has different modes of existing and being, the coral brain would actually shift into completely different dynamics and and modalities of of thinking and engaging with the world. [00:16:41] S2: While that sounds totally fascinating, and I could redo a whole podcast on, like, know, the brain of coral reefs, was there any, like, practical takeaway that you took from that? Like, oh, that's interesting that the way that functions, therefore, it can be applied in a certain way in this completely different place. Or was this just like you're an academic, you're going full academic, and you're studying coral reef brains.
00:16:41 · Danny Fortson, George Sivulka
[00:16:41] S2: While that sounds totally fascinating, and I could redo a whole podcast on, like, know, the brain of coral reefs, was there any, like, practical takeaway that you took from that? Like, oh, that's interesting that the way that functions, therefore, it can be applied in a certain way in this completely different place. Or was this just like you're an academic, you're going full academic, and you're studying coral reef brains. [00:17:10] S4: Yeah. I think there was the super high level goal of if we could create a coral brain in silica, then we could actually apply evolutionary algorithms and get it to do more complex things to mimic an animal brain to then be useful one day. And there is all of this kind of longer term idea. Well, if I could do this for a coral, could I do this for a human? Could we actually start to have cognition biologically motivated? It's actually not too different than the idea of trying to approach machine learning from maybe instead of all the way down to the neural recordings of animals, the words that we use today to train large language models. So it was just a different way to try to get to the same end goal. [00:17:51] S2: When you pitch that, did you did you ever kinda, like, crack a smile? Or were they ever like, you know, okay. Good luck. [00:18:00] S4: I I really committed to the bit, and I showed up with a a actual prepared Neuralink for coral reefs. And I think they were almost incredulous. What... You know, is this kid serious? And I was I was dead serious, not even a crack smile.