00:18:00 · George Sivulka, Danny Fortson
[00:18:00] S4: I I really committed to the bit, and I showed up with a a actual prepared Neuralink for coral reefs. And I think they were almost incredulous. What... You know, is this kid serious? And I was I was dead serious, not even a crack smile. [00:18:14] S2: I... The reason I ask is I I did something far less elaborate but similar. I I got a scholarship to... Through the rotary to go to Spain to study journalism because I figured I needed to study journalism and I need to study study it in Spanish. And so I figured out where I wanted to go in Spain and then kind of reverse engineered my pitch to go there, and it was complete. I mean, God bless the rotary, but they refunded me, and it was, like, the best thing ever. But it was not exactly, you know, wasn't the most practical idea, but it was... For me, it kinda changed my life. But [00:18:50] S4: Hey. You you got what you needed out [00:18:52] S2: of it. Exactly. [00:18:53] S4: And there's an there's an educational element to that way outside of learning Spanish or coral reefs that I think helps in business too.
00:18:53 · George Sivulka, Danny Fortson
[00:18:53] S4: And there's an there's an educational element to that way outside of learning Spanish or coral reefs that I think helps in business too. [00:18:59] S2: Exactly. So I'm curious as a mathematician, are you surprised we have got to where we are in terms of what these LLMs can do or more to the point that it's LLMs that have become the thing? Right? Because I've talked to some people in the field who are like, you know, LLMs, you know, for people are kinda like, there's a thing over there that some people are working on that's not gonna... It's not gonna be productive in any way. It's kind of like this fringe thing, and all of a sudden chat GPT happens and all... And everybody all of a sudden is like, oh, this actually works. When we talk about AI, this is the first kind of thing that a lot of people are, like, grabbing onto and be like, oh, actually, this is different. This is a new thing. [00:19:45] S4: It's a good question, and it's a funny one. And the way I phrase it is is is, like, why this? Why did LLMs work? You know, looking back, it's always actually relatively obvious. In hindsight, it's it's actually the simplest explanation, actually. And, kind of in in math, the most elegant or the the best proof is the one that gets there in the fewest amount of steps. And I actually think LLMs are are... Is is is kind of, if you look at all of the data that we have available and and the way that we can constrain machine learning algorithms, I think it's actually the most obvious answer. [00:20:21] S2: Right.
00:20:21 · Danny Fortson, George Sivulka
[00:20:21] S2: Right. [00:20:21] S4: We have the most text. Language truly is the only way that we encode logic. That's actually what it's for. And if we could map and constrain a a neural network to be very good at understanding language and sequences, then it obviously would be very good at, you know, generalizing to more tasks. It would it would be obviously very good at that. But if you perspective take forty, fifty years ago when some of the first AI research was going on, we just didn't have the scale or we didn't So have the proof from an analytical perspective, as a mathematician, it's like, well, that's the the most obvious play. But all the scientists, right, the people that collect data over the last forty, fifty years had to wait until we had enough compute and enough data and and these models to be... To to really just be large enough and for us to commit enough for that to actually carry through. [00:21:10] S2: When did you start Hevia? [00:21:13] S4: Started Hevia in 2020, 08/14/2020. [00:21:16] S2: August 2020. So we're almost exactly four years on. And what was the initial idea and has it changed? [00:21:26] S4: Yeah. As as I mentioned, never intended to work on any of this. [00:21:31] S2: Was there a moment where you're like, you're kind of an oh shit moment where you're like [00:21:36] S4: Actually, yes. So so the most interesting topic of research for me in grad school was this idea of meta learning or multitask learning. And it's the idea of, even whether it's a coral reef or an LLM, teaching a machine to learn to learn. [00:21:51] S2: Right.
00:21:51 · Danny Fortson, George Sivulka
[00:21:51] S2: Right. [00:21:51] S4: So if I could give you a few examples, then the AI would generalize in completely new examples. And all of AI up until that point had been like, okay. I'll show you a thousand different stop signs. Now you're really good at detecting stop signs. You could drive a car. [00:22:04] S2: Or tell me the difference between a dog and a blueberry muffin. You know what that [00:22:08] S4: A duck in... Yeah. ImageNet. Exactly. But even even NLP was mostly just refined to to classification or really particular document extraction or, like, understanding characters. And this promise of AI that could learn to learn was the most alluring, impactful technology that I could work on that with. And, yeah, I tried it with Coral Reefs. I tried it with a bunch of other things. Then one day, I believe OpenAI released GPT three [00:22:36] S2: Yep. [00:22:37] S4: In the summer of twenty twenty. And I recall sitting in kind of coronavirus just playing around with this thing and realizing that my life's goal, which was to build meta learning, to build multitask learning had been achieved. [00:22:51] S2: You're like, oh, [00:22:52] S4: no. And I'm like, oh, I've I've committed my whole life to building to building this important technology and I'm looking at it right here. [00:23:02] S2: Right. Back to the coral reef.
00:23:02 · Danny Fortson, George Sivulka
[00:23:02] S2: Right. Back to the coral reef. [00:23:03] S4: And so back to the coral reef. And I was like, well, at least I got to be on a boat in Micronesia for a little bit. Yeah, I made the right call. So The natural question then, especially for an academic is like, so what? Okay, I committed so thoroughly to this direction. Really want to do it. It's here. What's the next step? And I thought to myself for a long time about maybe an equivalent technological transformation, and I thought it would be... Was actually the computer. And when the computer was finally invented as technology, it was not valuable until people built products on top of it. Yeah. You know, I I would actually say the calculator, the TI 84, all the the terminal itself, right, the the earliest operating systems were not really that valuable or that human first until Microsoft invented Excel. [00:23:55] S2: Yeah.
00:23:55 · Danny Fortson, George Sivulka
[00:23:55] S2: Yeah. [00:23:56] S4: And once we had Excel, then billions of people around the world were able to use a computer to do complex multi step work. And so I was looking at this technology and I hadn't invented it, And I was a little angry and I was like, well, what's the most important software or the most important product that I could build on top of that? And I had no idea, but I was heck... Like, I was hell bent on finding it. I just wanted to go and build the most important product. And I thought that it wasn't gonna be a chatbot. I didn't think it was gonna be the LLM itself. I thought that was all technology, but that the product layer would be complex and human first, explainable, and let you do complex work. And we've iterated many times. We actually were some of the first people to build the most used AI architectures in the world. We have, like, invented and reinvented a million search engines and chatbots, and from that perspective, we pivoted a [00:24:47] S2: lot. Right. [00:24:48] S4: But we've always been trying to get to the point we're at today where the market... You know, there's a lot of noise in it, but people are actually driving immense large language model usage, and they're actually getting value from the technology because we've built, I think, the Excel for words. As Excel is to numbers, heavy as matrix is to words. [00:25:06] S2: Today's episode of the Times Tech Podcast is sponsored by ServiceNow.