Invest with AI
00:06:16 · Robert Yang
[00:06:16] S0: Yeah, yeah. Uh I guess so I was thinking about the I had always wanted to do something important. It's like profound. Um, and I felt like there are not that many profound things out there left to do, right? Like Newton and Einstein already did a lot on physics, we already under we know there's gene, right? So but AI just felt like an obvious, like if we can do this, then this is gonna be extremely important, right? Maybe the most important thing in the in like after Newton, right? This is my my opinion. Um, is I think after I think it's like Newton, Darwin, and AI, right? Um so yeah, I thought I just thought that's what I want to do. Um and I'm naive in the sense that I wasn't really thinking about, oh, like, is the timing gonna be ready, right? Maybe it's not gonna be ready for another 30 years, right? Then uh and and then I also didn't think, am I the right person to do this, right? So uh I didn't really think about these things. Uh in fact, I went to um I went to physics department. Uh I actually considered going to biology, but I went to biology and I was like, what do you study? They're like, we're studying like plants, right? And animals, and that's boring. So so I went into physics because I thought, okay, physics is how you understand the world through mathematics, right? And um essentially you want to understand the intelligence through mathematics. Uh so I went to physics and we we were introducing ourselves, and everyone is like, oh, like I really love physics, blah blah blah. And I was like, I love AI, and if you're interested in AI, please come tell me. Um people were very confused because most people have not even heard of AI, right? It's not, it's like um for people who have heard of it, it's like a sci-fi thing. It's the people didn't think it's a real thing. That was 2008.
00:08:17 · Brett Caughran, Robert Yang
[00:08:17] S1: This was what this was what year? [00:08:19] S0: 2008. [00:08:21] S1: 2008, okay. [00:08:22] S0: Yeah, yeah. Yeah, so that's some so that's how I got into computational neuroscience. Um, so something that a lot of people don't realize is is um really some of the uh AI uh pioneers, uh like for example, Dario at um Anthropic, right, the co-founder CEO, uh he actually did uh computational neuroscience in his PhD. Uh I actually know some of his co-authors. Um and uh and the DeepMind co-founder and CEO, um uh Demis Hassabis also did uh neuroscience. He just did straight up neuroscience in his PhD and then did computational neuroscience in his postdoc. Uh, and DeepMind was kind of spun out of um his postdoc. Uh it's um my I guess what I did is you can call it a mistake, at least from my career standpoint. It's like I spent a little too long uh doing it, right? Um kind of probably the right thing would be at 2017, maybe just pivot to go all the way to like building intelligence, right? Because that that was always my goal, right? Um, but I was I think a bit more like risk averse back then. I was like, I already did PhD, I think I can get a faculty job. [00:09:36] S1: And um uh so yeah, so it's uh but but what does it what does a faculty MIT PhD do in in computational neuroscience do? Like what were what were your responsibilities in that in that seat?
00:09:52 · Robert Yang
[00:09:52] S0: Yeah, so um uh yeah, absolutely. So so uh a faculty is like is essentially these like professors, right? I mean, there's rank, there's like assistant professor, associate professor, right? But we can just call them all professors. And so these these professors they do essentially two things. One is they uh teach uh they they teach undergrads, right? Uh teach undergrads and teach grad students. And for a lot of like people uh right who after the graduate college, this is their main understanding of what these professors do, right? But at a lots of research institutes like MIT and like Stanford, Harvard, um, you can say the main responsibility of the professors are actually to conduct research, right? Typically you you run a research lab that has um somewhere between like five to maybe like 30, 40 people, and uh you kind of just guide them to do research, right? Um so so this is what I was doing. Uh I was at MIT for uh a few years um before I essentially um resigned. And uh I had a pretty good lab, but we actually had uh we had I I didn't have any student who's like I wasn't there long enough to have a student graduate PhD, but I had postdocs. These are people who already have PhDs. I had postdocs who did really well. I had one postdoc who went to CMU to be uh assistant professor in the machine learning department, another one went to Brown uh in the applied mass, and I think I mentored like five, six students who ended up going to like MIT, Berkeley, all these good schools. Uh so I was really, I think, proud of our team. Uh I am proud of our team we had at MIT. Um, but during almost the whole time at MIT, I was like suffering from this existential crisis. Uh to be fair, our our apartment, um I would say I was in part of two apartments. One is uh brain cognitive science, another is uh EECS, electroengineering and computer science. Uh and I would say uh these two apartments, uh departments are are really like the best, right? Maybe like uh the MIT department is definitely among like the top three, four departments in the world in computer science. And the brain cognitive science department I was in, I would argue it's like the number one department in the world to do it.
00:12:15 · Robert Yang
Um but what I realize is it's really not about like okay, is this school it's not about like the school, it's not about MIT, it's not about the department, it's about this entire like tsunami that's coming to everybody, which is AI, just like the AI buildup in industry, right? And they're just like and we're seeing this this year in mathematics, right? Like this AI tsunami just like is is come completely transforming how everyone do their work and making a lot of the old way of doing things just irrelevant, and and you have to adapt, right? So I was feeling that um back at MIT, and I felt to do what I've always wanted to do, which is to build, to build humans, to build um, to build AI, I have to uh go to industry. Uh in fact, I I I didn't want to just join a big lab. Um, so I wanted to do it in a startup. Um and the reason really is is I mean you didn't ask it, but I'll I'll explain it. The reason why I wanted to do a startup is is I think the way that the big labs think about AI is still a bit different from my vision of um of of a human, right? So the big lab is very much focused on um very much focused on intelligence, like capability, right? Um, which which makes a lot of sense. I mean, this is how you how you um how you have a real business model, right? Um so capability that is aligned with human, right? So that's how I would describe it. Uh but there are things in human that's uh that we don't really understand the capability yet. Uh like why why this is useful, right? So I'll give you an example, like why why should human feel happy when they when they get a reward, right? Like you you eat some chocolate, you're not only reinforcement learning, which is a major way that people train neural networks, right? You you give you give a neural network chocolate, and you give through reinforcement learning, you would teach them to to more likely to get chocolate in the future, right? Whatever action that led them to get that chocolate, they're more likely to get that chocolate. But humans, we not only do that, we're also happy to eat that chocolate, right? The current neural networks, we don't think that they're happy, right? When something good happens to them, right? Maybe they will say they're happy, but uh it's I think most people don't really believe that they experience the happiness, right? Um, so so now you can say, well, why is that useful, right?
00:14:50 · Robert Yang, Brett Caughran
So like when I wanted to when we first started the company, I just wanted to be able to just build the human without without really answering why this is useful, right? So just wanted to like put in like emotions, long-term memory, right? Just motivation, all of this good stuff into an agent and have them just live alongside human um without really answering the business model. Um, so and that's kind of what we did. [00:15:18] S1: And what what was there a final moment or something you saw when you were at MIT that really was the the catalyst to lead? Because that's a pretty cushy life lifestyle. Yeah. Um what uh what was sort of the final deciding insight?
00:15:35 · Robert Yang
[00:15:35] S0: Yeah, yeah, absolutely. So I was having a lot of um, so I was having a lot of existential crisis, but at the same time, I felt like I missed the last wave, right? So the major wave of NAI between about 2017 and 2023 was essentially scaling of language models, right? So uh 2017 was a transformer paper, and then uh and then in 20, I think 2019, um 2020 was like around that time was the scaling papers, right? So that's that was a main paradigm, and I felt like I just missed out on that entire paradigm, and I just didn't want to like um be a follower. But in 2023, early 2023, so the language models started to be available to people to experiment with, and there were a couple papers from academia that were really influential and just essentially wrapped these language models and built agents, right? Now we're very familiar with agents, but in 2023, early 2023, they were still very new. One paper is called the Voyager paper. They essentially build an agent that um that goes in Minecraft and would like like just like play the game. And um, but it it's it's not really just like playing the game for the sake of like achieving a specific goal. It was actually just exploring, exploring the game, which I thought is really, really uh cool. And another thing that's similar is um there was a paper from Stanford, the Stanford AI Town paper by June Park, who is actually our first, our first investor, an NGO investor. So so June Park had this paper that had like a 20-something agents, also kind of like this language model wrapped with memory, right? Um, that would run around in a virtual tongue. That's like a Pokemon tongue. And they would run around and they would like host birthday parties and they would like try to cook and like buy stuff, right? None of them are real, right? None of these things are real. There is no real food and they cannot consume food. But it was uh it was really uh like a shocking demonstration of the future, right? Where you you don't just have chatbots, you have agents that live and just live their life, right? Um so I read that paper and I I was pretty shocked. But what was even more, I think, shocking for me is we had an intern, uh she she was great. Um her name's uh Ivy Zhao, and uh she was essentially asked to present this at a journal club.