This Week in Startups
00:26:20 · Hon Weng Chong
[00:26:20] S3: I think the thing to understand here as well is that LLMs could not exist twenty years ago, right? Even if we had the compute available, we could not make it work because LLMs required large datasets. I mean, they're called large for a reason, right? We... We've gotten to this point of, what is it, forty years? What are we now? Twenty, twenty five? Let's say forty years of of the public Internet being this Sure. Repository of, you know, language training data. There are lots of other domains that are not language based, that do not have large datasets that are publicly available. And this is the second point that we've discovered along the way is that if you do a head to head comparison and there's a paper coming out actually, this was in Europe, it's about two years ago where we put it in the RL workshop as one of the posters. Did you took reinforcement learning agents? So we we did an experiment where we took three of them, DQN, which is your classic alpha go. It's kind of, you know, the... I guess the gold standard, it's kind of old now. There are better algorithms out there that are more what's so called sample efficient. And we put that against the biological system and we get to the same game. But we said, here's the one thing, we're gonna we're gonna constrain the reinforcement learning agents so that they get the same amount of data that the biological system gets. Right? Because if you think about reinforcement learning systems, nobody really talks about this, but the way they actually learn in a simulation is that they spawn millions of parallel processes and they speed up the time of the game by, you know, 200, 300 fold. And so if you read the AlphaStar paper or something like that, DeepMind kind of said that, you know, if you were to train a a thing or human to play at the amount of data that AlphaStar was playing, it would take four hundred years of continuous gameplay to get to that level of performance. And so what we did was we said, we're gonna temple constrain it because if you think about this, the real world, you can't speed up time. Time ticks at the same rate that you and I have. And if data is a factor of time, right? Say bit rate and so forth, then if you wanted to get, say ten minutes of data, you would have to spend ten minutes collecting that data.
00:28:40 · Hon Weng Chong, Alex Wilhelm
Knowing that these systems use far less data, when we did the comparison with the reinforcement learning agents and data being a factor of time, we realized these things are actually potentially very good for problems that don't have a dataset, problems that are real time specific. And that is actually the vast majority of the domains that have not actually been touched by language. [00:29:03] S1: This is when I get really excited. And I I don't I don't wanna make you project the future too much because that's not really fair. But to me, with with neurons that we can interact with with software, it seems like we've managed to take out the best of both worlds, the programmability of of computers as we understand them, and also the probabilistic low power consumption and kind of generalized intelligence of brains and brought them together, Hans. So to me, in time, it feels like we should be able to use both, you know, sure, our mass GPU clusters that everyone's building, but also we should have probably large tanks of biologic, you know, neurons that are also helping us do a lot of a lot of work. Is that too kind of science fiction y, or am I on the right path?
00:29:55 · S4, Hon Weng Chong, Brendan Foody
[00:29:55] S4: We're all familiar with AWS, Amazon Web Services. That's the cloud platform that powers so many of your favorite brands. But do you know about AWS Activate? That's their program for startups, where they provide up to a $100,000 in AWS credits for all startups. Whether you're backed by an investor or you're bootstrapping, money is time to keep innovating, time to delight customers, and time for you to keep gaining traction. You need runway, and AWS's Activate is going to help you with that runway. We hear this story from so many of our founding university companies. They're finding product market fit, the words getting out about their product, just a little boost, finding some savings here or there to get a little extra time can be the difference between getting traction and bringing in revenue, or hey, let's call it what it is, running out of money, okay, and shutting down. AWS knows this. That's why they've created the ultimate toolkit for early stage startups looking to boost growth. With AWS Activate, you're gonna get up to a $100,000 in AWS credits, hands on support, and training, plus exclusive discounts with some of our favorite companies and tools. So start getting the support you need at every stage of your startup journey. To learn more, visit aws.amazon.com/startups/credits. That's right. Aws.amazon.com/startups/credits. [00:31:16] S3: I think you're on the right path. I mean, thing about it is that it's... We're good at some things that machines aren't, and machines are good at things that we aren't. Right? [00:31:23] S5: Right. So this is called
00:31:23 · Brendan Foody, Hon Weng Chong, Alex Wilhelm
[00:31:23] S5: Right. So this is called [00:31:24] S3: Moravec's paradox, particularly in robotics. And I think this is the motivating factor for why Elon set it up, things like Neuralink, right, and the BCI people, which is that, as AI gets smarter and better, we also need to skill up, right? They're getting, so if you think about it, the machines are getting better at what we reserve as traditionally human or biologically centered tasks. It's not beyond the realms of possibility that we could also go the other way, assuming that DCR has really become a thing, where not only do we utilize our brains for the low power stochastic computation, the probabilistic computation, but we can also just, you know, have a... I don't know, an iPhone chip in there that can just give us the square root number at the speed of thought. [00:32:03] S1: Right. So I could essentially allow compute to handle the math that I don't wanna do myself, but all the learning, can use my brain for. So Correct. I... Once again, you guys are kind of Neuralink in reverse. So Yep. Look ahead of just a just a couple of years. Not super long term, but the c l one gets into the market next year, customers. You guys are also gonna have an API to let people access the technology remotely if they don't have the lab setup necessary to run this thing. Yep. What's the next generation? Like, what are you guys gonna build after the c l one? What is the c l two?
00:32:32 · Hon Weng Chong
[00:32:32] S3: Well, I think the c l two is... We've been thinking about it and there are a lot of features that didn't make it into the CL one that we'll probably start thinking about putting it into CL two. Mean, things, your standard things, right? Make it like smaller, make it probably cheaper to obtain, easier to program, maybe more modular units. There are quite a few things that we've been thinking about. Now the question is does it take the CL two or does it get pushed out to the CL three is a discussion for the team. But again, I think it's still a little bit premature. What we really wanna do is get the CL1 out to our partners, our research collaborators, and really get their feedback, what works, what doesn't work. So that way we can sort of triage and prioritize what we wanna do for the CL2. No point of putting in features that no one else wants. I mean, we might want it, but we have the ability to just spin up whatever, CL3, four, whatever internally. But if there are features that are missing that we have on our list that's kind of lowered down our sort of priority, but it would, you know, tremendously help the the community get going faster or do more work for less. You know, I think that's something that we wanna be prioritizing.
00:33:50 · Alex Wilhelm, Hon Weng Chong
[00:33:50] S1: You talked about using mice neurons at the top of the show, and anyone who knows anything about biology and experiments knows that a lot of a lot of mice and rats die. So that Yes. Is kind of par for the course for how we currently handle ethics. Are mice neurons and human neurons radically different? Are they relatively fungible? Silly question, but I'm not sure about the answer. Before I get to an ethics question, I thought we'd start there. [00:34:14] S3: So... And and this is the really interesting thing. We we used to think that there was no difference between a human and a mouse neuron. If you read any of the papers before, I think twenty twenty two, twenty, everyone said, yeah, there's no difference, all mammalian neurons are the same. And it turns out that I think on average, human dendrites are actually longer than mouse dendrites, but they have the same number of ion channels. So these are the gates that allow the potassium and sodium to go back and forth between the cell and the external environment. And that's what generates the action potential. But having the space out more, you actually have the ability to hold more so called electrical states. Well, that's the theory that's been pushed by some of the researchers at Harvard and MIT. And so, yeah, it turns out they are different and that probably contributes to the differential in the overall performance. And we've seen that as well. The human neurons are better at playing and processing information.