"The Cognitive Revolution"
00:31:06 · Samvit Ramadurgam
[00:31:06] S3: I think that over the next five years, I think, you know, you guys have probably really bought into this, the world is gonna get very interesting, and there's gonna be a lot of surprises that come from the capabilities of AI. I think we're seeing this right now with mathematics. Like, it's completely incredible. Like, that... Like, math has been revolutionized so quickly, and it's just the beginning of it. And so, like, that's about to happen to many domains. And and so I think you could expect, like, what happened in math and coding in the last year to happen to as as domains that are less and less trivially verifiable with software. The further we get from verifiability, the longer it may take. But this... It's gonna get there because the feedback loops, are gonna be there just like with with longer feedback loops. And so what does that mean for us? Well, I think that one very big surprise that will hit the world is that at some point in the next few years, people will wake up and then realize that huge number of the world's markets are being coordinated by agents. Thousands of markets from, like, you know you know, dairy in, like, a a a country all the way to real estate in in China to, like, the the the pre IPO market in The US. Like, all these markets are gonna be coordinated better with agents, like, period. Because what it is... The way it's being coordinated right now is that there are human... Gonna be humans who are really good at what humans are really good at, building relationships, having taste and judgment, and and advising their friends and trust. Humans will have a place, and they'll be, like... I think they'll to actually... The best ones will actually, I think, do better. I think by and large, most of the work is data coordination, connecting the dots, logistics, operations, orchestration, all of which software and agent's gonna be way better at. So, yeah, we'll have, like, a a period where the same way that software and math is really getting taken over, one of the surprises is that a lot of this deal making that makes the economy function is actually something that agents are quite amenable to that we're focused on.
00:33:03 · Nathan Labenz
[00:33:03] S0: I asked whether he saw reason to worry about disempowerment. What he describes sounds empowering, but some would hear it as the agents running the world.
00:33:14 · Samvit Ramadurgam
[00:33:14] S3: My big take is actually that, like, right now, we are really disempowered because we don't have the ability to coordinate in a more open decentralized way. Like, a lot of the coordination is happening, like, really top down by, like, companies with a lot of, like, power and a lot of, like, brand power and things like that. And, like, the the vision for this now is that you can actually have, like, grassroots movements come up very easily. So you can imagine, like, a a world... If you take the political example, which is more pertinent to the power question. Let's suppose that I wanna start a new movement for, like, some initiative. I have, like, some political cause. Like, for me to be able to go and light a spark today on social media lets me be empowered to a degree to go start that. That gets amplified much further. In addition to letting me get in a forum with people, I can actually, like, coordinate activity. We can, like, coordinate movement of money and signature of documents and, like, make decisions together and, like, take actions together. So the ability to go take collective action becomes very strong. So I think that it's a really powerful tool for empowerment to... If if you have something that can tap into what a lot of people want. And, like, that empowerment is, like, in a way, like, the bigger risk because you can empower things that are are harmful. I do think this probably is worth going deeper into the disempowerment side as well. I think that if you're I think if you're a human being who is in the business of doing a lot of, like, logistical coordination, I think that that's a rough place to be because the AI is gonna be better at, like, the kind of, like, transactional mundane logistics by, like, a 100 x better than a human. This is not even gonna be close. I think if you hear a human who is operating with a level of taste and relationships and judgment, you're at, like, the the the thing that humans are gonna uniquely... Humans are... I think humans have a monopoly on one thing, and humans have a monopoly on their individual life story and life perspective.
00:35:03 · Nathan Labenz
[00:35:03] S0: I think the general concern is always just that we have so much agent activity flying around, doing so many things. You know, we can't really make sense of it anymore. And before you know it, we're kind of in a zoo of our own creation where the AIs are really running the show and, like, hopefully, we're, like, cared and fed for, but we sort of lose the ability to even understand, let alone, like, rearchitect the superstructures that surround us.
00:35:31 · Samvit Ramadurgam
[00:35:31] S3: Yeah. Yeah. I think I think that that ties to, like, one of the questions on AI alignment. So AI alignment, I think, has, like, two parts. There was, like, have an AI execute on the genuine intentions of the person who asked to do something in a in a skillful way. To avoid the paperclip stuff and all that. I think that's one one part of it. The other part of it, which I think people talk less about, is obviously, like, who are you aligning the AI to? Like, what what is it being aligned to? Especially when people have disagree over what... How the world should be. And so in that world, I think you need a system that can actually get a measure in some sense of, like, what collective well-being means. And I think the way you get a measure of collective well-being is by having a system that's, like, deeply tied into the lives of many people. It's helping them, like, with their work, helping them do, like, the important stuff that they have to do. So I think that you can't... You cannot optimize something you can't measure. And so in a world where you have, like, agents that are coordinating the world, you have no idea, like, what direction that's headed in, like, what the... Know, you this kinda be like... If you imagine, like, a hand of the market is guiding the market, like, what is the direction? The hand is guiding it. If we don't know and we have no way to measure it, then I think we're in trouble. But I think to get to that as as a certain point, you have to have a really refined understanding. Like, what am I measuring? What am I optimizing for? And I think what we've we've learned mistakes from the social network world, which is that any measurement you can make, which is an objective measurement, is wrong if you're trying to optimize for certain click through rate or engagement rate or metric. Any metric you can measure is wrong because human happiness is, I think, fundamentally immeasurable. But there is a thing you can do, which is you can continually converge towards a bet... A better way of, like, measuring. The the the the the kind of, like, the way to optimize a system is not to have a static metric optimized for, to have it be continually trying to discover, like, an optimal point. Have it, like, converge towards an asymptote of, like, this kind of perfect behavior.
00:37:26 · Samvit Ramadurgam, Prakash Narayanan
Even even in the world of of behavior, I think that there's two components to good behavior. Having good intention and then just making sure you had a good outcome. Having... You can have... You can do something that's well intended and have that cause harm, and it was still, a bad action. And so being able to measure the outcome of your action at a systemic level is totally missing from the kind of zeitgeist of alignment. I think that we would hope to be in a place to contribute to that. [00:37:51] S6: What's your token usage like? [00:37:53] S3: It is very high. I think that we... So we have we have, like, an internal target we're setting for our team. This is obviously not what we're... What... Like, the way to optimize for productivity. As, like as, like, a correlate, we we want our team usage on the on the coding agent side. Each person be doing, like, a little... At this... At the current we have our infrastructure set up, like, between one and two billion tokens per day. [00:38:15] S6: One and two billion tokens per day.