00:00:00 · Molly O'Shea, Joon Sung Park
[00:00:00] S0: Elon Musk thinks there could be a probability that we live in a simulation. [00:00:04] S1: You know, I do think we live in a simulation. So, Simile is a company that is creating foundational model of human behavior, partnership with over 40,000,000 real people. For the first time really in the field, we could show that we can predict people's behavior with 85% plus accuracy. [00:00:19] S0: You have raised 300,000,000 in total at evaluation of 2,000,000,000 reported. You have really great names like I mentioned, Green Oaks, Index. So what happens if someone comes to you and offers you $15,000,000,000 to sell your company? June, welcome to Sorcery. [00:00:46] S1: Thank you for having me. [00:00:47] S0: I know we're here in Index's office, someone who is a big contributor or investor in your company. Congratulations on the recent large announcements. Well, today I'm like super excited. This is an area I don't really quite understand. I don't really understand where the company is positioned in the market, like what you're replacing, what you're now making, like I don't know exactly what simulations are, but I find it fascinating. Elon Musk thinks there could be a probability that we live in a simulation. So what does that mean to you?
00:01:24 · Joon Sung Park, Molly O'Shea
[00:01:24] S1: Well, you know, I do think we live in a simulation. You do? I do. Explain. Well, and of course, not surprisingly because of the company that we're building and technology that we're building, we do get asked this question quite frequently. At some point I started to just say, Hey, you know, we live in a simulation. But I think the part that actually does matter more to me is at the end of the day, is the experience that we're having actually real? Does it feel real to us? Is this true to who we are? I think at the end of the day, that's what matters. Probability wise, I think it's a pretty good argument that humans and societies like ours will always have desire to better understand itself. And what we found is the best way to understand something is to recreate it. And one way I sometimes describe what simulation is, is it's a little bit like creating a portrait of who we are as a society. So I think this is actually a very innate desire for humanity or civilization like ours to have. In which case, then yes, any civilization would create a simulation, and at some point it likely is the case that we live in a simulation. [00:02:30] S0: So how in real world are we in a simulation? Like what would make that up? And then we'll get into what you're building. [00:02:37] S1: Right, and tell me a little more about what makes up the simulation. What do you mean?
00:02:37 · Joon Sung Park, Molly O'Shea
[00:02:37] S1: Right, and tell me a little more about what makes up the simulation. What do you mean? [00:02:42] S0: Like, could humans be living in a simulation? Just like we all have the same, we're all programmed the same way, we all have the same psychological behaviors, like it's just everything is kind of already written, like what does it mean? [00:02:55] S1: So the way I often think about a society or an experience like what we are having right now is there's always this quantum building blocks of people's tastes and behaviors. So it might actually be things like my inherent characteristics. People's risk tolerance, not really known to change. There are certain aesthetics that I really enjoy. And these all come together and rebuild individual building blocks of these larger societies, which is individuals. So really, when we think about simulations, we think about individuals first. Can we create simulations of individuals that faithfully and accurately represent a diverse persona and characteristics and behaviors of that person? Then to create a civilization wise simulation, that basically means we put all these people into a same environment, let them interact with each other. And the part that's always complex and chaotic is what actually gets born in their interactions. So that's ultimately, I think, what building simulation here in terms of cytoscalability is going to look like. [00:04:01] S0: And so this all started with Smallville? It did. Okay, explain what that is and like your whole background behind that.
00:04:10 · Joon Sung Park
[00:04:10] S1: For sure. So Smallville was an experiment that we run back in 2023 and 2022, where we had the initial observation that these large language models, initially we were using GPT-three and then GPT-3.5, the models were fairly janky. But you could see a glimpse of opportunity where these models were trained on such broad human data that when you poke at the right angle, you can actually extract a lot of human data that appeared in its training data. This might be from the web, Wikipedia, some of the social media. And what we really start to ask ourselves is, can we actually use this to create an interaction paradigm that is truly new, that goes beyond just simply generating context or classifying different tasks, but can we actually generate truly new behaviors? And we decided that the most ambitious version of this is actually to create a simulation of an entire person. And then once we had an entire person simulated, can we actually go create an entire community? So Smallville is an experiment that came out of that effort. So it was a game town. Computer scientists love games because it's an amazing way to demonstrate new capabilities. So we created a game town that looks a lot like what Pokemon, Sims, like these kind of towns might look like. And then we populated this game town with 25 agents. And these agents all had different names, different characteristics, different life stories. And they were basically language models paired with what we called back then an agentic architecture that had memory, planning, and reflection. So they would actually wake up in the morning, do their routines, go to work, have relationships. The simulation itself actually takes place the day before Valentine's Day. So these agents will actually come together to have party. So there is a character that's actually organizing Valentine's Day party. There are characters that are asking themselves out. And none of this was preprogrammed, but what we could see is that we can basically create simulations of these people with emergent phenomena just coming out of the simulation. So that was the experiment that we ran in 2023. That was Smallville. That particular work ended up introducing a few key concepts. One was that we can create simulations of people, and the other was the core agentic architecture and that we can create agents using generative AI models.
00:06:29 · Joon Sung Park, Molly O'Shea
And it was very early days of these agentic workflows, So we could really start to see them becoming possible. And then we've been building on the technology basically for us to then create simulations that are more and more accurate and that's more diverse and that's bigger in scale. [00:06:46] S0: How do you test the validity of this? Like, do you know if it's actually working?
00:06:51 · Joon Sung Park
[00:06:51] S1: So our ability to test actually has shifted quite a bit also alongside the technological progress. On day one, when we created Smallville, what we actually did was we interviewed them. We interviewed the agents. What we've done is we pulled individual agents out of this game town, set them in this hypothetical chair on a command line, and we would basically ask them questions. And these questions would probe certain qualities of these agents that's similar to how people would reason about different tasks in their lives. So simple question, tell me your name. And then we would ask, tell us your life story. We would then ask questions like, who's the one person you want to spend time with? And then we would then ask, if you had to get that person a present, what would you get them? We would ask these questions because these are questions that when you think about human brain and human mind, we can only answer these questions because we can remember, because we can plan ahead, but also because we can reflect. Then what we ended up doing was we generated all these responses, and then we actually had human evaluators come in and take a look at these responses and said, given the experience that the agents had in their game town experience, are these experiences or insights actually human like? So we would actually basically have human evaluators compare between human authors' generation and these agents' responses to see which one's more realistic. And we've actually done quite well that these agents were actually quite realistic. And the way to sometimes tell is, imagine you're asking this agent, who do you want to hang out with the most? It's a little bit strange if the agent says whoever they happen to see the most on the street. The agent should likely say the person they actually care about. Who do you have special relationship with? What kind of conversations did you have with people? What actually stood out? And that agent should really reflect on that and pick those people. So those are some of the experiments that we ran, and those are the signals that we were able to get. But then over time, we wanted to go beyond that. So here, really the initial evaluation really was about believability that we start to see what we call the illusion of life in these agents.