Entrepreneurial Thought Leaders (ETL)
00:42:50 · Michael Bernstein, Ravi Belani
[00:42:50] S2: Yeah. We absolutely compute our training data coverage. We've got all sorts of stuff. As I said, it doesn't matter what I show you. It's actually your own experience using the thing that builds your trust. So what many companies will do is they will actually... Once they get access to our tool, the first thing they'll do is start asking it questions that they know the answer to, but we don't. So they've been running some studies over the past whatever year or something like that, and they want to see what our system does. And I've had multiple users tell me, I didn't trust your system until I got my hands on it and I asked something that like was a private research that we had done and it just nailed it. And at that point, it's actually interesting, like trust goes way, way up and like you... Then gives you a lot of like mutual trust to to work from if later you need to talk about where are the boundaries. But many people are totally cool if you... As long as you're clear with them about, like, we think that these kinds of questions are good to ask the system. These ones are, like, again, like yellow light, red light. People are totally willing for us to be opinionated about about that as long as it's clear and they have a good mental model of what they should and shouldn't use the system for. You just call your shot. [00:43:57] S1: Thank you. We're gonna move on to the next question just because we have too many questions. I appreciate it.
00:43:57 · Ravi Belani, S5
[00:43:57] S1: Thank you. We're gonna move on to the next question just because we have too many questions. I appreciate it. [00:44:01] S5: Hi. Thank you, professor Bernstein. My name is Michael. I'm working on collective behavioral simulation with my friend James, and we're wondering in particular about tipping points because tipping points have outsized influence on crowd behavior. And I'm wondering what your thoughts are on perhaps probing a simulated collective with these exogenous shocks to try to predict, prevent, or at least, you know, alert of potential tipping points?
00:44:22 · Michael Bernstein
[00:44:22] S2: Yeah. Exogenous shocks are a little bit different than a tipping point, but generally I think what I take your question to be asking is when does a product go from out there to now everyone's expecting everyone else to use it? That requires simulation that is longitudinal, like over time. Remember that the world is not predetermined. It's not like I've produced some crystal ball that's going to call exactly what's going to happen. What I should be able to do is give you the space, like the distribution of likely outcomes. How likely is this to happen or that to happen? What are the accelerants? What are the decelerants on that? And I think that helps people, again, if they can interactively play with this, they can build an intuition for what's really driving things. So, I mean, these kinds of tipping point behaviors, mean, it's a collective action thing. There are very clear closed form models that predict this, but what we want to do on our side is actually we want it to be an outcome of having trained our model to predict literally everything else about human behavior. Some of the first things that Joon did when he was doing some of the research is he found that he could recreate the sort of link structure of the web or recreate studies of social influence where songs become popular. These are all things that you should be able to test your simulation and prove that we can recreate even if you're not tipping the scales to say it should do that. Yeah. This is why our paper was originally... One thing that went viral about our paper was that the agents threw a Valentine's Day party. Right? We had inserted literally one memory into one agent saying, hey, you're interested in planning a Everything else from there, where word got out and then one agent asked another one out on a date and this kind of stuff, that was all emergence. That was the entire point. We didn't tip the scales.
00:46:08 · S5, Ravi Belani, Michael Bernstein, Beckett
[00:46:08] S5: Okay. Thank you very much. [00:46:09] S1: Cheers. We'll do a final question. And again, please don't talk if you can. We can hear it up here. [00:46:14] S2: Except you. You can talk. Yeah. [00:46:16] S1: Yeah. [00:46:16] S6: Hi. I'm Beckett. And for the fourth level of simulation you're talking about with the world, like, dynamic world Yeah. I was wondering how you guys would think about approaching that, because I would assume as there'd be some small errors that would accumulate over time, and also what kind of time scales you guys are thinking about simulating with this dynamic world. [00:46:34] S2: Yeah. I mean, I think we want to be as ambitious as possible, but again, trust is the ultimate thing. Right? And when you're talking about emergent multi agent simulation, you're like, how would you validate it? Because the world only unrolls one way. Right? So maybe you predicted something else, but maybe you're not wrong. Maybe that was as equally likely. There are different ways you can trust this. One is to say, look. [00:47:02] S1: Fifteen seconds. [00:47:04] S2: You can make the individuals so accurate that you don't get a game of telephone. The other way is that you take the complex systems thing and just think of it as phase changes. There might be errors, but are they so big that they would change the phase, like, you know, that the solid becomes a gas?
00:47:04 · Michael Bernstein, Ravi Belani, S0
[00:47:04] S2: You can make the individuals so accurate that you don't get a game of telephone. The other way is that you take the complex systems thing and just think of it as phase changes. There might be errors, but are they so big that they would change the phase, like, you know, that the solid becomes a gas? [00:47:20] S1: Interesting. Fifteen seconds. Thank you. That was fantastic. Thank you, everybody, for Thank your you, Michael, for that awesome share, for all the insights. And we'll see you again next week where we're gonna be talking about math and AI. [00:47:33] S0: Entrepreneurial Thought Leaders series is an original production of Stanford eCorner and the Stanford Technology Ventures Program. To learn more, please visit us at stvp.stanford.edu/ecorner.