Entrepreneurial Thought Leaders (ETL)
00:39:23 · Ravi Belani, Michael Bernstein, James
[00:39:23] S1: And I know there's a long pipeline, so I'm gonna discuss one more signature question. We'll move to the students, which is what have you refused to compromise on and what has that cost you? [00:39:31] S2: I think throughout my career and even since starting Simile, there is a real temptation to get pulled toward things that can be sort of narrowly defined technically. Are you, you know, benchmarking on this particular benchmark that you care about? You know, can we solve math proofs? Can we like... I've been told so many times early in my career especially, oh, what you're doing, that's not computer science. I like... I think that people have these narrow blinders on around what problems matter, and I think as soon as you take those off but you still have the technical depth to be able to actually tackle it, that is the thing that, like, I just... I have not compromised on. Like, it is wild to me that when I was an undergraduate, I was told that what I was doing was not science, and here I am now a professor of computer science. That's wild to me, but don't compromise on it. [00:40:27] S4: Don't comp... That's great. Thanks a lot for your talk. I'm James. Hi. My classmate Michael and I are building in the behavioral simulation space, and we have a question for [00:40:35] S1: And, Ging, if you [00:40:35] S4: can keep... How do you ensure that contrarian or abnormal or pathological human behaviors, which can sometimes cause, like, BACS one zero one events, are accurately modeled in your, like, data?
00:40:35 · James, Michael Bernstein
[00:40:35] S4: can keep... How do you ensure that contrarian or abnormal or pathological human behaviors, which can sometimes cause, like, BACS one zero one events, are accurately modeled in your, like, data? [00:40:45] S2: Yeah. So how do you get these long tail events modeled? Like, the post training procedures behind most language models really... Like whatever the distribution would be in this... In society, they push the the model toward very high conviction at, like, sort of the mean. This is exactly what you don't want for a simulation engine. But there's no... Nothing written that says it has to work that way. Essentially, what it requires is that you have training data, again, this sort of like dark matter that I'm describing, that teaches the model what the actual population distribution is. It should know that there are outliers who are whatever you're saying, trolls or people with unusual opinions. Actually, can give you an example. We spoke to one of these early, early on in Simile's days. We spoke to someone who was like head of data science at a really high end fashion firm and they didn't care about what the average person... They're like, show me the tails. Because we don't... They're like, we don't care what people think, but we're interested in what people are doing. That's like the unusual stuff because that's where we get inspiration. So if you want have simulation work, you have to focus... I think it's easy to focus on the mean, but it's the variance that actually, in some sense, really matters.
00:40:45 · Michael Bernstein, Ravi Belani
[00:40:45] S2: Yeah. So how do you get these long tail events modeled? Like, the post training procedures behind most language models really... Like whatever the distribution would be in this... In society, they push the the model toward very high conviction at, like, sort of the mean. This is exactly what you don't want for a simulation engine. But there's no... Nothing written that says it has to work that way. Essentially, what it requires is that you have training data, again, this sort of like dark matter that I'm describing, that teaches the model what the actual population distribution is. It should know that there are outliers who are whatever you're saying, trolls or people with unusual opinions. Actually, can give you an example. We spoke to one of these early, early on in Simile's days. We spoke to someone who was like head of data science at a really high end fashion firm and they didn't care about what the average person... They're like, show me the tails. Because we don't... They're like, we don't care what people think, but we're interested in what people are doing. That's like the unusual stuff because that's where we get inspiration. So if you want have simulation work, you have to focus... I think it's easy to focus on the mean, but it's the variance that actually, in some sense, really matters. [00:41:52] S1: Thank you. We'll go to the next question. And, if you can keep keep the conversations... Actually, don't talk if you can because we can hear everything up here. And if you do need to leave, just try to leave quietly. Thank you, gang.
00:41:52 · Ravi Belani, Michael Bernstein, James
[00:41:52] S1: Thank you. We'll go to the next question. And, if you can keep keep the conversations... Actually, don't talk if you can because we can hear everything up here. And if you do need to leave, just try to leave quietly. Thank you, gang. [00:42:02] S2: Thank you. I appreciate it. [00:42:03] S1: Next question? [00:42:04] S4: Cool. Yeah. I want to jump back to you talked about a lot about trust, right? And building trust is a big challenge for you. I personally would have a big challenge in how just some lights, like an orange light, and some validation on maybe a product which is unrelated to the product I'm thinking about launching is conveying enough trust that I'm now convinced, okay, your model will actually help me planning this campaign and launch 2 millions on it or something. Yeah. So what types of guarantees do you see people requesting? And you also being able to provide to them to make them understand what your model can and can't do. Yes. So are you able to compute actual training data coverage or what amount of the population they are interested in is covered or something like this?
00:42:50 · Michael Bernstein, Ravi Belani
[00:42:50] S2: Yeah. We absolutely compute our training data coverage. We've got all sorts of stuff. As I said, it doesn't matter what I show you. It's actually your own experience using the thing that builds your trust. So what many companies will do is they will actually... Once they get access to our tool, the first thing they'll do is start asking it questions that they know the answer to, but we don't. So they've been running some studies over the past whatever year or something like that, and they want to see what our system does. And I've had multiple users tell me, I didn't trust your system until I got my hands on it and I asked something that like was a private research that we had done and it just nailed it. And at that point, it's actually interesting, like trust goes way, way up and like you... Then gives you a lot of like mutual trust to to work from if later you need to talk about where are the boundaries. But many people are totally cool if you... As long as you're clear with them about, like, we think that these kinds of questions are good to ask the system. These ones are, like, again, like yellow light, red light. People are totally willing for us to be opinionated about about that as long as it's clear and they have a good mental model of what they should and shouldn't use the system for. You just call your shot. [00:43:57] S1: Thank you. We're gonna move on to the next question just because we have too many questions. I appreciate it.
00:43:57 · Ravi Belani, S5
[00:43:57] S1: Thank you. We're gonna move on to the next question just because we have too many questions. I appreciate it. [00:44:01] S5: Hi. Thank you, professor Bernstein. My name is Michael. I'm working on collective behavioral simulation with my friend James, and we're wondering in particular about tipping points because tipping points have outsized influence on crowd behavior. And I'm wondering what your thoughts are on perhaps probing a simulated collective with these exogenous shocks to try to predict, prevent, or at least, you know, alert of potential tipping points?