00:15:03 · Joe Weisenthal, Nate Silver, Nate Silver, Tracy Alloway
[00:15:03] S3: Oh, Joel's been on our podcast. [00:15:04] S2: He's great. Yeah, and clearly, I think the University of Michigan survey changed the composition to online in a way that just has, like, way too many Democrats in the sample. So, like, ironically, like— Ironically, like— The complaint that people make about election pollsters, which is more or less false, we can talk about that more, is kind of true of these consumer sentiment surveys that like, for better or worse, election polls get a dose of reality every two years. Election polls are made in an environment where pollsters are actively fearful of missing, particularly missing high on Democrats, right? So that's why like, that's why people talk about like, what's the difference between like polls and prediction markets? Polls kind of are prediction markets, right? or polling averages are at least, right? [00:15:47] S1: Okay. [00:15:47] S2: A poll is a data-informed opinion at this point, right? [00:15:55] S0: Because you have to weight it. [00:15:56] S3: The pollster has to do so much modeling, right? Is this what you're getting at? [00:16:00] S2: So like Marist released a poll of Maine last week, and the top line number, let me see if I'm getting this right in my head, said Troy Jackson, the Democrat in Maine, is up nine points, which would be a very good result for Democrats. The average, our average, had been like D plus three in that state, right? That's their registered voter poll. They applied a likely voter model that brought that lead down to plus two.
00:16:00 · Nate Silver, Tracy Alloway, Joe Weisenthal, Nate Silver
[00:16:00] S2: So like Marist released a poll of Maine last week, and the top line number, let me see if I'm getting this right in my head, said Troy Jackson, the Democrat in Maine, is up nine points, which would be a very good result for Democrats. The average, our average, had been like D plus three in that state, right? That's their registered voter poll. They applied a likely voter model that brought that lead down to plus two. [00:16:21] S0: People self-correct for the Democratic. [00:16:24] S2: Bias, right? Yeah, there's no way. First of all, the math, just I don't want to get too deep in the weeds, even though it's odd lots. Like the math doesn't, the math doesn't make sense, right? They decided what result they felt comfortable with and reverse engineered. [00:16:36] S3: You see, we just, okay, we don't have to get into the math. But what you're saying is they had a poll that said Troy Jackson was up 9%. But in our likely voter model, it's 2%. And what Nate Silver is saying is like, I don't know how you got from nine to two. [00:16:51] S2: No. And the governor's race went from 11 to zero. Like the math literally doesn't add up. And like, so it's like, okay, so this is your poll and this is your opinion. [00:16:59] S0: Yeah. [00:16:59] S2: Is the way I see it. And like, and we're aggregating opinions. And so like on the one hand, when you see a poll like that, I'm like, I have existential doubt about the accuracy of polling for sure. [00:17:08] S1: Right.
00:17:08 · Nate Silver, Nate Silver, Joe Weisenthal
[00:17:08] S1: Right. [00:17:08] S2: On the other hand, if you have a model that's like a consensus model, thermometer that looks at the correlations and translates poles into probabilities. Like, that's really what Flipper, my model, is doing. And, like, so, like, oh, actually, I kind of like that from a modeling standpoint because it proves the kind of underlying hypothesis that, like, polsters are rational actors and, therefore, although poles are imprecise instruments, there is a lot of incentive not to systematically miss. I got it. [00:17:39] S3: So what you're saying is because you can look at the math and say, look, You can't really justify turning a plus nine into a plus two as someone who aggregates polls and then creates a model that can give you some sort of confidence that like. [00:17:56] S2: Confidence that the errors in the long run, if you have elections every two or four years, will never have data to prove it. But confident that the errors are symmetric in the forecast. [00:18:05] S3: Yeah, I get exactly what you're saying. So, OK, people don't pick up the phone like they used to. [00:18:10] S1: Yeah.
00:18:10 · Nate Silver, Joe Weisenthal, Tracy Alloway
[00:18:10] S1: Yeah. [00:18:10] S3: You know, certainly they don't recognize it. And if someone calls me and asks for my political opinion, no way. So then there's this vision. Well, why don't we just create an AI replica of Tracy Alloway and an AI replica of Joe Weisenthal and an AI replica of a high school educated oil worker in Texas and an AI replicated version of a HR professional who works for General Motors in the Detroit suburbs. And we just pull them. And then we don't know. And they're always going to pick up the phone. What's wrong with this? [00:18:42] S0: This is the digital twin idea. [00:18:43] S3: Why don't we just create a digital twin of all America?
00:18:46 · Nate Silver
[00:18:46] S2: So the problem is like, and by the way, I think these companies are smart. I know some people that work for them, right? But like polling cannot be outsourced to AI agents, right? Because it's one of the few sources of ground truth that we have anymore, number one. And also the models are kind of, so if you ask a poll to create like a synthetic Hispanic respondent, right? 99% of their synthetic responses Hispanic respondents love soccer. It's a bit like if you ask like Claude to pick a number from zero to 10, like always pick seven. It's a bit like, I mean, this gets into the whole existential thing about how like large language models work and like they're kind of, it's kind of the ideal type or something, but like, but no, we need original data, right? Models like Flipper or prediction markets can aggregate that data, but, but, you know, because like voters aren't all that predictable. You know, I don't even like the kind of slicing and dicing that you see. Oh, the Hispanic vote, right? Well, the Hispanic vote is younger Hispanics and older Hispanics, people who are relatively new to the country and people who have been here for a long time, right? Cuban Americans and Puerto Ricans and Mexican Americans and Dominicans and everything else. And like those groups don't always agree with one another. Sure. And people miss it. Like the Hispanic vote is one of the most swingy demographics in the United States. And so like, yeah, it just gets overly reductive and kind of almost like a snake eating its tail a little bit.
00:20:08 · Tracy Alloway, Nate Silver, Nate Silver, Joe Weisenthal
[00:20:08] S0: I feel like turnout in particular would be difficult for a digital twin to model? Like maybe you can come up with political preference based on demographics, but like predicting whether someone is actually going to leave the House and vote seems particularly difficult to me. But tell me if I'm wrong. [00:20:23] S2: Yeah, we did a little simulation in South Carolina, which is one of the more surprising races to be polling closely. And, you know, and one more, frankly, be a bit more surprised Democrats won. [00:20:32] S1: Right. [00:20:32] S2: But like, you know, what if black voters in South Carolina turned out at 2024 rates and white voters turn out at 2022 rates, like that shifts the top lines by like 10 points, right? It's a deliberately extreme example. And in South Carolina, the white vote is very conservative. But, you know, that's an example of like, you know, because you have had some midterms where 2018 in particular, you had some states getting up to presidential type turnout. But if it's differential turnout, that would you need that for a state like South Carolina to flip. [00:21:04] S3: The Democrats have 14 percent in Mississippi. on polymarket. [00:21:07] S2: Yeah, that one's a little weird. [00:21:09] S3: These are funny states. These southern states are pretty funny, right? Because the top line votes are actually offered frequently fairly close. It's just that every white is a Republican and all the non-whites are not Republicans. And so while it looks really close, it's just like, it's just purely demographics.