https://fivethirtyeight.com/features...out-the-polls/
538 podcast looking at polling error.
https://fivethirtyeight.com/features...out-the-polls/
538 podcast looking at polling error.
Everyone should seek out the Smerconish interview with the Trafalgar senior pollster from this morning's show. I know many don't want to acknowledge him, but he's gotten things right over the last several years that others have completely missed. He has an interesting take (one that I also think I'm seeing here in Florida*) about what's going on with some portions of the GOP voter base.
*Note that me saying I'm seeing something does NOT mean I like what I'm seeing or am in any way happy about it.
I will watch, but pushing back against the gotten things right thing. There is no proof his use of "social desirability" theory is correct. Since 2016 his basic polling numbers have shown the same thing everyone else's have. He adjusts those numbers towards Republicans because he feels like it's closer than the poles are showing. The last two presidential election cycles have had a democratic bias. But we don't know that isn't just happenstance and that he's gotten lucky. Maybe he has stumbled onto something. There's just no proof this is correct. And I'm very skeptical of someone who adjusts based on a feeling.
So, the other polls missed in the wrong direction but are somehow more trustworthy? Help me out here. If I'm a client, I'm paying for right answers. I don't care if you introduce the element of art to the mathematics* as long as the information you provide is accurate - or more accurate than the others in the final analysis.
While pollsters will talk all day long about methodologies and data sets, at the end of the day, they are polling human beings - irascible, contrary, prone to fibbing - which muck up their pretty models and pollute the data sets. That doesn't include biases introduced into the model. Over-sampling R's to make a race a toss-up versus a lean-D, for example. (Cook Political Reports moved OR Governor contest to Toss-up. I haven't seen the internals, but put me in the highly skeptical category for this one.) The element of art when evaluating human beings is important.
So, back to the point bolded above, help me out because it sounds a lot like "It isn't flying, it's falling with style."
*(Ignoring that mathematics has its own beautiful art.)
Hold up. This is like you saying Duke will beat Carolina 100 to nothing and me saying Carolina will win by one but they lose by one and you thinking you got it right. So yes, I would have have been waaaaay closer than you to the outcome. He's not just calling the election. He has a systemic Republican bias of three to five points.
Trafalgar outperformed "conventional polling" in 2016. Hard to tell if they are on to something or just got lucky* that particular year.
Since then Trafalgar has been about as good as 538, they just happen to miss in opposite directions. As far as I can tell, Trafalgar has overestimated R performance by 2.5% in their final 2018-2020 polls. In these same races, 538 overestimated D performance by 2.3%. Trafalgar did a bit better than 538 in 2020. 538 did a bit better in 2018.
Here is a comparison of Trafalgar and 538 in 2018-2020 races polled by Trafalgar.
Predicted Republican Margin Republican Bias Race Trafalgar 538 Actual Trafalgar 538 FL-Pres (2020) 2.1 -1.5 3.4 -1.3 -4.9 GA-Pres (2020) 4.3 -1 -0.2 4.5 -0.8 NV-Pres (2020) 0.7 -6.1 -2.4 3.1 -3.7 PA-Pres (2020) 1.9 -4.8 -1.2 3.1 -3.6 OH-Pres (2020) 4.8 0.6 8 -3.2 -7.4 MI-Pres (2020) 2.5 -8 -2.8 5.3 -5.2 NC-Pres (2020) 2 -1.7 1.3 0.7 -3 AZ-Pres (2020) 2.5 -2.6 -0.3 2.8 -2.3 WI-Pres (2020) -0.4 -8.2 -0.6 0.2 -7.6 MN-Pres (2020) -3.2 -9.1 -7.1 3.9 -2 LA-Gov (2019) 1.2 -1.8 -2.7 3.9 0.9 FL-Gov (2018) 3.4 -4.2 0.4 3 -4.6 FL-Sen (2018) 1.7 -3.2 0.1 1.6 -3.3 AZ-Sen (2018) 2.1 -1.7 -2.3 4.4 0.6 TX-Sen (2018) 8.9 4.9 2.6 6.3 2.3 MT-Sen (2018) -1.1 -4.8 -3.6 2.5 -1.2 MI-Sen (2018) -9 -11.3 -6.5 -2.5 -4.8 OH-Gov (2018) -3.6 -1.6 3.7 -7.3 -5.3 MO-Sen (2018) 3.9 -1.2 5.8 -1.9 -7 GA-Gov (2018) 12.3 2.2 1.4 10.9 0.8 NV-Gov (2018) 2.1 -0.3 -4.1 6.2 3.8 NV-Sen (2018) 3.3 -1.1 -5 8.3 3.9 SC-Gov (2018) 16.2 13.4 8 8.2 5.4 ND-Sen (2018) 9 4.6 10.8 -1.8 -6.2 Average
2.5 -2.3
Source data from 538
FWIW, Trafalgar performed poorly in the they polled in 2017 and 2019.
2019 KY Governor (prediction R+5.9, result D+0.4)
2017 AL Senate (prediction R+5.1, result D+1.6)
2017 VA Governor (prediction D+1.6, result D+8.9)
I can't find 538 predictions for these races, but "conventional polling" appears to have outperformed Trafalgar in these 3 race. For example, the Cook Political Report classified the 2019 KY Gov & 2017 AL Senate races as "tossups" and the 2017 VA Gov race as "Lean D".
*In what is either a sign or a giant cosmic coincidence, the song "Get Lucky" by Daft Punk came on the radio the moment I hit submit after editing this sentence.
Last edited by House P; 09-17-2022 at 01:16 PM.
Ha! In an effort to appear as argumentative as possible, 538 is an aggregator and not a polling site. It seems to me that a more accurate comparison would be to compare Trafalgar with a higher rated polling site. 538s aggregation takes Trafalgar into consideration. So Trafalgar polls are a data point encompassed in the 530 polling average/trend. I promise I'm not arguing. I just think 538 should be compared to a site like Real Clear Politics, while Trafalgar should be compared to Survey USA or Marist College
Traditional polling has performed quite well in non-Trump years. I don't believe in the shy Trump voter theory that Trafalgar bases their polling on. I do believe Trump can get people who have traditionally been apolitical to show up to the polls. Which would explain why Trafalgar has guessed fairly correctly in Presidential years. No one is finding those low propensity voters.
Last edited by ClemmonsDevil; 09-17-2022 at 02:47 PM. Reason: Fat fingers
I'm not here to argue how or why or to what degree Trafalgar got things right in 2016 and 2020 (more so than some other pollsters). I'm only suggesting folk listen to that portion of Smerconish's show today because I thought it was an interesting, and insightful, interview.
I'm curious why you wouldn't forecast error by state size? Missing in Montana by 2 pts means you might have been off by 2k voters. Missing in GA by 2 pts might mean being off by 30k voters.
Well, that 100 point differential is a nicely constructed strawman. You were not originally arguing the magnitude of variance, just that Trafalgar has their thumb on the scale to one side and, by virtue of superior intellect or dumb luck, managed to be closer in some years. My point is simply that if they give better data, they win. They don't, they lose. Each poll puts their thumbs on the scale by design of poll questions, sample selection, weighting, and a hundred other ways. Singling out a single polling operation for bias when they all have innate biases seems unproductive.
This pretty much matches my philosophy. I don't trust pollsters. Not Trafalgar. Not Marist. Not Gallup. Not Emerson. Not CNN (sorry, Jason.) If polling were truly scientific, we would see highly similar results. Maybe not to the extent of 9.8m/s^2 consistency of results, but the variances would not swing by upwards of 10 percentage points in the same race depending on who is performing the polling. As an example, I've seen Hochul up 5% and up 15%. That range is outside the combined margins of error. Somebody is wrong, probably by a lot. Maybe both are wrong, in different directions. As I said above, all the scales have thumbs on them.
Among new voter registrations, women outnumber men, by a lot. Women and their feelings about access to abortion will decide most races this time around. It's on every single ballot whether the candidates want it to be or not.
OK, local evidence that yard signs have absolutely nothing to do with outcome. I know, I know, I have criticized the unscientific use of yards signs as a measure of prevailing sentiment. I only bring it up because the signs are very lopsided, for a candidate that has absolutely no shot of winning. I have not seen a single, not one, yard sign for Maura Healey in the upcoming Massachusetts governor's race. I have seen multiple yard signs for Geoff Diehl. He'll get votes but he'll be lucky to break 30%.
The Republican party in Massachusetts is in absolute disarray and I find it very disheartening. Democrats in Massachusetts need a check and Republicans are not going to nominate anyone that can win statewide for a long time unless they change direction. Despite being a deep blue state, the people of Massachusetts have a healthy history of electing fiscally conservative Republicans as governor, but if you want to win statewide here, you have to at least lean left on most social issues. Those types of Republicans used to exist here as recently as Charlie Baker's last campaign, but Governor Baker is the last one left. There are less than 500,000 registered Republicans in Massachusetts, only about 10% of registered voters (~30% registered Dem and ~60% registered Ind) and just under 7% of the whole population. Primary turnout this year was low with only ~275,000 casting votes in the Republican primary.