In late November, the sage of the Sagebrush, Nevada, journalist Jon Ralston, uncovered a fascinating post by UNLV political science professor David Damore. The subject was one that has been often discussed both here and elsewhere: Why are pollsters always getting Nevada wrong?
For those with short memories, in 2008, polls showed Barack Obama with a slight lead in Nevada. When the votes were tallied on Election Day, it wasn't even close—Obama wound up winning the state by a 55-43 margin.
The failure of pollsters to nail the Silver State was even more glaring in 2010, when virtually every pollster in existence had Republican Sharron Angle dispatching Democratic Senate Majority Leader Harry Reid. As it happened, Reid was the victor, and it really wasn't terribly close (50-45).
While the polling this year wasn't quite so far off the fairway (the polls only underestimated Obama's margin of victory in Nevada by roughly three points), once again the Democrat looked better on Election Day in the state than he did in pre-election polling.
Which begs the question that Damore sought to answer: What is it about Nevada?
Some of the expected perils were cited by Damore (a high proportion of Latino voters, as well as a higher-than-normal number of cell phone-only households). But Nevada also had a few other interesting eccentricities that would make it tough to poll, according to Damore:
- The state’s largest source of employment, the hospitality industry, necessitates that many Nevadans work and live non-traditional schedules.
- During the past two decades, Nevada has been the fastest growing state in the country and the state has some of the highest rates of annual in and out migration as just 24.3 percent of residents were born in the state; the lowest share in the country.
- Nevada’s voter registration is highly variable. Between March and October statewide voter registration increased by 21 percent and in Clark County, home to nearly three quarters of the state’s population, registration increased by 26 percent.
Even though the effect wasn't quite as magnified in 2012, it was still present. In the nearly two dozen polls conducted after Oct. 1, President Obama led by an average of 3.6 percentage points. At last check, the margin was 6.7 percent. A similar overstatement of Republican performance was present in the state's Senate race: The polls had Republican incumbent Dean Heller up quite a bit more than the final margin, which landed at just a tick over one percent.
However, Nevada is not the only state where this phenomenon was present. And, the multiple examples of states that seem to defy pre-election polling raises the question as to whether a "house effect" exists in states.
(Continue reading below the fold.)
For those unfamiliar with the term "house effect," it is a term that has been utilized by political analysts to describe a generic lean that certain pollsters seem to have in their results. The most obvious example of this, of course, in the current polling landscape is Rasmussen, whose polls frequently tend to err on the side of the Republican candidate, and often by several points.
The question to ponder today is whether or not, due to factors like the ones outlined above with Nevada, some states just tend to perform more amenably for one party or another over the pre-election polling in those states.
To test the theory, I looked at all the presidential polls released on or after Oct. 1. All told, that added up to 536 statewide polls in a total of 42 states.
(For those scoring at home, the following locales were never polled, at least for the Obama-Romney trial heat, in the final month of the campaign: Alabama, Alaska, Delaware, Kentucky, Mississippi, South Carolina, Vermont, West Virginia and Wyoming.)
Much like the national polling, an analysis of statewide polling showed a tendency for the polls in the final month to underestimate the performance of President Obama versus his Republican challenger. Obama outperformed his final month polls in 28 different states, while he underperformed his polling in 14 states.
What is somewhat intriguing is the fact that at least some of the states where Obama beat his polling were states were Democrats also outperformed their polling in 2010.
Here, for posterity, were Obama's "best five" states, as measured by the number of percentage points by which he outperformed his final month polling:
1. Hawaii: 13.2 percentage points (Polls: D+29.5; Election: D+42.7)Hawaii is a state that flummoxed pollsters in 2010, as well. It was, lest we forget, the state where team Rasmussen had one of the most eye-popping examples of clearly crapping the polling bed. For those who have forgotten, in that year, Rasmussen posited that the legendary (and recently departed) Democratic senator there, Dan Inouye, only held a 13-point lead over his little-known Republican challenger. He wound up winning by ... over 50 points.
2. California: 6.9 percentage points (Polls: D+16.4; Election: D+23.1)
3. New Jersey: 6.4 percentage points (Polls: D+11.3; Election: D+17.7)
4. Connecticut: 5.9 percentage points (Polls: D+11.5; Election: D+17.4)
4. Oregon: 5.9 percentage points (Polls: D+6.2; Election: D+12.1)
Why is Hawaii so hard to poll? Ethnically, it is one of the most diverse states in the union, with a high Asian-American population. That is a constituency that has gone sharply to the left in recent elections, and some argue that it is a constituency that has proven more reluctant to respond to polls. If undercounted in polls, that could easily explain some of the swing here.
California, like Hawaii, was a state where Democrats outran their polling margins in 2008, as well. While polls prior to the election did forecast a pair of victories for Democrats Jerry Brown (governor) and Barbara Boxer (U.S. Senate), virtually none foresaw the double-digit victories achieved by both Democratic candidates. This year, as well, polls undersold the level of Democratic dominance in the state.
What happened here? One guess is that California is a state with both a fast-growing Latino population and a fast-growing Asian-American population. If pollsters did not account for that, that could explain being off the fairway by several points. This might've been particularly true with the Latino segment of the electorate. The "gold standard" poll in California, the Field Poll, forecasted a sizable decrease for Barack Obama in the state's "Inland Empire", which includes Riverside and San Bernardino Counties. Those areas have large pockets of Latino voters, and they defied the polls on Election Day by turning out for the Democrats. Not only did Obama carry both Riverside and San Bernardino Counties (by margins almost identical to his 2008 landslide win), but the Democratic wave also took down veteran GOP Rep. Mary Bono Mack, whose Riverside County district fell to Democrat Raul Ruiz.
With New Jersey and Connecticut, meanwhile, we may well see the legitimate evidence of a "Sandy" bump. There is little evidence of a big movement in national polling post-Sandy, but there seems to have been some movement in these two states that were sharply impacted by the super-storm. That "bump" was felt across the ballot, pushing Democratic Senator-Elect Chris Murphy to a wider-than-expected win over Republican Linda McMahon in Connecticut, and saving one imperiled House seat for the Democrats in the Nutmeg State.
New York also saw Obama perform slightly better than he polled, but not to the same magnitude as its neighbors. Of course, inexplicably, the results in the Empire State are still not final, and Cook Report elections analyst David Wasserman has estimated that nearly a half million votes in New York remain to be counted. So, perhaps, in the final analysis New York will join New Jersey and Connecticut in rewarding Obama for his attentiveness in the wake of disaster.
The one that's tough to figure in that top 5 list for the Democrats is Oregon. Oregon was (obviously) not impacted by Sandy, nor does it have an outsized number of nonwhite voters: Oregon's electorate, according to the 2012 exit polls, was 88 percent white in 2012.
Perhaps the pollsters in Oregon anticipated a sharper drop in voter turnout than actually took place, skewing the numbers a bit. After all, Oregon's election turnout of 82.8 percent was only a couple of points lower than the 2008 elections.
Finally, what about the states where Obama's electoral performance was weaker than polling might've indicated? Explaining those results, quite frankly, is a little bit tougher:
1. Oklahoma: 8.6 percentage points (Polls: R+25.0; Election: R+33.6)In the case of Oklahoma, I think it is safe to say the issue there was sample size—the state was only polled once. With South Dakota and Tennessee, there were also small sample sizes (three polls). What's more, in both cases, the issue might've been the "house effect" of a polls. All three South Dakota polls were conducted by a single pollster (Nielson Brothers) and two of the three polls in the Volunteer State were conducted by the same pollster (YouGov).
2. South Dakota: 7.7 percentage points (Polls: R+10.3; Election: R +18.0)
3. Nebraska: 6.2 percentage points (Polls: R +15.6; Election: R +21.8)
4. Tennessee: 5.4 percentage points (Polls: R +15.0; Election: R +20.4)
5. Montana: 5.0 percentage points (Polls: R +8.7; Election: R +13.7)
Nebraska was just a pure miss, across the board. The error extended to the Senate race, which despite the late hype wound up being an easy double-digit win for Republican Deb Fischer over Democrat Bob Kerrey.
One final, and somewhat interesting, finding: The most consistent finding in the study was the fact that margins of victory, no matter who won the state, were almost always underestimated by the pollsters in 2012.
Over the 14 states that had Barack Obama polling better than he actually performed, Obama lost all but one of them (Illinois). Meanwhile, in the states that Obama overperformed his polling, he carried 24 out of 28 of them. All told, an eye-popping 88 percent of the races had the winner polling worse than his actual final margin of victory.
And, in a shoutout to the polling community, the Oct-Nov polling average correctly picked the winner in 41 out of 42 states: Only Florida (where the final month average went to Romney by a fraction of a percent) kept America's pollsters from running the table. But, once again, this makes a ridiculously strong case for the virtues of averaging polls to forecast electoral outcomes.