Saturday, October 24, 2009

Exit polling underrepresents Asians as well as whites?

Steve Sailer has repeatedly pointed out that election-time exit polling consistently overstates the Hispanic share of the vote. Just as consistently, this tendency putatively provides evidence of the rapidly increasing importance of courting Hispanic voters. Inevitably, the more thorough and reliable Census phone survey of some 50,000 people released months later downwardly adjusts Hispanic representation among the electorate. In consequence, while the media fire of election night coverage is blazing, whites are short shrifted. Only later, long after the embers have cooled, is the record set straight.

Because pollsters must physically be present at polling places, the organizations conducting the polls are only able to cover a small fraction of total voting locations. Among large demographic groups like whites or women, there is no concern that too few members will be interviewed. But among smaller groups like Asians and Hispanics, it's a concern. And since these groups are growing in size (proportionally as well as absolutely), overrepresenting them doesn't carry the risk of embarrassment (ie, "Only 1% of voters were Asian? Obviously your methodology is poor.") that underrepresenting them does.

At least that was how I conceptualized it. However, the Census surveys show that exit polling underrepresents Asians as well as whites. Using state level exit polling data from the 2004 and 2008 Presidential elections to get precise numbers*, I calculated electoral representation by race as reported by Edison Media Research and used by all the major media sources during and directly after each election cycle. Steve has provided a handy table containing data for the same groups from the Census surveys from these years. The following table compares them and shows to what extent exit polling reported on by media sources during the election cycle inflated the size of the non-white vote at the expense of white voters:

Race04 exit poll04 CensusIn(de)flation08 exit poll08 CensusIn(de)flation

The table shows changes in terms of absolute representation among the total electorate. Among Asians, who comprise something approximating 1 in 50 voters, the relative variance between the two methods is substantial. The Census surveys suggest that exit polling underrepresents Asians by more than 20% (and overrepresents "others" by a similar factor). For whites, by comparison, although white underrepresentation in exit polling appears to exist, it amounts to a 3% or so reduction in participation.

The methodology behind the Census surveys is explained in this pdf (see chapter 7, 56 pages in) and it appears that addresses are confirmed before phone interviewing begins. Thus the lack of an old landline phone does not lead to those only possessing a cell phone (like myself) being overlooked.

It is not surprising that whites are short-shrifted, but I am at a loss to explain the Asian variance among exit polling and Census surveys. Any ideas? State-level exit polling data from Presidential elections are apparently only available for the two most recent elections, so the presumed underrepresentation of Asians and overrepresentation of NAMs in exit polling might just be a fluke. Whatever the explanation, it's worth being aware of this when considering exit polling data.

* The exit polls at both the national and state levels report figures in whole percentages. At the state level, a 7.54% reported as 8% here should presumably be balanced out by a 12.46% reported as 12% there while still allowing percentages to be broken out into tenths of a percent when amalgamated to constitute national totals. Further, participation in state-level polls totals around 50,000 people, compared to 17,000 participants at the national level. Keep in mind, too, that national-level polling is even less accurate than state-level polling is when it comes to accurately determining the demographic characteristics of the electorate.


Steve Sailer said...

Asian exit polling numbers are frustratingly incomplete because for 2008 you only got three numbers:


For every other state, the exit poll company says it has too small of a sample size.

It would have been very helpful if they had oversampled Asians in, say, Texas and New York enough to have a statistically significant number so we could have gotten some sense of Asian voting in a red state and in an East Coast blue state.

I suspect the exit polling organization doesn't have a method for not using over-sampled minority voters. They seem to have a Use It or Lose It approach where they oversample Hispanics because there is a lot of interest in Hispanic voting trends at the state level, but then that leads them to include too many Hispanics in the overall figures. Meanwhile, they don't oversample Asians because there isn't much demand for Asian trends, so it's hard to tell what's going on with Asians.

This isn't a technically difficult problem to solve. Heck, they solve problems like coming up with the wrong winner in 2004 (President John F. Kerry) by just "adjusting" all the numbers.

The basic problem is that the exit polling business is a monopoly so quality isn't too much of a priority.

Audacious Epigone said...


I know you've written about how Hispanics, far from being swing voters, tend to vote similarly to whites but skewed 15 points or so to the left. Have you looked at whether or not this trend holds among Hispanics at the state level? Are Texas Hispanics considerably more conservative than California's are? As you say, it would also be nice to have this information regarding Asians.