A Thought Experiment On Polling
Ed Morrisey notes that the latest ABC/WaPo poll appears to have a badly skewed – toward Democrats – sample.
The sampling comprises 33% Democrats, as opposed to only 20% Republicans. That thirteen-point spread is two points larger than their September polling, at 32%/21%. More tellingly, it’s significantly larger than their Election Day sample, which included 35% Democrats to 26% Republicans for a gap of nine points, about a third smaller than the gap in this poll. Of course, that’s when they were more concerned about accuracy over political points of view.
Remember when I wrote that poll watchers need to remember the recent Gallup poll on party affiliation? Gallup polled 5,000 adults and found that the gap between Democrats and Republicans had closed to the smallest margin since 2005, six points, and had been reduced more than half since the beginning of the year. For the WaPo/ABC poll, though, their sample gap has increased almost 50% during that time.
What is missing from the crosstab data (at least I did not see it) is how many responses were discarded to obtain the sample they report.
I think that makes a potentially huge difference in the polling. Think about this. If they had to discard 10 opinions to reach their sample number of 1,004 that tells one story.
If they had to discard 1,000 it tells a completely different story, indeed.
So, what are the raw contact numbers, as opposed to the final results posted by WaPo? What criteria did they use to make the discards? That information is actually vital to deciding what the poll is actually telling you – as opposed to the media interpretation of the results. Think about how that would make you read a poll for a moment. Skewing a sample to match the demographics of a population is quite regular in the world of statistics.
How you get there is also part (an unreported part) of the story.
(This is why I take polls with a grain of salt the size of Detroit, incidentally.)
Blogosphere, Media | Gaius October 20, 2009
By ropelight, October 21, 2009 @ 10:42 am
I’ve been wondering what ever happened to our old friend, Baghdad Bob. I’m glad to know he’s alive and well working for ABC/WaPo. This is a great opportunity for ol’ Bob, truth is he never really was very persuasive on camera. It isn’t his fault, no matter how hard he tries, he’s just unable to project sincerity, but behind the scenes he’s always been reliable, and no one can say he isn’t a team player.
By Tully, October 21, 2009 @ 12:07 pm
As Morrisey points out, the poll SERIOUSLY oveersamples Democrats. ABC/WaPo has consistently produced “outliers” from their poll results, and I see no reason to expect anything accurate from them.