« You Down With DDT? Oh No, Not Me | Main | Rigging Election for Fun and Profit »
September 23, 2007
Polling
In my review of Mark Penn's new book Microtrends, I got a bit of flack from pollsters who otherwise loved the piece because I widened the conclusion from Penn to polling. I wrote:
Pollsters occupy a uniquely powerful space in American political discourse: They bring science to elections. Armed with heaps of raw data, they elevate their opinions into something altogether weightier: Conclusions. When an organization sends out a press release saying the organization is right, it’s ignored. When a pollster sends out a poll showing the electorate agrees, ears in Washington perk up.
The enterprise has always been dodgy. Populist pollsters reliably discover that the electorate thirsts for more populism. Conservative pollsters routinely discover a small government consensus pulsing at the heart of the body politic. When the libertarian Cato Institute commissioned a poll of the electorate, they found—shockingly—that the essential swing vote was made of libertarians. Remarkably, whenever a politician or self-interested institution releases a poll, the results show a symmetry between the attitudes of the pollster’s employer and those of the voters. But Penn’s book shines light on this phenomenon: If he is the pinnacle of his profession, then the profession uses numbers as a ruse—a superficial empiricism that obscures garden-variety hackery. And that’s a trend worth worrying about.
Lots of pollsters protested that, sure, Mark Penn weights his numbers and doesn't show his work and has a bit of a bad reputation, but he's an isolated case. But take Celinda Lake's push-polling. Celinda Lake is not, so far as I know, a pollster with a bad reputation. Rather the opposite, in fact. Yet she's pushing out a "sobering" poll that shows support for Democratic congressional candidates drops after respondents are told, "Some people say [your Democratic incumbent] is a strong supporter of Hillary Clinton/Barack Obama and will support her/his liberal agenda of big government and higher taxes if she becomes president." And this poll is being reported a problem for those candidates.
But that's moronic. If the Democrat is defined as being a candidate of high taxes and an amorphous, evil-sounding "liberal agenda," they will do poorly. If they're defined in more positive ways, they'll do better. If the question was, "Some people say [your Democratic incumbent] is a strong supporter of Hillary Clinton/Barack Obama and will support her/his attempts to withdraw troops from Iraq and ensure every American has health care if he/she becomes president," support would increase.
This poll wanted a result. It got it. It also could have gotten the opposite result. This happens all the time. It just depends on who's paying, and what they want to show. It's certainly true that good polling can be and often is, conducted, but far too much of it is of this type, and nether the polling industry nor the media polices these practices.
September 23, 2007 | Permalink
Comments
WTF? Why is Celinda Lake running polls like that? She's supposed to be on our team, right?
Posted by: Neil the Ethical Werewolf | Sep 23, 2007 2:56:40 PM
If supply-siderism is voodoo economics, polls done by interest-pushing firms are the comparable voodooism.
I trust a few, very few of the polls: Pew being one I take seriously.
Much of the polling is pseudo-scientism that can't muster any claim of being either unbiased or 'scientic'. The pollsters and the people who pay for them have discovered what the Rovians have mastered: counter any unfavorable news (to your side) with as much lather and lies as is necessary to totally confuse the public and pundits. Look what they have accomplished on global warming, health care, and Iraq. Six pounds of truth is the same as a half-dozen pounds of bullshit.
Posted by: JimPortlandOR | Sep 23, 2007 3:09:32 PM
I've been opposed to a lot of polling for some time now; I wish people would question the logic of polls more, challenge the statistical work, and look askance at nearly every number that's supposed to tell us what we (or "the People") think. Even something as simple as "Do you approve of the way the President is doing his job" is in its way a simplistic, meaningless question that says little about how individuals view the President in totality. The problem, I think, and it's serious, is that polling plays into America's general hands-off approach to math and science, to take "science reporting" at face value and not understand the underlying work. And I don't know how you fight that - no one gets very far calling the American public stupid, and saying "you don't really understand polling" will, in many ways be meaningless.
I don't know whose side Celinda Lake is supposed to be on; seeing her interviewed, I've never seen her as being on my side, that much I know. I don't know why people call her onto programs and treat her as if her expertise is beyond challenge (I also don't know why people who need to question her don't familiarize themselves more closely with her approach and her methodology and how she actually works with data, but that, of course is asking for something totally unrealistic). But let's remember, we've had two conversations on this blog going just yesterday... about poll numbers, and how they tell us, conclusively, what to expect in the upcoming election. Saying that "other people take polling too seriously" only works if you, well, don't. And from what I've seen, almost everyone thinks that polling - at least polling that confirms what they thought anyway - is to be taken seriously and presented as firm fact, and that's simply adding to the problem. And that can't change until we insist on changing it, and stop playing into the game by choosing the polls we like. It's the whole industry around polling that needs to be questioned, and hard, before we go on, I'm convinced of it. But my hope for seeing that happen is not very high at all.
Posted by: weboy | Sep 23, 2007 3:14:50 PM
I neglected to mention above that Ezra, although he makes Penn look like a fool, was far too easy on him.
And, Hillary has done nothing to indicate that Penn isn't taking over her campaign - as the well-placed leaks are saying. She can't be unaware of what is being said about Penn and her campaign, so I guess it is fair to conclude she likes the fit between them. That scares the crap out of me, far more than weasel words and triangulation.
Posted by: JimPortlandOR | Sep 23, 2007 3:23:36 PM
Neil:
Matt says she's working for the Biden campaign, which would be why she's saying this. http://matthewyglesias.theatlantic.com/archives/2007/09/consultant_power.php
Posted by: Anthony Damiani | Sep 23, 2007 4:30:37 PM
Playing devils advocate- here's the thing. What they list are actually stereotypes of Democrats that are out there. Even if the poll is crap, I don't think the general perception is.
Posted by: akaison | Sep 23, 2007 5:27:32 PM
I second akaison. Remember how John Kerry looked like a strong candidate until he was Swift Boated. If the Republicans can pull off the same tactic on Hillary Clinton or Barack Obama, then there will be another Republican in the White House for another term.
Posted by: stm177 | Sep 23, 2007 6:03:14 PM
Yes, but the poll doesn't test that. It just gives a hypothetical -- if these negative premises could be argued (and they could be argued against any Democrat), than what would you think?
Posted by: Ezra | Sep 23, 2007 6:06:56 PM
Ezra's right - if this was testing Clinton or Obama's negatives the question would have been about them and not about a fictitious "liberal agenda of big government and higher taxes". Hell, given the question as phrased my knee jerk would be to respond with a negative, despite my plans to vote for whoever gets the Dem nomination and also knowing about the disgusting lizard-brain psychology of push polls.
This poll is crap, the question is loaded, and if Biden actually paid someone to run this the Congress critters in the "vulnerable" districts it was run in should string him up and apply the tar and the feathers. They're going to have to fight stuff like this coming from Republican opponents, they don't need this crap coming from Biden too.
Posted by: NonyNony | Sep 23, 2007 6:18:44 PM
This is where your wonkishness gets in the way. It doesn't matter if it's true. It matters if enough voters are so easily persuaded despite all evidence to the contrary. In other words, it's a lense into perception. It's like the 30 percent of the people who believe that Saddam was behind 9/11. Truth isn't a factor in my point. Perception is.
And you do at the end get to the point. Could they be argued against any Democrat? I would like to know that too. It would have been more interesting to see if such polling would have worked with Biden or Edwards or Richardson? This poll may have done us a favor by openning up precisely the question that Democrats are afraid to ask because they know they may not like the answer. It's precisely what the GOP will do. I would prefer to know that now rather than engage in the hubris I've seen over the last few months coming out of the blogs.
Posted by: akaison | Sep 23, 2007 6:28:12 PM
And people can say as much as they want, Nony, this poll is crap because it's biased. but the fact is so is perpetion- that's the point. None of the voters comes to an election tableau rasa despite what you or other wonks think.
Posted by: akaison | Sep 23, 2007 6:29:10 PM
Okay- thought of one other point. This is also like other polls I've seen a clear test as well about how much people are really paying attention to what the candidates are saying. Many blogs have been saying that due to polling people are paying a high level of attention, and yet, here polling pushing them in a direction that is false easily persuades them. What does that tell us about the vaulted ability of the campaign machines running to blunt any GOP hanky-panky or what does it tell us about the amount of attention really being paid to candidates right now?
Posted by: akaison | Sep 23, 2007 6:34:58 PM
No, but the question is whether voters can be convinced of this. The poll didn't test that. It just presupposed that the voters already believed this about them, and asked what their reaction in such a world would be.
Posted by: Ezra | Sep 23, 2007 6:37:38 PM
I see your point, but I also have to think that if voters didn't believe that to be true, they wouldn't have voted the way they did in the poll? I keep returning to the 30 percent that still believes Saddam was behind 9/11. That alone tells me that people believe what they want to believe, and there is no amount of campaigning that will change that. The only real question to me is degree and not if.
Posted by: akaison | Sep 23, 2007 6:55:13 PM
I'm a DC pollster. Penn/Schoen is probably the worst polling firm in the city. Lake is NOT a trained researcher. She is a woman with an In-Focus machine and plenty of connections. For that matter, I would estimate that 80% of pollsters (both Dem and Repub) are without any formal training in survey research or statistics. This is not to say that they aren't right some of the time. But it is to say that have precious little understanding of the data they generate.
Posted by: Jeff | Sep 23, 2007 9:31:09 PM
Do you think we can learn anything from the polling data here?
Posted by: akaison | Sep 23, 2007 10:25:00 PM
I keep returning to the 30 percent that still believes Saddam was behind 9/11. That alone tells me that people believe what they want to believe, and there is no amount of campaigning that will change that. The only real question to me is degree and not if.Yes, and if there were 60 percent of such people instead of 30 percent, democracy would be a utopian pipe dream and we might as well start looking for a philosopher-king.
But there aren't. The ability of democracy to function depends on the fact that only a minority are such complete sheep that they will follow anyone who promises to protect them and show them who to attack.
Posted by: Chris | Sep 23, 2007 10:25:01 PM
uhm we are the middle of a war thats gone on for 5 years and will last probably 8 years at mimimum because at one point most people did believe that Saddam was behind 9/11. if you think that's a sign of things "working" you and I have different definitions of the word.
Posted by: akaison | Sep 23, 2007 10:30:55 PM
by the way- I think its quite sad that you would make that argument because its quite clearly an attempt to one up rather than get the main point- namely how people can be influenced. it only takes a small percentage of the persuadable voters to shift elections.
Posted by: akaison | Sep 23, 2007 10:32:08 PM
While I am distrustful of the questions asked by certain polls, I think that sometimes polls are useful. A case in point is the polling that has been done in Iraq. In a video debate (linked to by Tapped) sponsored by the Carnegie Institute, Reul Gerecht dismissed the poll results that have shown for years - in clearly identifiable trend lines - that the majority of Iraqis want the U.S. to fix a timetable for withdrawal and withdraw by, a., dismissing polls, and b., substituting his opinion for what the majority of Iraqis think. It is quite an easy way to argue: "most people think like me, in spite of polls claiming the contrary, because I distrust polls." This cavalier way of treating facts is common to the pro-war side - especially as the polls in Iraq, and the election results, made nonsense out of everything the pro-war side claimed about Iraqis. Trends over several polls should at least operate as a constraint on what one claims is the popular viewpoint.
Posted by: roger | Sep 24, 2007 11:16:12 AM
Just a minor thing, but what you call a causal/correlational error by Penn is actually two errors of a slightly different nature.
First, its a bad descriptive inference (taking a handful of examples and extrapolating to a wider population). What is the confidence interval around this inference Penn is making about lefties? To know that we'd have to know something about the population, and something about the proportion of gifted individuals that are left-handed in that population who achieve remarkable things.
Second, there is a causal inference (which I gather is what you were referring to). This is the suggestion that A causes B (being left-haded causes remarkable success). Ever since Hume philosophers of science have been trying to figure out how to make those inferences accurately. Penn is obviously completely unaware of this difficulty.
Posted by: andy | Sep 24, 2007 11:47:49 AM
High quality pills.
buy valtrex
buy tetracycline
buy prozac
buy ativan
buy lorazepam
Posted by: andru | Nov 30, 2007 7:19:44 AM
Free prescription.
alprazolam pharmacology
difference between clonazepam and alprazolam
paxil withdrawal symptoms
cheap oxycontin
buy klonopin
phentermine 37.5mg
buy tenuate
Posted by: asdas | Dec 1, 2007 3:56:47 AM
Special Offers for the Mens Health Medications
wellbutrin sr
buy zolpidem
buy prozac
buy oxycontin
tylenol with codeine
Posted by: doca | Dec 1, 2007 5:57:41 AM
Low price on brand pills.
buy cialis
buy cialis online
buy cialis soft tabs
buy corisoprodol
buy diazepam
buy female viagra
buy lexapro
buy phentermine
buy renova
buy retin a
Posted by: dasdas | Dec 2, 2007 1:26:53 AM
The comments to this entry are closed.
