Ezra Klein: The Myth of Nonpartisan Research

« EITC vs. The Minimum Wage | Main | You Don't Know The Shape I'm In »

February 28, 2007

The Myth of Nonpartisan Research

I want to direct you to EPI's Jared Bernstein in the comments of the think tank discussion. I noticed a lot of folks casually claiming that EPI is a "tool" of the unions or deriding it as some sort of biased -- and thus untrustworthy -- organization. There's no doubt that EPI has a slant. But they are meticulous, serious, empirical, and careful. Jared is right when he says:

The commenters above who disparage our work are wrong. In the last 15 years, I've co-written eight versions of State of Working American, each one about 500 pages of tables, figures and analysis. My colleagues and I have written tons of other books and papers, all a mouseclick away. It's a friggin' huge paper trail.

DRR et al, show me one table, one figure in all that output, that's 'loopy' or wrong or cooked in such a way to carry water for some group. One table or figure, out of literally 1000s!!! If you can't find one that fits that description, find one wherein you even disagree with the methods, one where you think we crunched the numbers in a less than rigorous way, or tilted the data in a biased manner to make our case.

And this goes beyond EPI. It's true for folks at Cato, at CEPR, at New America, and even at AEI. The media has spent so long delegitimizing anyone with an opinion and stickering every nonpartisan or bipartisan endeavor with big labels reading "TRUTH" that we've begun to believe them. That's how you get odd spectacles like in the comments section, where various outlets are simply dismissed, their research understood, a priori, to be unmeritorious and tainted by ideology.

Good research is good research, and the impulse for caution and establishment acceptance and broad political appeal is as pernicious and distorting as any opinion -- all the more so because it's layered atop whatever beliefs already exist. At least with EPI and others, the assumptions and premises underlying all economic analysis are rendered transparent. Too often, they remain opaque, even as they color the conclusions. If you've a serious methodological case against an institution, that's worth exploring and taking seriously. But simply buying into the bullshit glorification of nonpartisanship is indefensible.

February 28, 2007 | Permalink

Comments

More like this please.

You are entitled to your opinion, not your facts.

And, with tables and charts and stats, the only serious way to approach the situation is to be transparent about assumptions, shortcomings, interpolations, etc. Those that don't provide the backdrop should cause serious suspicion.

Posted by: abjectfunk | Feb 27, 2007 11:44:11 PM

I noticed a lot of folks casually claiming that EPI is a "tool" of the unions or deriding it as some sort of biased -- and thus untrustworthy -- organization.

Where was this? Not in the thread you linked to, where exactly one person said that EPI is "a propaganda arm of the AFL-CIO," and exactly no one derided it as untrustworthy. No one called EPI "loopy" either. "Anonymous" said the mainstream media thinks EPI is loopy (which isn't true), but he clearly didn't agree.

The media has spent so long delegitimizing anyone with an opinion and stickering every nonpartisan or bipartisan endeavor with big labels reading "TRUTH" that we've begun to believe them.

Who's "we"? I don't know anyone who believes this, and I don't hear it in the bogeyman media.

But simply buying into the bullshit glorification of nonpartisanship is indefensible.

Well, yeah, but no one here is doing that, and I don't think it's done so much elsewhere either.

Posted by: Sanpete | Feb 27, 2007 11:49:27 PM

The media has spent so long delegitimizing anyone with an opinion

Don't blame the media: blame the proliferation of corner-office wingnut welfare fronts with 501(c)(3) status that exist primarily to provide fuckheads like Clifford May, Cakewalk Ken Adelman, Brent Bozell and William Donohue with an official-sounding name to hide behind.

Posted by: pseudonymous in nc | Feb 28, 2007 1:13:34 AM

The media has spent so long delegitimizing anyone with an opinion and stickering every nonpartisan or bipartisan endeavor with big labels reading "TRUTH" that we've begun to believe them.

I know it's attractive to project the diametric opposite of your own view onto the mainstream media. However, that's not a very good methodology for discussing politics.

Social sciences have an outrageously low signal to noise ratio. How couldn't they? They a) attract people who're on average less smart than those who do natural sciences, b) give people a lot more incentive to stick to indefensible conclusions, and c) has a proliferation of methodological minutiae that can get you any result you want. For instance, multiple regression analysis depends on the variables you enter so much that the same data can be shown to lead to wildly different conclusions.

With so much chaff and so little wheat, people have to develop mechanisms of throwing out the junk. One good mechanism is the rule of thumb that if a thinktank disagrees with an academic study, the thinktank is always right; to take a slight refinement, academics whose chairs are funded by ideologically motivated thinktanks, like John Lott, are barely more reliable than thinktanks.

Right now, the media gives equal credence to peer-reviewed studies and to thinktank hatchet jobs. Even if it does what you imply it should, it won't improve matters much. Laypersons won't be any more informed, and experts won't learn anything new.

A single news article can only offer a cursory analysis of scientific debates, and is hardly more informative than, say, direct-to-consumer drug advertising. It's impossible to write a 1,000-word New York Times article doing justice to, say, debates within biological community about evolutionary developmental biology. And that's an issue where it's relatively easy to produce rigorous research; when you venture into economics, things get exponentially harder.

It's possible for a newspaper to publish a debate between Lewontin and Dawkins, or even to commission both to write regular features about evolution. But what's the point? Professional biologists, who know all the relevant facts and have been familiar with those debates for decades have already decided. There's no point in a rematch in a far less professionally competent arena.

The same principle applies to social sciences. When a thinktank publishes a research, the ideal media's reaction should be to ignore it. There's nothing that privileges thinktank fellows over real economists enough to exempt them from peer review. Likewise, there's nothing that privileges ordinary people's views. Serious scholars may not be able to describe poverty or unemployment in lurid detail, but frankly the media could use fewer heartwrenching stories and more facts.

The layperson doesn't need to know anything more than what mainstream expert opinion is, and to what extent expert opinion can be trusted (more so on climatology and evolutionary biology than on welfare economics and political science). If he cares enough to delve into the subject, that's what books and professional reviews are for. The mainstream media can never be a substitute for scholarly books and articles; even semi-popular magazines like Scientific American are inferior to actually reading what current research is.

Posted by: Alon Levy | Feb 28, 2007 4:03:29 AM

'Social sciences have an outrageously low signal to noise ratio. How couldn't they? They a) attract people who're on average less smart than those who do natural sciences'

Exhibit 1 In the case against the obnoxious grad student.
Someone call MIT to see if their physics department is missing its socially awkward adolescent savant.

Posted by: Dr. Levy, failed academic | Feb 28, 2007 5:07:27 AM

They a) attract people who're on average less smart than those who do natural sciences

Alon Levy is Larry Summers... who knew?

Posted by: DivGuy | Feb 28, 2007 7:31:51 AM

Oh, and let's not forget Cindy Sheehan's Gold Star Families for Peace where she claims this status. Of course, she would never attempt to get involved politically (/sarcasm).

https://secure.democracyinaction.org/dia/organizations/gsfp/shop/custom.jsp?donate_page_KEY=864

Posted by: Fred Jones | Feb 28, 2007 8:09:44 AM

AEI "meticulous, serious, empirical, and careful"? Puh-leeze! They're the fine folk who kept John Lott on their payroll until things became way too hot. Their stuff was good in the 1980's, but they have since become a Heritage-style whorehouse. I'll read Cato material, but I won't waste my time with AEI stuff unless the author's name rhymes with "Ornstein."

Posted by: Joe S. | Feb 28, 2007 8:28:00 AM

Sanpete,

You've got a point that I am a bit indiscriminate in my rant, but this debate goes beyond the few negative comments over the last day or so.

Given that there actually may be an opening for some progressive policy ideas to have a bit of traction, our credibility--earned by decades of careful research--is especially important to us right now.

We've literally been talking, writing, and crunching the numbers on inequality, globalization, health care, pensions, min wgs, unions, for longer than anyone else, and these are precisely the issues that have some currency in today's debate. Maybe, if we (you, me, Ezra, anyone else interested in economic justice) play our cards right, we can make some good stuff happen.

We're buffed and we're ready. Lock and load those calculators, open up the spreadsheets and bring it on.

Posted by: Jared Bernstein | Feb 28, 2007 10:26:50 AM

I agree with Joe S. about AEI. At one time they might have done good research, but that time was long ago. Now, with the exception of Norm Ornstein, it's just a bunch of hacks.

I'm becoming convinced that we should take some of the billions being dumped into Iraq, invent a time machine, and send someone back in time to about 1999. That person would take Norm Ornstein aside and then level AEI and kill everyone else there.

I'm joking, of course, but when one considers the amount of blood, treasure, and American prestige those intellectual whores have squandered, the time-machine plan begins to make sense.

Posted by: Kenneth Fair | Feb 28, 2007 10:48:44 AM

Read my language again -- I specified this was true for "folks" at AEI rather than the institution. They've got a lot of hacks there, and also some serious policy scholars.

Posted by: Ezra | Feb 28, 2007 11:13:40 AM

Alon Levy:

"With so much chaff and so little wheat, people have to develop mechanisms of throwing out the junk. One good mechanism is the rule of thumb that if a thinktank disagrees with an academic study, the thinktank is always right; to take a slight refinement, academics whose chairs are funded by ideologically motivated thinktanks, like John Lott, are barely more reliable than thinktanks.

Let me give you a hint - when paragaphs contradict each other, you should try to separate them a bit. When paragaph N+1 works against paragaph N, it's a bit too obvious.

Posted by: Barry | Feb 28, 2007 12:15:36 PM

AEI is a very mixed bag, quality-wise. Kevin Hassett is very smart; Marvin Kosters, John Makin, & Robert Hahn are always worth a hearing. Herbert Stein was a saint (albeit unique). Then on the other hand you have some very, very bad people.

Cato has Bill Niskanen, a brilliant guy. Chris Edwards is a smart fellow. They do good stuff on foreign policy, defense spending, and the drug war.

Heritage . . . let me think . . . hmmm . . .

Posted by: Miracle Max | Feb 28, 2007 3:11:26 PM

Funny thing is, 'Fred Jones' has a 501(c)(3) with a staff of twenty, all of whom are actually pseudonyms he uses as blog troll, stalker and sociopath.

Posted by: pseudonymous in nc | Feb 28, 2007 4:24:11 PM

Miracle Max, Kevin Hassett authored a book that you might have heard of - 'Dow 36,000'.

Posted by: Barry | Feb 28, 2007 5:43:42 PM

Let me give you a hint - when paragaphs contradict each other, you should try to separate them a bit. When paragaph N+1 works against paragaph N, it's a bit too obvious.

Yeah, that was my fault for writing "the thinktank is always right" when I meant "the thinktank is always wrong."

Instead of writing "teh," "nad," "thinknig," and "Erza," I reverse the intended meaning of my sentences with my typos.

Posted by: Alon Levy | Feb 28, 2007 8:08:57 PM

I know you were alluding to more than just me when you were talking about being accused of being a "tool" of the AFL-CIO. If you thought I was accusing you of such than my apologies.

This is not about appeasing the god of nonpartianship, which doesn't even tell you anything anyway. EPI is a "nonpartisan" organization. Nor am I concerned about "political" consensus. It's, for me at least, about how much you can trust the research involved.

It's easy to say "throw caution to the wind" or "The advocacy doesen't matter, it's the research" which I agree with in principle,and in a perfectly informed world that would the case. The reality is a lot of this research is of a fairly specialized nature, carried out by specialized professionals with advanced degrees (usually Econ PhDs). The result is, the number of people in the general public who have the training, expertise or knowledge (or leisure & predisposition for that matter) to thoroughly check this research and make sure all the T's are crossed and I's are dotted is quite small. And in the last few years, it's become clear that despite their best intentions, professional political writers & even Business/Econ beat writers for daily publications really aren't up to the task.

In this scenario, if you're a person who's interested in the debate but without the neccessary expertise to check "Kosher" or "Foul" yourself, then the only thing you have to go by is which people/positions/institutions you can trust.

My take on this is largely influenced by my experiences Daniel Davies, who deals a lot with economics & economic commentary. The general gist of which is, most "Think Tanks" have a depressingly low serious scholar to hack ratio (with Cato apparantly being the worst), any halfway clever economist or econometrician, sufficiently motivated, can put forth a paper arguing almost anything, and unless you are absolutely sure of the good faith & rigorousness of the researchers involved, take any claims they make with a good measure of caution & skepticism(if you don't have the expertise to check yourself).

This is getting way too long but to wrap up; I have a cultivated number of economists, of who'm through constant reading and recommendation by others I admire, I've come to regard as trustworthy, as in if they something, I know I can trust it, and if it contradicts my expectations I know they arrived at it in good faith. Guys like Barkley Rosser, Mark Thoma, Galbraith Jr., Kash Mansouri, Brad Setser and Daniel Davies. I'm sure that most of EPI's work is sound, but it's position on one issue in particular gives me pause. Not just because it's way out of step with accepted wisdom of similiar professionals, or out of step with these particular scholars who I trust's opinions, but because, in these people's cases, I know that their position on this issue wasn't arrived at due to some kind of idealogy or absolutism.

Posted by: DRR | Feb 28, 2007 8:29:54 PM

托盘
 托盘
 钢托盘
 钢制托盘
 塑料托盘
 木托盘
 木制托盘
 纸托盘
 木塑托盘