« Why Consumer-Directed Health Care Won't Work | Main | Newt Comes Through »

September 07, 2006

Hmm

That wage decline map many of us posted on a few days ago may be based on shoddy statistical work. Jane Galt has more.

September 7, 2006 | Permalink

Comments

What does the map using correct data show? Most likely, you will still see large area of the country that would show median incomes falling. Incomes fell during this period. Jane has the data. She should post it.

Or an enterprising blogger from the left might do the same.

Thats what I would like to see someone on the left do, just for comparisons sake.

Posted by: mickslam | Sep 7, 2006 3:09:31 PM

Mick, I did post it. You can't use the ACS, because the data doesn't exist, so the correct source is the Census' Current Population Survey, which is linked in the piece. Here's the link again: http://www.census.gov/hhes/www/income/histinc/h08.html

Best,

Jane

Posted by: Jane Galt | Sep 7, 2006 4:21:58 PM

It may be that the "it" Mick was referring to was the map he described that uses Census data.

I'd hope that people would put the changes in annual compound growth rate over the period rather than lump sum changes, which exaggerate.

Even better would be to put it side by side with a similar map from the 1960's.

Posted by: BruceMcF | Sep 7, 2006 4:30:20 PM

I'd love to put up a map, but that's waaaaaaaaaaay beyond my incredibly rudimentary graphical skills. But the data's right there, for anyone who wants to have a go at it.

Posted by: Jane Galt | Sep 7, 2006 4:35:01 PM

I think it's a mistake to dismiss the map as based on incorrect data. Jane Galt's criticism seems based on the fact that the Detroit Free Press compared 2000 Census data with 2005 American Community Survey data, rather than using her preferred CPS data.

The ACS has been designed and tested to be the successor to decennial Census long form data. There may be something funky going on with the income numbers, but not because they were collected by two dissimilar surveys. The Census Bureau has spent years testing survey methodology to ensure comparability. If there is a problem (and there certainly could be, it's a tricky issue, and the numbers do look odd), it won't be nearly as simple as Jane Galt suggests.

The ACS is better than CPS, in my opinion. The ACS sample is much larger, spread across more communities, and is mandatory, rather than voluntary.

I would personally be cautious about using the numbers, because the change is so dramatic, and would want to spend some time investigating the difference. However, it is not justified to dump these numbers in favor of numbers from another source on the basis of bad methodology.

Posted by: CRH | Sep 7, 2006 4:41:04 PM

I'm not enterprising, but I thought Jane was claiming that the trend was real but exaggerrated by the map. Two and three percent drops rather than over 10 percent as the map showed in a few places.

Maybe she should have, but posting such a map might let them keep the implicit message that the MSM/journalists/liberals (fill in boogeyman of your choice here) is dishonest, but would keep people from getting the impression that the economy is actually going well. You'll notice she bolded Stuart Buck's strongest statement about the Detroit Free Press ("That's an increase of 5.4%, as opposed to the 7.2% decrease that the Detroit Free Press claims to have found."), not Stuart Buck's strongest statement about wages nationwide ("Adjusting for inflation, that is a 2.9% decrease, not the 6% decrease found by the Free Press.").

I don't know, maybe I'm being too hard on Jane Galt here. Maybe I'm reading too much into this; someone who reads her or right-wing blogs more often than I could say. She is right that the map is the product of sloppy reporting at best. Maybe she didn't mean to make any more conclusions than that or maybe she even, like me, wasn't enterprising.

But it sure looks like she's more interested in scoring points against all the leftists at the Detroit Free Press than figuring out exactly what the state of the economy is.

Posted by: Cyrus | Sep 7, 2006 4:43:00 PM

I'm bolding what Stuart bolded in his email to me. I'm not much of a bolder, actually; more of an italics gal.

My purpose is not to show that the economy is going well; I don't really have a purpose, except to try to curb the use of crappy statistical method among journalists. And the size of the drops on that map should have given a lot of people pause . . . a 10% fall in median incomes, which they claimed for 9 states, is HUGE. If the economy is really that bad, it should be showing up in numbers like unemployment and consumption.

It's hard to tell exactly, but it looks to me like more than half of the drop reported by the Freep is due to the switch between surveys, rather than a real change in the economy. Using numbers like that without even footnoting the change in data sources is a big no-no in my profession. Or at least it is at my employer, and I would hope elsewhere.

The ACS may well be better than the CPS--I'm sure we could have a rousing argument about it if we could only keep ourselves awake--but it has a big limitation, which is that you can't go back past 2000 with it (or past 2003 for most of the more granular data). That means it's not an appropriate tool for longitudinal comparisons before 2000--and certainly not without doing some adjustments for the change in data sources. That means the CPS should be the preferred tool for such comparisons, no matter how much we all wish the ACS went back farther. When your dataset is inadequate, the answer is to study the comparisons you can make, not to gum together two disparate numbers.

Posted by: Jane Galt | Sep 7, 2006 5:10:43 PM

Fair enough. I wish I could claim it was a bad day, but it was only like an hour ago. Sorry.

Posted by: Cyrus | Sep 7, 2006 5:25:29 PM

Jane-

I was looking for something like the original survey - with a state by state breakdown. You ran the numbers because you hinted in your post about the general dismal actual numbers, but I just wanted to see them.

The map itself is shoddy work, and I'll admit, I didn't actually look at the numbers on it until a few days after it came out and thought, well those simply can't be true. When you pointed it out I wasn't surprised. I just glanced at the colors and thought - yep, income growth is down - no new story here.

The story that particular map tells is largely true, despite the error of the order of magnitude on the map. My critique of your work is right along the lines of Cyrus up there, you frequently address the weakest lines of the argument of critics, not the strongest. A real honest critique could easily have given the alternative numbers, as you had worked them out. But you lead with the most egregious error. Its not dishonest, its just a little lame for someone who pretends to be open minded.

I note that many people on the right are still claiming that this recovery has been excellent for all income groups. You can listen to Larry Kudlow say this every day, if you want. I don't see a post refuting his daily BS - and its not just a question of magnitude. His claims are simply false in this case. And he is on every damn day - not a few day flash in the left blogosphere.


regards,

Slam

Posted by: mickslam | Sep 7, 2006 5:41:05 PM

Slam, I only looked at this because Stuart Buck emailed me with the question. Blogging is by definition idiosyncratic; we write about what happens to come to our attention. If you email me a Kudlow claim that you want me to call bullshit on (and it is in fact bullshit) I'll be happy to look at it.

The reason I didn't do a state-by-state breakdown is that getting data out of a text file into excel is an enormous pain in the ass, and I'm on vacation. Which, incidentally, is undoubtedly one of the reasons none of the people who linked to the original map checked the Freep's work.

Posted by: Jane Galt | Sep 7, 2006 6:58:01 PM

My main point was that the ACS is the successor to decennial Census long form data, intended to replace it completely. Are you arguing that they have devised a system with no comparability to the survey it is replacing? The Census Bureau expects people to compare ACS data to decennial data.

It is valid to explore the differences between the median income measure from the decennial Census and the same statistic in the ACS. A quick search of the Census site did not come up with any studies of the differences, but it's clear there could be a few problems: different reference periods (CY1999 vs previous 12 months), a differently defined universe, or the difference in the data collection procedure, for instance. The question, however, appears to be intended to be substantially the same.

The map may very well be wrong; as I've said, the numbers are suspicious. But it's not sufficient to say that it's drawn from two dissimilar surveys, because it is not.

Posted by: CRH | Sep 8, 2006 12:30:35 AM

CRH -- this document gives several specific reasons why the two surveys are not comparable:

Income – Not Comparable – While the concepts are similar, there are several issues that make the data not comparable: differences in the time periods for which data are collected in the ACS versus the Census; adjustments for inflation in the ACS data; accuracy of the respondents' answer; and the rates of imputation when the Census Bureau cannot get answers to these questions.
Reference Period – The 2005 ACS asks respondents for their income over the 12 months prior to completing the questionnaire. The 2000 Census asks respondents about their income in calendar year 1999.
Inflation Adjustment – Since the income data on the ACS is collected over an entire year, it refers to incomes received over a 23 month period (12 months prior to January 2005 through 12 months prior to December 2005). The Census Bureau adjusts incomes to represent the same time period using the Bureau of Labor Statistics' Consumer Price Index for all urban consumers – research series (CPI-U-RS).
Accuracy of the Respondent's Answer – Respondent accuracy can depend on the relative stability of the respondent's income and their ability to recall changes, especially if there are major fluctuations in their income. The 2000 Census asks about income in 1999 at a time when most respondents have the information needed to complete their income taxes. It may be more difficult for an ACS respondent to recall income over the previous 12 months.
Imputation of Non-Response – Historically, rates of imputation for non-response in the ACS have been much lower than in the 2000 Census because of the use of highly training interviewers in the ACS. In the 2000 Census about 33 percent of all New York residents 15 and older had income imputed versus about 25 percent in the ACS.


Household and family incomes – Not comparable - Concepts are comparable but in addition to the issues above, they are also affected by differences in household composition due to the different residence rules used in the 2000 Census and the ACS.


Posted by: Stuart Buck | Sep 8, 2006 12:40:56 AM

Notice that with the new, correct data posted at the Washington monthly, there are some states that were just HAMMERED. As I count em, 8 states had wage losses of over 10% during this time.

Essentially all of the midwest has lost extremely large amounts of income during a full, extended recovery. Wisconson has lost over 16% inflation adjusted during a period when the economy has grown by over 10 inflation adjusted%. A 26% difference, if wages had simply kept pace with economic growth ( of course this never happens).

We don't have to wonder why Indiana might actually vote more Democrats to office, they are simply voting with their wallets.

In many ways the actual data is a worse picture for the economy than the incorrect data.

Jane,

There are so many its hard to single out one. And its not that any single one is so bad, its that he selectively uses every statistic he can to paint a picture that simply isn't true. A common claim of his is that this economy is the 'Greatest story never told'. Its possible he has said this every time he has a program about the economy for the last 5 months. He says it nearly every time, in any case. He then quotes a few of the current days statistics to back this up, usually quite selectively, for example using whichever employment number was higher over the last month.

Take a quick look at the Washington Monthly numbers. Would you in good faith call this 'The greatest story never told'? But these numbers are exactly what you would expect when you know that median wages are down 2.3% across the country over the last RECOVERY.

People are suffering out there. There are real people behind those statistics. In wisconsin, average wages down 16%? At least the Freep numbers attempted to explain this reality to their readers in Michigan (whose numbers are down 15% btw), even though they were incorrect in their methodology. They were trying to explain reality. Larry, he just lies.

Posted by: mickslam | Sep 8, 2006 8:47:02 AM

The comments to this entry are closed.