Research as Politics

January 15, 2008
Posted by Jay LivingstonBy now, I should be used to dishonest research by the politically motivated – the careful selection of time periods or samples so as to magnify effects. That sort of thing. But I’m not. Maybe Max Weber should have written a third essay – political scholarship as a vocation.*

I was browsing at Intel Dump, a reality-based blog about military affairs. Blogger-in-chief Philip Carter was ripping into the Sunday NY Times article on homicide among Iraq vets both for its method and its implied stereotype image of the crazed combat vet.

But in the comments – and Intel Dump gets a ton of comments – there was a reference to a Heritage Foundation study: The press release title was, “Post-9/11 Military Recruits Wealthier, Better Educated, Study Shows.” The lead was, “Wartime recruits who joined the United States military in 2004 and 2005 tended to be better educated and wealthier than their civilian peers.”

That didn’t sound right. It certainly did not square with news reports about Army recruiters shanghaiing near-imbeciles and schizophrenics in order to meet their quotas, or promising signing bonuses to tempt impoverished youths.

As the press release says,
This disproves the idea, expressed on Oct. 30 by Sen. John Kerry, that only those who fail in school end up in the military. “If you study hard, do your homework and you make an effort to be smart, you can do well. If you don’t, you get stuck in Iraq,” the former presidential candidate told college students.**
The Heritage Foundation is a conservative think tank, and highly political. Their view of the war in Iraq is basically, it’s all good. But how did they get the data to show the beneficial effects the war was having on recruitment?

I didn’t read every word of the report, but it appears that the author, Timothy Kane, uses a semantic trick to obscure an important distinction. Unless you pay close attention, you might forget that “the Army” is not the same as “the military” or “the armed forces.” The report focuses almost exclusively on “the military.”

But the branches of the military differ greatly in their involvement in Iraq and in their recruitment. The Air Force and Navy have suffered fewer than 125 fatalities and fewer than 250 casualties serious enough to require medical air transport. (The Heritage report didn’t mention these figures. I had to find them here [this link is broken, sop you're on your own in finding casualty figures on the separate branches of the military].) So Navy and Air Force recruitment has apparently been going along as usual.

The Army has suffered the heaviest losses – more than twenty times the combined Air Force and Navy figures – and it’s the Army that is having trouble recruiting and that in fact had lowered its standards. But of course you wouldn’t know that from reading the Heritage press release or even the full report. All the tables on education and income show only data for all military recruits; they do not break down the sample by service branch.

* For non-sociologists: Ninety years ago, Max Weber wrote two famous essays, “Scholarship as a Vocation” and “Politics as a Vocation.”

**There is some question as to what Kerry meant by this remark. He later claimed that he meant to say “you get us stuck in Iraq” the way that President Bush, not an outstanding student, had done.

3 comments:

Brad Wright said...

Good example of "damned lies and statistics." I guess they didn't realize that you were on the web, otherwise they would have never tried!

It's hard not to be skeptical about most social statistics presented in the media...

Anonymous said...

I love the crazy coincidences of the Internet. I actually had a student write a paper citing a very similar Heritage study (http://www.heritage.org/Research/NationalSecurity/cda05-08.cfm). I was also surprised when I read those claims about recruits, so I followed the student's citation to the report and checked it out. Not only were you spot on about the careful choice of population, the author (also Kane) made an interesting comparison in the first table presented in that report (seen here: http://www.heritage.org/Research/NationalSecurity/upload/cda05-08_t1.gif).

The top-left box says, "Recruits are not disproportionately poor. Mean household income for recruits in 1999 was $41,141 (in 2000 dollars), compared to the general population median of $41,994."

That's great and all, except that it's means to medians. The mean gen pop household income in 1999? $66,235. But really, that's only 60% more than the recruits - doesn't change the results much.

Jay Livingston said...

Kane's comparison is not quite so egregious as it might seem. That "mean" income for recruits is really a mean of medians. He didn't have income data on recruits so he took the median income of their zip code. Then he took the mean of these medians. Still, why didn't he take the median of the zip code medians? And why did he use zip codes rather than census tracts, which are much smaller and have a smaller variance and are therefore more accurate for estimating individual income from neighborhood income?