A Cautionary Tale

July 30, 2013
Posted by Jay Livingston

Andrew Gelman (here) summarizes and links to a “killer story” that “ is so great that all quantitative political scientists (and sociologists, and economists, and public health researchers)” should take notice.  He’s right.

First, there’s the clever finding that political moderates literally see more shades of gray than do people whose views are more extreme, right or left.  Literally.  It’s a test of color perception.*

But the more important part of the story is Part II.  The authors (Brian Nosek, Jeffrey Spies, and Matt Motyl) could have gotten the study published, but they decided to do a replication first just to put the clincher on their findings.  The result: a p = .01 effect completely disappeared: p = .59.

I’ve commented before (here) on difficulties with replication and the more general problem of diminishing effects.  (See also Jonah Lehrer’s New Yorker article  “The Truth Wears Off.”) But this is as dramatic a turnaround as I know of.

In a comment on Andrew’s blogpost, Ashok Rao suggests that authors post the odds they would give on replication.  Making the authors bet on their results “seems like a pretty good way to discern papers where authors believe what they publish from, well, that where the ‘ample incentives’ dominate.”  (Rao also links to his own paper where he quotes Alex Tabarrok: “bets are a tax on bullshit.”)

------------------------------
* If you’re thinking “50 Shades of Gray,” Andrew already beat you to it.  That, in part, is the title of his blogpost.

Anagrams - Combinatorial Probabilities

July 27, 2013
Posted by Jay Livingston

Maybe you’ve just taken a course in advanced probability.  Here’s a problem. Consider the following tweet*


What is the probability that someone else within the next day or two, coincidentally and without any knowledge of this tweet, would tweet a message that is a perfect anagram of this one? 

I have no idea even how to start thinking about it. The tweet has 29 letters, probably the more frequently used letters.  How many groupings of them form words, how many of those groupings make sense, and so on.  I give up.  But here’s one answer.


Second question.  What is the probability that someone would create a program to cull the Twitter universe, extract anagrams, and post them to a Tumblr page?  I’m not sure how to calculate that one either, but when you see the site, you might well think the probability approaches 1.0, i.e., “It had to happen.”**


This Tumblr has been up for less than a week, and so far there are about thirty examples, most of them short. It’s possible that the pool of matches has been edited to include only those that sound like they might be a conversation.  Like this:


Or this conversation between hooker_225 and FutureShrink:

You can find the entire collection at Anagramatron (here).

---------------------------------
* Ignore whatever else Victoria and Larry, with their interesting @ might be doing. Focus on the letters in the message.

** UPDATE:  My advanced probability informant tells me that it can be done with a fairly simple algorithm. Take two phrases, strip out everything but letters, sort alphabetically, and check to see if they are identical.  For the 400 million tweets in a single day, your computer has to do only about 80 trillion such comparisons.

Poll Puzzle

July 25, 2013
Posted by Jay Livingston

People responding to opinion surveys don’t have to be consistent or logical.  That’s why using only a few questions to gauge public preferences on policy is a risky business.  Here are results on three questions from a recent NBC/WSJ poll.

(Click on the graphic for a larger and clearer view.)
Obama’s approval rating is nearly four times greater than that of Congress. Yet by ten percentage points (48% - 38%) people prefer that Congress “take the lead in setting policy for the country.” 

This paradoxical result is not a one-off.  Approval of the president almost always greater than approval of Congress.  Since January 2005, approval of Congress has never been above 40% and is often below 20%.  Yet since at least 1994, the earliest year shown in the survey, only once has the president topped Congress on the “take the lead” question.  That one time was January 2002, only four months after the attacks on the Trade Towers and Pentagon.

Political scientists must have an explanation for this.  I just don’t know what it is.

Symbolic Events and Public Opinion

July 24, 2013
Posted by Jay Livingston

A single event can take on great symbolic importance and change people’s perceptions of reality, especially when the media devote nearly constant attention to that event.*  The big media story of the killing of Trayvon Martin and the trial of George Zimmerman probably does not change the objective economic, social, and political circumstances of Blacks and Whites in the US.  But it changed people’s perceptions of race relations.

A recent NBC/WSJ poll shows that between November of 2011 and July 2013, both Whites and Blacks became more pessimistic about race relations.


(Click on a graph for a larger view.)

Since 1994, Americans had become increasingly sanguine about race relations.  The Obama victory in 2008 gave an added boost to that trend.  In the month of Obama’s first inauguration, nearly two-thirds of Blacks and four-fifths of Whites saw race relations as Good or Very Good.** But now, at least for the moment, the percentages in the most recent poll are very close to what they were nearly 20 years ago. 

The change was predictable, given the obsessive media coverage of the case and the dominant reactions to it.  On one side, the story was that White people were shooting innocent Black people and getting away with it.  The opposing story was that even harmless looking Blacks might unleash potentially fatal assaults on Whites who are merely trying to protect their communities.  In both versions, members of one race are out to kill members of the other – not a happy picture of relations between the races.

My guess is that Zimmerman/Martin effect will have a short life.*** In a few months, we will ascend from the depths of pessimism. Consider that after the verdict in Florida there were no major riots, no burning of neighborhoods to leave permanent scars – just rallies that were for the most part peaceful outcries of anger and anguish.  I also doubt that we will see the optimism of 2009 for a long while, especially if employment remains at its current dismal levels. 

---------------------------

* Journalist Martin Schram called the coverage, “a roadblock . . . stretched across all lanes of democracy’s information highway.  It blocked the far right lane, the center lane, and the far left lane. Which is to say, Fox News, CNN and MSNBC.”

** The percentages responding Very Good are so small – usually in single digits for both Whites and Blacks – that I combined the two categories.  For a .pdf with the original survey data, go here.

*** Language peeve. The term short-lived does not mean that something was lived for a short time. It means that it had a short life. Therefore, a trend is “short-lived,” pronounced with a long “i” just as someone with a short knife is short-knived.