Tuesday, September 24, 2013

Ignore Those Polls! (Shutdown Edition)

There are polls out that have people opposing a shutdown over the ACA; there's also at least one poll out that has blame split if there is a shutdown.

Ignore those polls!

The later -- "who would you blame?" type polls -- are particularly not to be trusted. People aren't very good at predicting their own reactions to things, especially things they don't know much about and haven't thought much about. But I'd be somewhat skeptical of the first set, as well. It's one thing for (even) Republicans say they are against a description of the Ted Cruz/Mike Lee shutdown strategy; it's quite another for them to oppose it once it happens. Perhaps they will, but it's just not the same thing.

Here's a couple of rule-of-thumb type advice that I use on these sorts of things. Is the question one that respondents, especially those who are not political junkies, actually would have solid opinions about absent a pollster phone call? A high-profile election next week; sure. A budget deal, or a budget negotiations scenario? Far less likely. And then: does the question ask respondents for their current reactions to something, or to predict future reactions?

On questions such as these, which violate both of those conditions, I think it's far better to treat the polling at best as a vague indication of vague first thoughts. At best. Meanwhile, to predict the actual reaction (at least in polling) to future events, it's much wiser to focus on how information flows are likely to develop from those events; that's what I was doing in my TAP column about Life During Shutdown. Granted: such analysis requires political judgement, and can easily be wrong. But in my view, that's still a better bet than trusting what people say now how they think they'll react in the future. 

2 comments:

  1. I disagree, but purely from my perch being interested in the data, and not in a predictive sense.

    The "pre" question on "Do you think this is a good way to run the railroad?" gives us the baseline. Now, post-shutdown, after a stronger signal from partisan elites about what is "right" (not that elites aren't sending those signals now, but not many people are paying attention), we get to see changes in opinions.

    That tells us more about the phenomenon of elite-driven opinion change, which is interesting in its own right.

    Now, we won't know more about what the American public "truly" wants to happen, but we never really know that...we rely on representative government to get us close to that notion, but that's all. It doesn't actually matter what the public "truly" wants; what matters is what the people the public have chosen to act as their delegates do. This isn't elitism; it's just recognizing that representative democracy isn't popular democracy.

    ReplyDelete
    Replies
    1. I will readily and happily concede that many polls which are useless for their predictive value or for what they tell us about "real" public opinion may nevertheless be highly useful for those doing research on public opinion.

      Delete

Note: Only a member of this blog may post a comment.

Who links to my website?