< Margins of Error

Transcript

Friday, September 05, 2008

BOB GARFIELD:
This is On the Media. I'm Bob Garfield.
BROOKE GLADSTONE:
And I'm Brooke Gladstone. The RNC balloons barely had time to deflate before the media jumped to assess the impact of the event. A CBS three-day poll released Thursday had the candidates tied at 42 percent. A Gallup poll, over the same period, put Obama up by seven points. A Rasmussen poll ending one day later had Obama at 46 percent and McCain at 45 or, if you include leaners, whoever they are, Obama at 48 percent and McCain at 46.

David Moore was a senior editor at The Gallup Poll for 13 years, and in his recent book, The Opinion Makers, he explains why we should take all this poll talk with a huge grain of salt. Consider, he says, the opinion polls in 2003 prior to the war in Iraq. They all showed overwhelming support for the war by a margin around two to one.

But these polls did not give respondents a chance to say they had no opinion, and that, says Moore, severely skewed the results. In fact, the one Gallup poll that did drill down a bit, asking if respondents would be upset if there were no invasion, found that only 29 percent really favored going to war.
DAVID MOORE:
In fact, the public was evenly divided, with a plurality who didn't have an opinion one way or the other. And that, I think, would have been a very different picture of public opinion at the time than the one that was predominantly portrayed in the media, which showed overwhelming support for the war.
BROOKE GLADSTONE:
And it would have prompted, no doubt, a different response from Capitol Hill.
DAVID MOORE:
Yes, I think it’s pretty clear that there were many Democrats, particularly in the Senate, who voted for the war who might very well have voted against it.

More importantly, what it did is it reinforced the media coverage, which pretty much went along with what the administration was saying and with what general public opinion was indicating, without essentially going and trying to find opposite points of view.
BROOKE GLADSTONE:
You say polls fail to reflect again and again and again, not only the depth of our commitment but the depth of our ignorance on an issue.
DAVID MOORE:
That’s correct. The media polls and the media pollsters refuse to acknowledge that there are probably around 40 to 60 percent of the public, at any given time, on any given issue, who are really so unengaged in the issue they really don't have an opinion.

Let's take, for example, SCHIP, which is the State Child Health Insurance Program. The truth is a good number of people had no real knowledge of what SCHIP was all about. Well, typically what they got were around 90 to 95 percent of the people expressing an opinion, and they were very much influenced by the way the question was asked.

So, for example, CBS found 81 percent support for SCHIP because they said, oh, by the way, here’s a program that would help poor children get health insurance. Would you like to see that funded or not? And sure enough, they did.

Gallup went out and asked the question that said it would be a government program and gave all sorts of other kind of information. Only 41 percent supported it, a 40-point difference simply in the way the question was asked, and mostly asked of people who didn't know anything about SCHIP, except what they'd been told by the pollsters themselves.
BROOKE GLADSTONE:
Let's talk about election polls. You make a point that media and polls that serve the media abhor the undecided voter. And although we see undecideds in every single poll that comes out in an election season, you say that they are never given enough weight.
DAVID MOORE:
Yeah, one can take a look, for example, at the recent Gallup Daily Tracking Poll. They asked the question - if the election were held today, who you would you vote for, you know, Barack Obama or John McCain? They find that somewhere around 95 percent of the people have made up their mind.

But, of course, the election’s not being held today, so it’s a hypothetical race. The way you could ask the question is, do you support Obama, do you support McCain, or haven't you made up your mind yet? Right now there are probably around 35 to 40 percent of voters who are undecided.

Well, I think the reason the media pollsters don't want to emphasize the undecided vote is that they think the media, the journalists, would not be interested in a poll that said there are 40 percent of the people undecided. They’d say - but gee, if people haven't begun to just make up their minds yet, or such a large segment, why don't we wait and start polling later?
BROOKE GLADSTONE:
A lot of people who follow the polls are very concerned about the fact that young voters might be underrepresented because they live by cell phone, rather than by landline, and so they're harder to find.
DAVID MOORE:
There is evidence that suggests that young people who do not have cell phones are different from young people who have cell phones, only. Young people who have cell phones, only, have a different lifestyle. There is a big question in the polling industry as to whether or not we are distorting the results when we don't include cell phones.

And the evidence seems to be that with respect to many issues, we are distorting the results. It’s not quite clear that with respect to the election issues we're distorting results, although I would say that by the next election cycle probably all of the pollsters will have to move to incorporate cell-phone-only users into their samples if they don't want to get really wild results.
BROOKE GLADSTONE:
We've noted the ways that results can be unintentionally distorted, but you also note that there are plenty of ways that they are intentionally distorted.
DAVID MOORE:
Well, there was a big example on whether or not there should be oil drilling in ANWR, the Arctic National Wildlife Refuge. Frank Luntz was working for a conservative organization and John Zogby is working for a liberal organization. They asked questions that were essentially the same, but they preceded the questions with different questions so that it influenced what the people said.

For example, in Frank Luntz’ who wanted to come up with the results that showed that people supported oil drilling, he asked maybe 18 or 19 [LAUGHS] questions about energy and about the need for getting oil, and then finally said, should we [LAUGHS] drill for oil in ANWR?

On the other side, when John Zogby was doing it for the Wilderness Society, he asked some questions about environment and about protecting the environment, and then finally came and asked about drilling for oil in ANWR, and he found the majority against it.

So those kinds of results are more often found among pollsters who are paid by some special interest group to do polling and to come up with the results.

I don't see those really very intentionally manipulated results occurring amongst the major media pollsters who typically don't have an ideological agenda in mind. Their major agenda is to make sure that they got enough people answering the question so that they don't have to report 40 to 50 percent [LAUGHS] of the people were undecided.
BROOKE GLADSTONE:
So if leaving out the undecideds skews the results so much, should right-minded pollsters just fall on their swords and stop asking questions that the public can't really answer?
DAVID MOORE:
No, I don't think it’s good not to ask questions. But I think what’s important to do is to ask some preliminary questions. We should say, do you favor it, do you oppose it, or don't you have an opinion? And then, among people who have an opinion, I do think we need to know whether they consider it something that’s really important or whether it’s just a kind of top-of-mind response. Then we could truly find out what the public is thinking, even about arcane subjects.

Probably about a lot of ‘em we're going to have to say, you know, 60 percent of the public has no idea what’s going on.
[BROOKE LAUGHS]
But it seems to me that’s a very important part that calls for our political leaders to be leaders, to help educate the public on issues that are really important, rather than just assume that everybody has an opinion and that that opinion is fixed.
BROOKE GLADSTONE:
Thank you very much.

DAVID MOORE:
Thanks, Brooke.
BROOKE GLADSTONE:
David Moore, a senior editor at The Gallup Poll for more than a decade, is currently a senior fellow at the Carsey Institute at the University of New Hampshire. His new book is called The Opinion Makers: An Insider Exposes the Truth Behind the Polls.