Jon Stewart says those who watch Fox News are the "most consistently misinformed media viewers"

EDITOR'S NOTE: On the June 21, 2011, edition of The Daily Show, Jon Stewart accepted our False verdict and apologized, saying, "I defer to (PolitiFact's) judgment and apologize for my mistake. To not do so would be irresponsible."

On the June 19, 2011, edition of Fox News Sunday, comedian Jon Stewart -- host of The Daily Show on Comedy Central -- sat down for an interview with Chris Wallace. Many readers asked us to review one of his claims.

"Who are the most consistently misinformed media viewers?" Stewart asked Wallace. "The most consistently misinformed? Fox, Fox viewers, consistently, every poll."

Wallace didn’t challenge Stewart’s assertion that Fox -- widely perceived as a conservative-leaning network -- produced more misinformed viewers. But we thought it was an assessment worth checking.

We found two polling organizations that have produced periodic "knowledge" surveys differentiated by the respondent’s frequent news sources. One is the Pew Research Center for the People and the Press, and the other is worldpublicopinion.org, a project managed by the Program on International Policy Attitudes at the University of Maryland.

Let’s start with Pew’s findings.

Pew periodically studies media usage and public knowledge. They ask whether a respondent is a "regular" reader, viewer or listener of major print, television, radio and Internet news sources, and they ask a series of basic factual questions about news and public affairs to gauge how well-informed the respondent is.

After conferring with Pew researchers, we found three surveys since 2007 that shed some light on how informed Fox viewers are compared to consumers of other media. Here they are:

• February 2007 Political Knowledge Survey. Pew asked respondents 23 questions, such as who the vice president is, who the president of Russia is, whether the Chief Justice is conservative, which party controls the U.S. House of Representatives and whether the U.S. has a trade deficit. The ability to answer 15 of these questions correctly earned the respondent a place in the "high knowledge" category.

Pew then categorized various media sources by the percentage of their followers who earned a high knowledge rating. The media outlets fell into three categories -- those that had 50 to 54 percent in the high knowledge group, those that had 40 to 49 percent in the high knowledge group, and those that had 34 to 39 percent in the high knowledge group.

In descending order, the 50-to-54 percent group included The Daily Show and its Comedy Central cousin, The Colbert Report; major newspaper websites; the PBS News Hour with Jim Lehrer; Fox News’ The O’Reilly Factor; National Public Radio; and Rush Limbaugh’s syndicated radio talk show.

The 40-to-49 percent category included national newsmagazines; television news websites; local daily newspapers; Internet news sources like Yahoo and Google; and CNN.

Finally, the 34-to-39 percent group included the network evening news shows; online news discussion blogs; Fox News Channel; local television news; and the network morning shows.

Now, let’s analyze the data.

Fox isn’t last on the list, although it’s close -- 35 percent of Fox viewers earned a high knowledge rating, which was tied with local television news and was one point ahead of the network morning shows.

However, Fox’s 35 percent score places it exactly at the national average. This seems paradoxical -- Fox ranks near the bottom of a long list of media outlets, yet it sits right at the national average. But there’s an explanation. Lots of respondents reported following none of the media outlets they were asked about, and those respondents did quite poorly on the knowledge quiz -- not surprisingly. That meant that the non-media-using respondents brought down the national average, but they didn’t constitute a separate category that ranked lower than Fox on Pew’s chart.

Since Stewart was referring to "media viewers," this doesn’t undercut his point. However, the data includes an important counterpoint to Stewart’s claim: Viewers of at least one show on Fox scored quite well -- The O’Reilly Factor, of whom 51 percent made it into the high knowledge group. That made it equal to National Public Radio -- a longtime target of conservative complaints about liberal media bias -- and only three percentage points behind Stewart’s own show, at 54 percent.

• April 2008 Media Survey. Compared to the 2007 survey, the 2008 survey looked at a wider variety of media outlets but used a narrower selection of questions designed to test the respondent’s current-affairs knowledge. The pollsters asked three questions: "Do you happen to know which political party has a majority in the U.S. House of Representatives?" "Can you tell me the name of the current U.S. Secretary of State?" And "who is the current prime minister of Great Britain?" Anyone who went three-for-three earned the high knowledge designation.

Here, the range of results was much wider. Once again, Fox News was just about at the national average -- 19 percent of Fox viewers scored in the high knowledge category, compared to 18 percent of all respondents -- but this time a handful of news outlets scored lower than Fox did. With scores ranging from 17 percent all the way down to 9 percent, they were CNBC, local television news, network news, morning news shows, television newsmagazines, personality magazines, religious radio, the Weather Channel, CBS News, Access Hollywood and similar shows, and in last place, the National Enquirer.

And once again, particular Fox shows scored well above the average. Hannity & Colmes was one of only four choices to exceed 40 percent -- the others were the New Yorker/the Atlantic, NPR and MSNBC’s Hardball -- while The O’Reilly Factor scored 28 percent, or 10 points above the national average. (Hannity & Colmes even exceeded Stewart’s Daily Show in this poll, 42 percent to 30 percent.)

In all, this poll undercuts Stewart’s position even more than the 2007 poll did.

• June 2010 Media Consumption Survey. For 2010, respondents were asked four questions -- which party controlled the House of Representatives, what post was held by Eric Holder, which company was run by Steve Jobs and which country has an active volcano that had recently disrupted international air travel. This time, Pew didn’t specifically use a "high knowledge" measurement but rather broke down responses by how many were answered correctly by each media outlet’s followers.

Once again, Fox News as a whole ranked fairly low among regularly used media outlets -- 20 percent answered all four correctly and 18 percent answered three correctly. Still, those numbers beat the national average of 14 percent and 20 percent, respectively. (The best-scoring outlet, the Wall Street Journal, posted scores of 51 percent and 23 percent, respectively.)

Fox actually scored better than its two direct cable-news rivals -- MSNBC, which is a liberal counterpoint to Fox, and CNN, which is considered more middle-of-the-road. Also scoring lower than Fox were local television news, the evening network news shows and the network morning shows.

And for the third time, particular Fox shows scored well. Hannity ranked fifth (just ahead of MSNBC’s liberal shows hosted by Keith Olbermann and Rachel Maddow) and O’Reilly ranked ninth. For the first time, Pew included Glenn Beck in its rankings, and the Fox host finished 12th -- slightly ahead of Stewart’s own Daily Show.

We asked Michael Dimock, Pew’s associate director for research, what he thought Pew’s data meant for Stewart’s claim. He said it’s crucial to understand that different news sources appeal to different types of people -- and that highly political programming of any type attracts regular readers and viewers "who are, most likely, already highly knowledgeable prior to their exposure to those particular sources. Separating what knowledge they bring with them from what they learn while reading or watching is virtually impossible."

By contrast, Dimock said, for media outlets with a much broader reach -- including Fox -- "the average regular consumer of these sources is less informed than the more niche audiences, because these sources, by design, reach and appeal to a broader cross-section of the public. In most of our studies, the regular readers and viewers of these broad-based news sources are not significantly more or less informed than the average American, and there is no systematic pattern showing one broad-based source has a more knowledgeable audience than any other."

Pew’s methodology is not immune from critique. Most notably, it’s not clear whether knowing the answer to a few current-affairs questions translates into being "informed." However, the experts we spoke to said that Pew is an unbiased source and that its data is credible.

Given that, we conclude that the Pew data demonstrates that the reality of who’s "misinformed" is a lot more nuanced than Stewart makes it out to be.

Now let’s turn to the studies by worldpublicopinion.org. We found two studies by the group that tracked current-affairs knowledge.

• "Misperceptions, The Media and The Iraq War" study, 2003. This study focused on the Iraq War and the lead-up to it. It asked three questions: "Is it your impression that the U.S. has or has not found clear evidence in Iraq that Saddam Hussein was working closely with the al-Qaida terrorist organization?" "Since the war with Iraq ended, is it your impression that the US has or has not found Iraqi weapons of mass destruction?" And whether, "The majority of people favor the US having gone to war."

On these questions, Fox clearly did the worst among the major news outlets. The "misperception rate" for Fox was 45 percent. The highest for other news outlets was CBS News at 36 percent; those with lower "misperception rates" included CNN, ABC, NBC, the print media and NPR/PBS, which was lowest at 11 percent.

This study is probably the strongest support we found for Stewart’s claim, in part because the difference between Fox and the other news outlets was so stark, and in part because the questions asked have pretty clear-cut "right" and "wrong" answers.

• Misinformation and the 2010 Election, 2010. This study inspired some of the most intense response of any we looked at. For instance, the liberal blog Daily Kos trumpeted the study with the headline, "CONFIRMED: New Study Proves That Fox News Makes You Stupid." But it’s also the study that prompted the fiercest counterattack on methodological grounds.

The study asked 10 questions. Some are fairly cut and dried -- which president signed the TARP law to bail out Wall Street (Bush), which implemented the automaker bailout (both Bush and Obama), whether the stimulus bill included tax cuts (it did),and whether Barack Obama was born in the United States (he was).

But a few were in a bit grayer area, often asking respondents to gauge what experts have concluded about policy trends.

One was, "Is it your impression that most economists who have studied it estimate that the stimulus has created (a) saved or created several million jobs, (b) saved or created a few jobs, or (c) caused job losses."

Another was, "Is it your impression that economists who have estimated the effect of the health reform law on the federal budget deficit over the next 10 years, (a) more think it will not increase the deficit, (b) views are evenly divided, or (c) more think it will increase the deficit."

The Baltimore Sun’s television critic, David Zurawik, wrote a column shortly after the study appeared, expressing skepticism about the study.

Zurawik wrote, "what you have for the definition of a respondent who is considered ‘informed’ is essentially someone who agrees with the conclusions of experts in government agencies. When specific questions in the survey are framed around facts, like who was president when a certain piece of legislation passed, you can say someone is misinformed." But that’s not the case with some of this survey’s questions.

One question from the study that struck us as one that ordinary Americans might answer differently than economists asked, "Do you think now that the American economy is (a) starting to recover, or (b) still getting worse?" The study based the "correct" answer -- that the economy has begun to recover -- on the widely accepted judgment of when the last recession ended, as well as gross domestic product estimates and statistics for personal income. However, given the phrasing of the question, a respondent might think the question was asking for a personal opinion of how the recovery was going, rather than what the official statistics say.

In a note, the study’s authors acknowledged such concerns, but defended their approach. "When dealing with topics that have been highly politicized, it is common to default to the position that all perceptions are relative and treatment of any position as more or less true is itself inherently political," the study says. "We believe that such a position is at odds with what is necessary for well-functioning democracy."

We should note that like this study, PolitiFact often uses the Congressional Budget Office and the Medicare trustees as credible sources, and we often query experts to come up with our rulings. We think there’s a difference between bestowing a False rating on an elected official -- whose job it is to know about public policy -- and calling an ordinary American "misinformed" for getting the exact same question "wrong." At the very least, these questions seem less clear-cut than asking who the vice president is.

For this reason, we believe that this study should carry less weight in analyzing Stewart’s comment.

So we have three Pew studies that superficially rank Fox viewers low on the well-informed list, but in several of the surveys, Fox isn’t the lowest, and other general-interest media outlets -- such as network news shows, network morning shows and even the other cable news networks -- often score similarly low. Meanwhile, particular Fox shows -- such as The O’Reilly Factor and Sean Hannity’s show -- actually score consistently well, occasionally even outpacing Stewart’s own audience.

Meanwhile, the other set of knowledge surveys, from worldpublicopinion.org, offer mixed support for Stewart. The 2003 survey strikes us as pretty solid, but the 2010 survey has been critiqued for its methodology.

The way Stewart phrased the comment, it’s not enough to show a sliver of evidence that Fox News’ audience is ill-informed. The evidence needs to support the view that the data shows they are "consistently" misinformed -- a term he used not once but three times. It’s simply not true that "every poll" shows that result. So we rate his claim False.

Featured Fact-check

Bill O'Reilly

stated on February 6, 2011 in an interview with President Obama

The majority of people in polls do "not support Obamacare."