The Harvard International

Journal of Press/Politics
http://hij.sagepub.com

Perceptions of Political Bias in the Headlines of Two Major News

Organizations
Jeffrey N. Weatherly, Thomas V. Petros, Kimberly M. Christopherson and Erin N.
Haugen
The Harvard International Journal of Press/Politics 2007; 12; 91
DOI: 10.1177/1081180X07299804
The online version of this article can be found at:
http://hij.sagepub.com/cgi/content/abstract/12/2/91

Published by:
http://www.sagepublications.com

Additional services and information for The Harvard International Journal of Press/Politics can be
found at:
Email Alerts: http://hij.sagepub.com/cgi/alerts
Subscriptions: http://hij.sagepub.com/subscriptions
Reprints: http://www.sagepub.com/journalsReprints.nav
Permissions: http://www.sagepub.com/journalsPermissions.nav
Citations http://hij.sagepub.com/cgi/content/refs/12/2/91

Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

Perceptions of Political Bias in the

Headlines of Two Major News
Organizations
Jeffrey N.Weatherly,Thomas V. Petros, Kimberly M. Christopherson,
and Erin N. Haugen
Although claims of media bias are abundant, systematic and scientific investigations
of potential biases are rare. The present study was an attempt to determine
whether a perception of bias would be found in the headlines of lead or major stories taken from the Web sites of two major American news organizations, CNN and
FOX News, during the final two months of the 2004 presidential campaign.
Significant perceptions of bias were found. Overall, headlines taken from CNN were
rated as significantly more liberal than those taken from FOX News. Headlines
taken from FOX News were rated as slightly on the liberal side of neutral. With
CNNs headlines slightly to the left of FOX News, instructing participants that the
headlines came from a particular source did not influence the results.Although the
study by no means provides the definitive answer to whether major news organizations have biases, it indicates that perceptions of bias exist.
Keywords:

media bias; perceptions; CNN; FOX News

Bias in the media has been widely discussed in recent years.The idea that major
American news organizations may have political biases or agendas was raised by
Goldberg (2002). Goldberg, a former employee of CBS News, argued that the
organization had a liberal bias. He asserted that the vast majority of employees
at CBS News, including those in decision-making positions, held liberal views
that indirectly biased the news coverage of stories because the views played a role
in what stories were deemed newsworthy. Others, like news personality Bill
OReilly, have gone a step further and accused most major news organizations of
intentionally conspiring to forward a liberal agenda under the disguise of unbiased reporting (e.g., OReilly 2004). For example, some have coined CNN the
Clinton News Network because of its supposed favorable treatment of President
Clinton while he was in office (e.g., Ponti 2004).
Press/Politics 12(2):91-104
DOI: 10.1177/1081180X07299804
2007 by the President and the Fellows of Harvard College
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

91

92

Press/Politics 12(2) Spring 2007

Still others have challenged these claims by saying that there is a conservative
bias in the media. Franken (2003) satirically attacked the views and statements
of Goldberg and others (e.g., OReilly) primarily affiliated with one news
organizationFOX News. Indeed, the Internet is scattered with Web sites purportedly providing documentation of a conservative bias at FOX News (e.g.,
Senti 2005). Although not claiming a conservative bias in the media, some
researchers have suggested that those with conservative viewpoints have waged
a campaign of sorts to build perception among the populace that the media is liberally biased (Watts et al. 1999). According to these researchers, this campaign
has been accomplished largely through media stories on the topic of bias rather
than on documented instances of biased news coverage.
For better or worse, most claims of bias consist of anecdotal evidence and
are forwarded by individuals with their own biases. Little in the way of systematic, scientific inquiries into bias has been conducted, especially using
experimental methods (but see Brescoll and LaFrance 2004). Furthermore, it
is unclear as to how the rampant claims of media bias have influenced the
publics perception of bias in the media (Watts et al. 1999). It is possible that
claims of media bias have actually created a situation in which people may perceive bias from a particular news source even if no real bias exists. It is also
possible that peoples own political leanings may buffer them from perceiving
bias even when bias does exist.
When dealing with bias in the media, there are many different forms it can
take. Some are indirect. Goldberg (2002) accused CBS News of bias, not necessarily in how a story was presented, but rather, in what stories were chosen
to be reported. Other forms are more direct. Taking a nonneutral stance on
the issues being covered once a story is chosen would be an example.Arguably,
both of these forms of bias can be intentional or unintentional. For instance,
news directors may be keenly aware that their political views are guiding their
choice of what is newsworthy. Then again, such choices may be made in the
absence of overt knowledge of a bias.The complexity of interpreting bias further increases when one considers that the recipients of supposedly biased
news have their own biases of which they may or may not be aware.
Thus, any single study on political bias in the media will likely be incomplete.The present attempt was to determine whether a certain aspect of news
reporting (i.e., headlines posted on the Web sites of two major American news
sources) would be perceived as biased. Headlines were chosen for several reasons. First, it was possible, in a relatively brief period of time, to expose participants to a number of different headlines from a particular source. Second,
and perhaps more importantly, many of the news stories found on the Web
pages of major news outlets come from the same source, the Associated Press.
Thus, the text of the story is largely, if not entirely, identical when read from
the different news agencies.What differs, however, is how those agencies pitch
the story.
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

Weatherly et al. / Political Bias in Headlines

93

During approximately the final two months of the 2004 presidential

campaign, we collected headlines from the CNN (http://www.cnn.com) and
FOX News (http://www.foxnews.com) Web sites. The sites were checked at
least once daily, and headlines were chosen according to two criteria. First, the
story had to have a reasonable relation, directly or indirectly, to a political issue
(e.g., a story about a mudslide would not be considered political, whereas coverage of a presidential campaign function would be). Second, coverage of the same
story had to simultaneously be available on the Web sites of both news agencies.
Participants then ranked each headline along a liberal-neutral-conservative
continuum. Some participants were accurately informed that the headlines
were drawn from two different news agencies. Others were informed that all
the headlines were drawn from one news source, either CNN or FOX News.
The hypothesized outcome depended on which of the lead authors was
queried. One predicted that the headlines taken from FOX News would be perceived as more conservative than those taken from CNN, with CNN headlines
being perceived as neutral. Another predicted that headlines taken from CNN
would be perceived as more liberal than those taken from FOX News, with FOX
News headlines being perceived as neutral. Similar predictions were made
about informing participants of the source of the headlines. One researcher predicted the ratings of the headlines would be skewed toward the conservative end
of the continuum when participants were told that all the headlines were from
FOX News.The other predicted the ratings would be skewed toward the liberal
end when participants were told that all the headlines were from CNN.
Method
Participants

Participants were 225 (seventy-four males, 149 females, and two who failed
to identify their sex) individuals recruited through the University of North
Dakota psychology department and its subject pool.The mean age of the participants was 19.71 years (SD = 5.41). Participants received research credit in
their psychology course(s) for their participation (if applicable).
Materials

Participants completed two measures. The first was the General Social
Survey (Davis et al. 2002). It consisted of nineteen questions that assessed
political party affiliation, general political views, and certain behavioral characteristics. This measure was used for two reasons. First, it was used to provide some assessment of how the current sample compared with the average
citizen. Second, it is possible that individuals perceptions of political bias by
the news organizations could themselves be influenced by the views held by
the participants. This measure was thus used to help identify those views so
they could be controlled for statistically if necessary.
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

94

Press/Politics 12(2) Spring 2007

The second measure was a list of twenty-eight pairs of headlines taken from
the CNN and FOX News Web sites between September 13, 2004, and
November 3, 2004.To be chosen, the headlines needed to have a political topic
and needed to appear as either the lead story or under the More News
section on the CNN Web site, and simultaneously, as the lead story or under
the Latest Headlines section on the FOX News Web site. Also included with
the real headlines were two pairs of fabricated headlines that were intentionally biased.Two of the headlines were biased toward a conservative viewpoint.
The other two were biased toward a liberal viewpoint. A list of the real and
fake headlines can be found in the appendix.
Procedure

On entering the room, participants went through the process of providing

informed consent. Once consent was obtained, the researchers distributed
packets that contained the General Social Survey followed by the headlines.
For the first group (paired-headlines group; n = 60), the headlines were paired
together such that the two headlines from CNN and FOX News on the same
news story were adjacent to one another. These participants read the following instructions before rating the headlines:
The following represent headlines that appeared on two major news Web sites during the recent national campaign period.We are interested in determining whether
the headlines were biased toward one political party or another.To do so, we would
like you to rate each headline with a percentage from 0 percent to 100 percent,
with 0 percents being liberally biased (i.e., pro-Kerry, pro-Democrat; anti-Bush,
anti-Republican) and 100 percents being conservatively biased (i.e., pro-Bush,
pro-Republican; anti-Kerry, anti-Democrat). If a headline showed no favoritism to
one side or another (i.e., totally neutral), you would rate it 50 percent.

Remaining participants were randomly divided into three groups (nonpaired

groups). These groups rated the headlines presented in random order. One of
these groups read the identical instructions as those that appear above.A second
group read the identical instructions with the exception that they were informed
that all of the headlines were from CNNs Web site.The third group of participants read that all of the headlines were from FOX NewsWeb site.
After participants completed the General Social Survey and rated the headlines, they were debriefed.They then received documentation of their participation (for purposes of earning course credit) and were dismissed.
Results

Examination of the data from the General Social Survey suggests that
the composition of the paired-headlines and nonpaired-headlines groups was
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

Weatherly et al. / Political Bias in Headlines

95

similar. In the paired-headlines group, twenty-four participants (40 percent)

identified themselves as affiliated with the Democratic party, twenty-three
(38.3 percent) claimed affiliation with the Republican party, five (8.3 percent)
claimed to be Independents, and eight individuals (13.3 percent) marked the
dont know option. As for political leanings, nineteen individuals (31.7 percent) identified themselves as being extremely liberal, liberal, or slightly liberal. Twenty-six people (43.3 percent) identified themselves as moderate.
Fifteen people (25 percent) identified themselves as slightly conservative, conservative, or extremely conservative. For the nonpaired-headlines groups,
fifty-four (32.7 percent) identified themselves as Democrats, seventy-three
(44.2 percent) identified themselves as Republicans, and thirty-seven (22.4
percent) identified themselves as Independents. As for political leanings, fiftysix (33.9 percent), sixty-two (37.6 percent), and forty-five (27.3 percent)
individuals identified themselves as liberal, moderate, and conservative,
respectively. A significant correlation was found between party affiliation and
political leanings (rs [177] = .509; p < .01). Results for this analysis and all that
follow were considered significant at p < .05.
The rationale of including fake headlines was to ensure that participants
could identify bias. Participants, however, had difficulty discerning the bias in
one pair of fake headlines. The headlines Bush Defends Ill-conceived HealthCare Policy versus Kerry Evades His Lackluster Senate Record generated
mean ratings of 58.5 percent (median = 60 percent) and 56.0 percent (median
= 60), respectively. Bias in the second set of fake headlines, on the other hand,
was well discerned by the participants.The headline Bush on Correct Side of
Moral Issues had a mean rating of 87.6 percent (median = 100), while the
headline Kerrys Morals Superior had a mean rating of 11.9 percent (median
= 0). The second pair of fake headlines was therefore used to screen participants. Participants ratings were eliminated from further analysis if the rating
of these latter fake headlines was beyond two standard deviations from the
overall mean for that headline. For the paired-headlines group (n = 51), this
criterion removed nine participants from the following analyses. For the nonpaired-headline groups (n = 137), it removed twenty-eight participants (group
informed that headlines came from two news agencies, n = 44; group
informed that headlines came from FOX News, n = 49; group informed that
headlines came from CNN, n = 44).
Ratings for the paired-headlines group were analyzed by conducting a twoway (News Agency Headline Topic) repeated-measures analysis of variance
(ANOVA).1 This analysis yielded a significant main effect of news agency (F[1,
50] = 14.54, p < .001), indicating that participants rated the headlines taken
from CNN as more liberal (M = 46.31) than those taken from FOX News
(M = 49.17). The main effect of headline topic was significant (F[27, 1,350] =
23.61, p < .001), indicating that ratings varied across the twenty-eight different
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

96

Press/Politics 12(2) Spring 2007

Table 1

Mean ratings (with standard deviations) of each headline by participants in the pairedheadlines group and whether the ratings differed significantly
Headline Pair
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

CNN

FOX News

Significant

37.31 (31.27)
63.24 (21.72)
27.16 (27.34)
43.78 (19.22)
73.83 (27.31)
35.10 (29.42)
30.35 (32.56)
37.96 (21.40)
64.41 (22.77)
29.47 (28.72)
55.00 (14.59)
17.94 (22.96)
43.33 (23.21)
48.63 (19.32)
55.98 (18.79)
45.10 (22.81)
44.12 (19.69)
47.88 (22.68)
48.73 (11.78)
45.59 (15.83)
35.78 (25.64)
37.06 (25.54)
37.53 (28.21)
41.37 (25.04)
49.51 (24.03)
73.31 (31.38)
50.47 (28.74)
76.67 (27.42)

28.39 (32.65)
62.75 (19.55)
35.29 (28.08)
46.08 (13.69)
62.45 (21.53)
48.04 (23.11)
33.18 (22.75)
49.31 (32.79)
75.00 (26.17)
33.43 (25.66)
47.55 (16.32)
52.45 (10.88)
44.61 (20.27)
72.25 (28.02)
53.92 (19.58)
51.76 (16.70)
48.33 (20.14)
39.76 (21.87)
51.37 (9.39)
49.71 (14.57)
39.90 (21.55)
37.65 (23.82)
42.35 (25.50)
42.45 (23.27)
48.53 (20.35)
82.45 (20.94)
46.59 (26.78)
51.08 (30.57)

X
X
X
X
X
X

X
X

topics. Finally, the interaction between news agency and headline topic was
also significant (F[27, 1,350] = 6.86, p < .001).
To follow up the significant interaction, a post hoc Tukey HSD was calculated. Results demonstrated that within-pair differences existed for nine of the
twenty-eight headlines. The mean ratings for each headline by participants in
the paired-headlines group can be found in table 1.
Ratings for the nonpaired-headline groups were analyzed by conducting a
three-way (Instructions News Agency Headline Topic) mixed-model
ANOVA.2 The instruction as to the news source of the headlines was the grouping factor, while news agency and headline topic were repeated measures. This
analysis yielded a nonsignificant main effect of instructions (F[2, 134] = 1.62),
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

Weatherly et al. / Political Bias in Headlines

97

indicating that ratings of the headlines did not vary as a function of the supposed source. The main effect of news agency was significant (F[1, 134] =
29.46, p < .001). Again, participants rated the headlines taken from CNN as
more liberal (M = 46.69) than those taken from FOX News (M = 49.12).The
main effect of headline topic was also significant (F[27, 134] = 119.39,
p < .001), as was the news-agency-by-headline-topic interaction (F[27, 134] =
33.78, p < .001). All remaining effects were not significant.
Because of the significant interaction, a post hoc Tukey HSD was again calculated to determine differences in ratings within each pair of headlines. That
analysis indicated that differences in ratings were present in fifteen of the
twenty-eight pairs. The mean ratings for each headline by participants in the
nonpaired-headlines groups can be found in table 2.
Discussion

The present study assessed whether headlines for the same news story
posted on the Web sites of CNN and FOX News would be perceived as biased
toward one particular political perspective and whether any potential bias
would differ between news agencies. Results indicate that perceptions of the
headlines varied as a function of agency. Participants who read the competing
headlines side by side (i.e., the paired-headlines group) rated nine of the
twenty-eight headlines to differ between news agencies. Participants who read
the headlines in random order (i.e., the nonpaired-headlines groups) rated fifteen of the twenty-eight headlines from the same story differently. Overall,
participants in both groups rated the headlines from CNN as significantly more
liberal than those from FOX News. The absolute difference in the mean evaluations was relatively small (3 percentage points), and the evaluations of both
FOX News and CNN were close to the 50 percent markthe neutral point.
Nevertheless, CNN tilted more to the left than did FOX News.
The conclusion that CNNs headlines were perceived as more liberal than
FOX News, rather than FOX News headlines being perceived as more conservative than CNNs, is supported by the finding that overall, the headlines
from FOX News were rated slightly on the liberal side of neutral. If FOX
News headlines had been perceived as biased toward the conservative viewpoint, then the overall ratings should have exceeded 50 percent (i.e., the conservative side of neutral).
The present study was also designed to determine whether information
about the news agencies themselves would alter perceptions of bias. The
results failed to support this idea. Participants who were not told the source
of the headlines, were told CNN was the source, or were told FOX News was
the source did not differ in their ratings of the headlines.Thus, despite accusations that certain media outlets are biased (e.g., Franken 2003; Ponti 2004),
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

98

Press/Politics 12(2) Spring 2007

Table 2

Mean ratings (with standard deviations) of each headline by participants in the nonpairedheadlines groups and whether the ratings differed significantly
Headline Pair
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

CNN

FOX News

Significant

35.11 (36.49)
78.66 (21.41)
15.46 (25.26)
41.09 (23.01)
83.57 (24.28)
33.95 (35.66)
36.62 (37.19)
31.87 (31.00)
72.48 (24.65)
24.13 (29.90)
52.81 (11.52)
9.07 (18.29)
38.37 (23.21)
44.88 (22.20)
60.75 (23.57)
49.04 (14.75)
47.02 (20.82)
46.91 (25.81)
47.48 (18.31)
46.41 (14.42)
31.48 (24.22)
50.26 (24.51)
32.02 (27.91)
37.45 (27.25)
39.55 (24.52)
81.07 (26.03)
55.97 (21.26)
83.91 (23.96)

18.95 (29.02)
80.83 (21.51)
30.21 (35.35)
43.53 (21.08)
81.65 (21.29)
56.52 (31.45)
24.17 (24.50)
35.80 (35.51)
86.48 (20.28)
15.43 (19.29)
47.89 (10.74)
49.58 (12.15)
43.04 (20.27)
78.18 (27.40)
61.61 (21.36)
51.15 (10.73)
46.12 (21.02)
36.43 (25.88)
51.12 (7.25)
52.68 (14.22)
34.21 (27.15)
35.82 (24.00)
31.79 (27.33)
34.99 (28.24)
52.33 (20.13)
85.83 (22.30)
50.97 (20.11)
57.95 (34.19)

X
X
X
X
X
X
X
X
X

X
X
X
X
X
X

either being informed of the source of the headlines had no influence on the
participants perceptions of bias, the participants were not aware of such accusations, or both.
Before labeling CNNs headlines as liberally biased and claiming FOX News
to be bias free, there are numerous aspects of the present procedure and results
to be addressed. First, the sample used in the present study was a convenience
sample of college students from the University of North Dakota.Although political party affiliations and political leanings of participants tended to mirror the
national statistics, it is undisputable that the participants were younger than the
average voter.Their education background also differed from a large number of
voters, given that many voters have never attended a university.
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

Weatherly et al. / Political Bias in Headlines

99

One could also argue that because the study was conducted in a red state
(i.e., a state that voted Republican in the 2004 presidential election), you
would expect favoritism toward the supposedly conservative news agency
(FOX News) and bias against the supposedly liberal agency (CNN).This argument cannot be completely refuted. However, it can be questioned on a
number of levels. For one, the sample included more people who self-identified as liberal than as conservative. Second, there were instances in which participants ranked the headlines from CNN as significantly more conservative
than those from FOX News (e.g., headline pairs five and twenty-eight).Third,
participants ranked several of the headlines from FOX News as being
extremely conservative (e.g., headline pairs nine and twenty-six). Finally,
information about the source of the headlines did not alter participants rankings, which you would expect if a preexisting bias was at work.
Another potential argument is that participants could not properly perceive
bias (or mistakenly perceived it). The fact that participants had difficulty discerning the bias in one of the fake headline pairs supports this argument.
Furthermore, some ratings were in the opposite direction than one might
expect. For instance, the CNN headline Bush Pushing Health Care Message
was rated on the conservative end of the spectrum by the average participant.
However, one could reasonably rate this headline as liberal inasmuch as the term
pushing carries a negative connotation. The argument that participants poorly
perceived bias cannot be completely refuted. However, it can be said that participants very accurately identified the bias in the second pair of fake headlines.
More importantly, perceptions of bias are determined by the perceiver. Thus,
although it may be the case that participants in the present sample perceived bias
differently than others might, we were specifically interested in reported perceptions of bias. According to the present participants, a bias was perceived.
A related argument would be that because of the youth of the participants,
they were relatively naive to political issues, and thus, insensitive to political
nuances present or absent in the headlines. A different sample of people who
were experts on political issues may have produced vastly different results.
Because the General Social Survey administered in the present study cannot
determine the extent of someones political knowledge, it is not possible to
fully assess or refute this argument.Then again, if this argument were correct,
it would suggest that the present results actually represent a pure measure of
bias in that the participants may not have been reading into the headlines.
Yet another argument against the finding of bias is that although headlines
from CNN were rated as significantly more liberal than those from FOX
News, the actual difference was not large. Indeed, as noted above, there were
several instances in which headlines from CNN were reported to be significantly more conservative than those from FOX News. Certainly, the small
difference and the instances of being more conservative can be used to argue
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

100

Press/Politics 12(2) Spring 2007

that the headlines from CNN were not liberally biased.The one point that cannot be argued, however, is that across these twenty-eight pairs of headlines,
participants reported a systematic difference in the political leaning in the
headlines from CNN versus from FOX News. That leaning may have been
small, but it was systematic.
One could, then, legitimately argue that the results of the present study
were determined not by the participants but by the researchers who chose
which headlines to use. It seems reasonable to suspect that if an entirely different set of headlines would have been chosen, less bias, more bias, or bias in
a different direction may have been reported. As with other arguments, this
one cannot be completely refuted. However, the present study was designed
to try to guard against the potential biases of the researchers. For one, the ratings of bias were not the only dependent measure. Rather, headlines from both
news agencies on the same story were rated so that even if there was a selection bias, the difference between the sources could serve as a second dependent measure. Second, the researchers were aware of this potential confound
and tried to ensure that stories covered the spectrum.The fact that some of the
headlines from both sources were rated as very liberal (e.g., headline pairs
three and ten) while others were rated as very conservative (e.g., headline
pairs nine and twenty-six) suggests that this effort was successful.
In guarding against this confound, however, the present study likely created
another one. One selection criterion for the headlines was that both news
agencies had to be covering the same story as the lead or major story at the
same time. There were numerous instances in which potentially biased (or
unbiased) headlines were not used because the opposing news agency was not
covering that particular story simultaneously. One such example was the coverage of CBS and its reporting of documents, now considered by many to be
forgeries, that supposedly indicated that President Bush did not properly serve
his time in the military (e.g., Associated Press 2004). Coverage of the CBS
scandal could be found almost daily on FOX News. It appeared less frequently,
at least as a major story, on CNN. The present study is silent as to potential
biases represented by excessive coverage or omissions.
The present data do not indicate that the reporting of the different news
agencies is or is not biased. As noted above, headlines were chosen because
articles are often similar on different Web sites because the sites get the articles from the same source (e.g., the Associated Press).Thus, in the instances in
which the same story ran under a different headline, there was no difference
in reporting between CNN and FOX News.A different type of study would be
necessary to determine whether biases exist between the two agencies in
terms of actual news reporting.
Although accusations of bias in the media are commonly heard, relatively
little in terms of systematic research has been done on the topic. The present
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

Weatherly et al. / Political Bias in Headlines

101

study represents one such attempt. It looked at one potential source of bias
between two (of many) news agencies on a sample of twenty-eight pairs of
headlines for news stories. Admittedly, the results of the present study do not
provide the definitive answer as to whether bias exists in the media, who may
be biased, and how much they may be biased. However, the results do indicate
that participants perceived a bias in the headlines that were rated.That bias was
that the headlines taken from CNN were rated as more liberal than those taken
from FOX News.
It must be noted, however, that the mean difference in evaluation of the two
sources headlines was relatively small and that headlines are likely a weaker
test of bias than the stories themselves.The last pointextending this type of
experimental study to the full storiesis clearly an area for future research.
Appendix
Pairs of headlines in the order seen by the paired-headlines group. The parentheses represent the pair number (see tables 1 and 2).The news agency is indicated after
each headline, and the date the headlines appeared is also presented. Participants did
not see the pair number, agency, or date.
(1) __________Kerry Blasts Loss of Weapons Ban, CNN, 9/13/2004
__________ Kerry Slams Bush over Ban, FOX
(2) __________Bush Addressing National Guard, FOX, 9/14/2004
__________ Bush to Talk to National Guard Convention, CNN
(3) __________Kerry Touts Health-care Plan, FOX, 9/14/2004
__________ Kerry Woos Wisconsin Seniors, CNN
(4) __________Americans Convicted in Kabul, FOX, 9/15/2004
__________ 3 Americans Guilty in Afghan Torture Case, CNN
__________ Bush on Correct Side of Moral Issues, FAKE
__________ Kerrys Morals Superior, FAKE
(5) __________Bush Pushing Health Care Message, CNN, 9/16/2004
__________ Bush to Talk Health Care, FOX
(6) __________ Laura Bush Speech Disrupted, FOX, 9/17/2004
__________ Laura Bush Heckled by Mom of Soldier Killed in Iraq, CNN
(7) __________ Kerry Challenges Bush, FOX, 9/20/2004
__________ Kerry Attacks Bushs Iraq Plans, CNN
(8) __________Kerry Advisor Spoke with CBS Document Source, CNN,
9/21/2004
__________ Kerry Aide Linked to CBS Docs, FOX
(9) __________ Iraqi PM: Thank You, America, CNN, 9/23/2004
__________ Allawi Thanks Bush for Iraq Effort, FOX
(10) __________ Kerry Outlines Anti-terror Plans, FOX, 9/24/2004
__________ Kerry Unveiling Plan for Terror War, CNN
(11) __________ Kerry, Bush Plan for Debates, FOX, 9/27/2004
__________ Bush, Kerry Gear Up for First Debate, CNN
(12) __________Candidates Downplay Poll, FOX, 10/3/2004
__________ Polls: Kerry Won Debate, CNN
Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

102

Press/Politics 12(2) Spring 2007

(13) __________ Report: No Iraq WMDs, FOX, 10/6/2004

__________ Official: No WMD Stockpiles in Iraq, CNN
(14) __________ Bush:WMD Report Justifies Iraq War, FOX, 10/7/2004
__________ Iraq WMD Report Enters Political Battlefield, CNN
__________ Bush Defends Ill-conceived Health-care Policy, FAKE
__________ Kerry Evades His Lackluster Senate Record, FAKE
(15) __________ Weapons Handover Begins in Sadr City, CNN, 10/11/2004
__________ Al-Sadr Militia Disarms, FOX
(16) __________ 108th Congress Wraps Up, FOX, 10/12/2004
__________ Congress Wraps up Rancorous Session, CNN
(17) __________ Insurgents under Attack, FOX, 10/12/2004
__________ Iraqi Forces Raid Ramadi Mosques, CNN
(18)__________ Source: U.S.Tried Twice to Rescue Hostages, CNN,
10/13/2004
__________ Iraq Hostage Rescue Failed, FOX
(19) __________ Candidates Duel in Final Face-off, FOX, 10/14/2004
__________ Domestic Issues Dominate Debate, CNN
(20) __________ Swing-state Stumping, FOX, 10/15/2004
__________ Kerry, Bush Search for Persuadable Voters, CNN
(21) __________ Poll: Military Has Iraq Doubts, FOX, 10/16/2004
__________ Poll:Troops, Families Question Iraq Strategy, CNN
(22)__________ Study:White Disproportionately Wealthier in U.S., CNN,
10/18/2004
__________ Study: Income Gap Grows, FOX
(23)__________ Dozens of Unarmed Iraqi GIs Killed in Ambush, FOX,
10/24/2004
__________ Dozens of Iraqi Soldiers Killed Execution Style, CNN
(24) __________ Explosives Missing in Iraq, FOX, 10/25/2004
__________ 380 Tons of Iraq Explosives Vanish, CNN
(25) __________ GI Answers on Al Qaqaa, FOX, 10/29/2004
__________ Pentagon Seeking Answers for Missing Explosives, CNN
(26) __________ Bush Camp Claims Victory, FOX, 11/3/2004
__________ Bush Camp Certain of Win; Election Still up in the Air, CNN
(27) __________ Senate: Daschle Loses, GOP Gains, CNN, 11/3/2004
__________ Daschle Loses Senate Seat, FOX
(28) __________ Kerry Concedes to Bush, FOX, 11/3/2004
__________ Bush Wins: Kerry Calls to Concede, CNN
Notes
1.

There is debate as to whether a covariate analysis should be used when dealing with completely within-subject designs.Without taking sides in that debate, an analysis of covariance, with political leanings (i.e., liberal, moderate, conservative) and party affiliation
(Democrat, Independent, Republican, and Dont Know) as the covariates, was conducted
separately for the paired-headlines group. Similar results were obtained. Furthermore,
both of these factors pulled out nonsignificant portions of the variance (Fs < 1).Thus, we
report results from the analysis conducted without covariates.

Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

Weatherly et al. / Political Bias in Headlines

2.

103

Separate one-way ANOVAs of political party affiliation and political leanings by group
yielded nonsignificant effects (Fs < 1), indicating that a covariate analysis was not necessary. However, as a safeguard, an analysis of covariance was conducted for the nonpairedheadlines group using political leanings and party affiliation as covariates. The effect of
these factors was not significant. Thus, we report results from the ANOVA conducted
without covariates.

References
Associated Press. 2004. CBS News Admits Bush Documents Cant Be Verified. Internet.
<http://www.msnbc.msn.com/id/6055248/>.
Brescoll, V., and M. LaFrance. 2004. The Correlates and Consequences of Newspaper
Reports of Research on Sex Differences. Psychological Science 15:515-20.
Davis, J.A., T.W. Smith, and P.V. Marsden. 2002. General Social Survey 2000. Chicago:
National Opinion Research Center (producer); Storrs, CT: The Roper Center for Public
Opinion Research, University of Connecticut (distributor).
Franken, A. 2003. Lies and the Lying Liars Who Tell Them. Penguin: New York.
Goldberg, B. 2002. Bias:A CBS Insider Exposes How the Media Distort the News.Washington, D.C.:
Regnery.
OReilly, B. 2004. The Media Strategy to Elect Kerry. Internet. <http://www.billoreilly.com/pg/
jsp/general/genericpageblue.jsp?pageID=474#a>.
Ponti, L. 2004. MSNBC, the New Clinton News Network? Internet. Feb. 18.
<http://www.frontpagemag.com/Articles/ReadArticle.asp?ID=12234>.
Senti, S., ed. 2005. <http://www.oreilly-sucks.com/>.
Watts, M.D., D. Domke, D.V. Shah, and D.P. Fan. 1999. Elite Cues and Media Bias in
Presidential Campaigns: Explaining Public Perceptions of a Liberal Press. Communication
Research 26:144-75.

Biographical Notes
Jeffrey N.Weatherly is a professor of psychology and currently serves as the chairperson of the
psychology department at the University of North Dakota. His research expertise is in the
experimental analysis of behavior.
Address: Department of Psychology, University of North Dakota, Grand Forks, ND 58202-8380;
phone: (701) 777-3470; fax: (701) 777-3454; e-mail: jeffrey_weatherly@ und.nodak.edu.
Thomas V. Petros is a Chester Fritz Distinguished Professor of Psychology at the University of
North Dakota. His research expertise is in developmental and cognitive psychology.
Address: Department of Psychology, University of North Dakota, Grand Forks, ND 58202-8380.
Kimberly M. Christopherson is a doctoral student in general/experimental psychology at the
University of North Dakota. Her research expertise is in psychological issues in educational
settings.

Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010

104

Press/Politics 12(2) Spring 2007

Address: Department of Psychology, University of North Dakota, Grand Forks, ND 582028380.

Erin N. Haugen is a graduate of the clinical psychology PhD program at the University of
North Dakota and is currently a psychology fellow at the UC Davis Medical Center in
Sacramento, California. Her research expertise is in anxiety disorders, especially as they relate
to postpartum issues.
Address: University of California, Davis Medical Center, 2230 Stockton Boulevard, Sacramento,
CA 95817.

Downloaded from http://hij.sagepub.com at ITESM-CAMPUS MONTERREY on March 9, 2010