Schizophrenia Research 68 (2004) 283 – 297

www.elsevier.com/locate/schres

The Brief Assessment of Cognition in Schizophrenia: reliability,

sensitivity, and comparison with a standard neurocognitive battery
Richard S.E. Keefe a,*, Terry E. Goldberg b, Philip D. Harvey c, James M. Gold d,
Margaret P. Poe a, Leigh Coughenour a
a
Department of Psychiatry and Behavioral Sciences, Duke University Medical Center, P.O. Box 3270, Durham, NC 27710, USA
b
Clinical Brain Disorders Branch, National Institutes of Health, USA
c
Department of Psychiatry, Mount Sinai School of Medicine, USA
d
Maryland Psychiatric Research Center, USA
Received 21 February 2003; received in revised form 9 September 2003; accepted 10 September 2003

Abstract

Studies of neurocognitive function in patients with schizophrenia use widely variable assessment techniques. Clinical trials
assessing the cognitive enhancing effect of new medications have used neurocognitive assessment batteries that differed in
content, length and administration procedures. The Brief Assessment of Cognition in Schizophrenia (BACS) is a newly
developed instrument that assesses the aspects of cognition found to be most impaired and most strongly correlated with
outcome in patients with schizophrenia. The BACS requires less than 35 min to complete in patients with schizophrenia, yields
a high completion rate in these patients, and has high reliability. The BACS was found to be as sensitive to cognitive
impairment in patients with schizophrenia as a standard battery of tests that required over 2 h to administer. Compared to
healthy controls matched for age and parental education, patients with schizophrenia performed 1.49 standard deviations lower
on a composite score calculated from the BACS and 1.61 standard deviations lower on a composite score calculated from the
standard battery. The BACS composite scores were highly correlated with the standard battery composite scores in patients
(r = 0.76) and healthy controls (r = 0.90). These psychometric properties make the BACS a promising tool for assessing
cognition repeatedly in patients with schizophrenia, especially in clinical trials of cognitive enhancement.
D 2003 Elsevier B.V. All rights reserved.

Keywords: Schizophrenia; BACS; Neurocognitive assessment batteries

1. Introduction one and a half to two standard deviations below

The cognitive deficits of schizophrenia are pro-
found and devastating. Patients with chronic schizo-
phrenia demonstrate impairments that range between
* Corresponding author. Fax: +1-919-684-2632.
E-mail address: Richard.keefe@duke.edu (R.S.E. Keefe).
healthy controls on several key dimensions of cogni-
tion, especially verbal memory, working memory,
motor speed, attention, executive functions and verbal
fluency (Saykin et al., 1991; Keefe, 1995; Harvey and
Keefe, 1997; Heinrichs and Zakzanis, 1998). These
components of cognitive dysfunction in schizophrenia
also have an impact on functional outcome (Green,

0920-9964/$ - see front matter D 2003 Elsevier B.V. All rights reserved.
doi:10.1016/j.schres.2003.09.011
284 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297
1996; Green et al., 2000) as they are all correlated with
poor functional abilities.
Since the second generation of antipsychotic
medications became widely available, dozens of
studies of the neurocognitive impact of these medi-
cations have been completed. While most of these
studies have concluded that second generation anti-
psychotics improve neurocognitive function, the in-
terpretation of these results has been challenged, in
part, by the variable test batteries used in each study
(Harvey and Keefe, 2001). There is no standard,
easily administered test battery that specifically and
efficiently assesses the important cognitive deficits in
patients with schizophrenia. Instead, current test
batteries differ widely in content, duration, proce-
dures, and implementation. The limited generalizabil-
ity of neurocognitive findings across studies has
reduced the ability of clinicians and researchers to
make clear conclusions about the relative impact of
the various antipsychotic medications on cognition in
schizophrenia. The absence of a standard measure
has also challenged studies of adjunctive treatments
for cognition in patients with schizophrenia (Fried-
man et al., 2001, 2002; Evins et al., 2000; Heresco-
Levy et al., 1996).
Another drawback of current assessments of cog-
nition in treatment studies of patients with schizophre-
nia is that many of the neurocognitive assessment
batteries used are long and complex. Most neuropsy-
chological assessment batteries used in schizophrenia
studies have been adapted from clinical neuropsychol-
ogy, which assesses the entire profile of neuropsycho-
logical strengths and weaknesses in individuals. These
batteries of tests may require several hours to admin-
ister. Their adaptation for schizophrenia research has
kept some of the length that was originally necessary
for individual assessment, but may no longer be
necessary for research such as clinical trials that
compares groups of patients. In fact, the length of
some assessment batteries may be a limiting factor in
assessing patients repeatedly throughout a clinical trial
(Harvey and Keefe, 2001).
In contrast to these standard assessment techniques
in schizophrenia research and clinical practice, cog-
nitive function in patients with dementia is usually
assessed with a widely used cognitive assessment
tool, such as the Mini Mental Status Examination
(Folstein et al., 1975), Dementia Rating Scale (Gard-
ner et al., 1981), or Alzheimer’s Disease Assessment
Scale (Rosen et al., 1984). These instruments can be
easily administered in typical treatment settings such
as a physician’s office, and are routinely employed as
indicators of treatment change in clinical trials of
anti-dementia drugs. The administration of These
tests shed light on the global severity of patients’
cognitive deficits, track progression, and measure
cognitive changes.
The availability of a quick and efficient tool for
measuring cognition in patients with schizophrenia
could be an extremely useful guide for clinicians
making decisions about potential rehabilitation and
for researchers implementing clinical trials to assess
cognitive improvement. There are several batteries of
tests that are available or in development for the
purposes of brief cognitive assessment.
Several computerized batteries, such as the Cam-
bridge Neuropsychological Test Automated Battery
(CANTAB) (Robbins et al., 1996), the CDR Cogni-
tive Assessment System (Hunter et al., 1997), and
the CogTest Battery (Cogtest, 2002) have been
applied to schizophrenia samples, and have the
option of reduced length. However, portability, reg-
ular software and hardware version changes, and
increased patient and tester burden present imple-
mentation challenges. Other batteries, such as the
Cognitive Screening Instrument for Schizophrenia
(CSIS), a very short (10 min) set of abbreviated
tests (Scot Purdon, personal communication), are
currently in development.
The work completed with the Repeatable Battery
for the Assessment of Neuropsychological Status
(RBANS) (Randolph, 1998; Gold et al., 1999;
Hobart et al., 1999) has clearly demonstrated the
utility of brief assessment approaches. The RBANS
is capable of providing reliable and valid assess-
ments of patients with schizophrenia in various
cognitive domains (Gold et al., 1999; Hobart et al.,
1999; Wilk et al., 2002). As noted in the RBANS
manual, however, the test was originally developed
as a screening measure primarily for elderly subjects,
and the test is heavily weighted by memory, lan-
guage, and visual– perceptual subtests. In addition,
the item difficulties are most relevant to the types of
impairment likely to be observed in patients with
dementing illnesses such as Alzheimer’s Disease,
with ceiling effects in some domains. Although the
 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297
RBANS is clearly sensitive to some of the impair-
ments observed in schizophrenia, and it has utility as
a screening tool for patients with schizophrenia, it
lacks measures of motor, executive, and working
memory performance that may be particularly im-
portant targets for cognitive enhancement in schizo-
phrenia. These limitations suggest the need for a
measure specifically designed for use in schizophre-
nia clinical trials that preserves the desirable features
of the RBANS: brief administration and scoring
time, portablity, repeatability, and availability of
alternate forms.
The Brief Assessment of Cognition in Schizophre-
nia (BACS) has been developed for clinical trials with
these key features. In addition, the domains of cogni-
tive function that are assessed by the BACS are those
found to be consistently impaired, and consistently
related to outcome, in schizophrenia: verbal memory,
working memory, motor speed, attention, executive
functions and verbal fluency. The BACS is fully
portable, and is designed to be easily administered by
a variety of testers, including nurse clinicians, psychia-
trists, neurologists, social workers, and other mental
health workers. It is designed to require about 30 min of
testing time with minimal extra time for scoring and
minimal training demands.
The development of a psychological assessment
instrument should determine the instrument’s test –
retest reliability, sensitivity, criterion-referenced valid-
ity (i.e. comparison to a standard measure), and
comparability of alternative forms. This study aims
to determine the quality of the BACS in these
domains of reliability and validity.
2. Methods
2.1. Subjects
Healthy controls were recruited from a variety of
sources, including bulletin board postings, newspaper
advertisements, and word of mouth. They were inter-
viewed with the selected sections of the Structured
Clinical Interview for DSM-IV Axis I Disorders, Clin-
ical Version, for healthy people and were required not
to have an Axis I disorder according to DSM-IV
criteria, nor any relevant neurologic illness such as a
history of brain trauma.
285
Patients were recruited from the inpatient and
outpatient facilities at Duke University, John
Umstead Hospital, the University of North Carolina
Neurosciences Hospital, and Dorothea Dix Hospital.
Patients were required to meet DSM-IV criteria for
schizophrenia, schizoaffective illness, or schizophre-
niform disorder, to have no history of brain trauma,
nor to be suffering from a current substance use
disorder. There were no specific medication criteria
for inclusion in the patient group. Ninety-four of the
150 patients were being treated with a single atyp-
ical antipsychotic medication (31 with risperidone,
28 with olanzapine, 15 with clozapine, 10 with
aripiprazole, 8 with quetiapine, and 2 with ziprasi-
done), 13 were being treated with a single typical
antipsychotic (6 with haloperidol, 4 with haldol
decanoate, 1 with prolixin, 1 with loxitane, 1 with
navane), 9 with a combination of antipsychotics, and
8 with an antipsychotic that was not currently
known because the patient was in a double-blind
study. Ten were receiving pharmacologic treatment
but not with antipsychotics (e.g. ativan, lithium), and
for 16 patients, medication information was not
available.
The demographic characteristics of the patients with
schizophrenia and the healthy controls are described in
Table 1. Study investigators made a concerted effort to
recruit healthy controls who would match the patients
on parental education, age, ethnic background, and sex.
As indicated in Table 1, groups did not differ signifi-
cantly on any of these measures except sex. Males
were more highly represented in the patient group
(79%) than in the control group (62%)(Fisher’s exact,
P = 0.023).
2.2. Assessment procedures
Subjects were tested on two separate days, with no
more than 3 days between assessments. Subjects were
randomly assigned to a sequence of BACS versions
A and B. On the first test session, subjects received
one version of the BACS followed by the first half of
the standard battery, which included the following
tests: Rey Auditory– Verbal Learning Test (Geffen et
al., 1994), Dot Test of Visuospatial Working Memory
(Keefe et al., 1997), Grooved Pegboard (Ruff and
Parker, 1993), Wide Range Achievement Test, third
edition, Reading subtest (Wilkinson, 1993), Trail-
286 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297

making A (Reitan, 1979), Trailmaking B (Reitan, 2.2.2. Working memory

1979) and Letter –Number Test of Auditory Working
Memory (Gold et al., 1997). On the second test
session, subjects were randomized to receive either
the same or different version of the BACS followed
by the second half of the standard battery, which
included these tests: Wechsler Memory Scale, third
edition, Logical Memory subtest (Wechsler, 1997b),
Controlled Oral Word Association Test (Lezak,
1995), Category Instances (Lezak, 1995), Wechsler
Adult Intelligence Scale, third edition, subtests (Digit
Symbol-Coding, Block Design, Arithmetic, and In-
formation) (Wechsler, 1997a), Continuous Perfor-
mance Tests (A-X, two-digit Identical Pairs and
four-digit Identical Pairs) (Cornblatt and Keilp,
1994), and computerized Wisconsin Card Sort Test
(64 card version) (Heaton, 1993).
The constructs measured with the BACS, including
the tests, procedures, and measures are listed in the
order administered as follows.
2.2.2.1. Digit sequencing task. Patients were pre-
sented with clusters of numbers of increasing length.
They were asked to tell the experimenter the numbers
in order, from lowest to highest. Measures: number of
correct responses (range: 0 – 28); longest sequence
recalled correctly (range: 0 –8).
2.2.3. Motor speed
2.2.3.1. Token motor task. Patients were given 100
plastic tokens and asked to place them two at a time
into a container as quickly as possible. A 60-s time
limit was imposed. Measures: the number of tokens
correctly placed into the container for the first half-
minute, second half-minute, and the 1 min total
(range: 0 – 100).
2.2.4. Verbal fluency

2.2.1. Verbal memory 2.2.4.1. Category instances. Patients were given 60

2.2.1.1. List learning. Patients were presented with
15 words and then asked to recall as many as possible.
This procedure was repeated five times. There were
two alternate forms. Measure: number of words
recalled per trial, in any order (range: 0 –75).
s to name as many words as possible within a given
category. Version A: supermarket items; Version B:
tools.
2.2.4.2. Controlled oral word association test. In
two separate trials, patients were given 60 s to

Table 1
Demographics of the sample
Measure Schizophrenia Controls t P
N Mean S.D. N Mean S.D.
Age 150 34.7 11.3 50 34.8 11.5 0.058 0.953
Education (years) 145 12.0 2.3 50 13.3 2.3 3.469 0.001
Paternal education 105 12.3 3.5 46 12.2 3.8 0.345 0.731
Maternal education 107 12.8 3.3 50 12.1 3.8 1.066 0.288
WRAT-3, Reading 148 41.5 7.4 50 44.4 8.8 2.336 0.035
Sex N (%) 0.023a
Male 119 (79) 31 (62)
Female 31 (21) 19 (38)
Race N (%) 0.352a
African 70 (50) 28 (56)
Caucasian 65 (46) 22 (44)
Other 5 (4) 0 (0)
WRAT-3 = Wide Range Achievement Test, Third edition.
a
By Fisher’s exact test.
 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297
generate as many words as possible that begin with a
given letter. Version A: F, S; Version B: P, R
Measure: number of words generated per trial.
2.2.5. Attention and speed of information processing
2.2.5.1. Symbol coding. As quickly as possible,
patients wrote numerals 1 –9 as matches to symbols
on a response sheet for 90 s. Measure: number of
correct numerals (range: 0 –110).
2.2.6. Executive functions
2.2.6.1. Tower of London. Patients were shown two
pictures simultaneously. Each picture showed three
balls of different colors arranged on three pegs, with
the balls in a unique arrangement in each picture.
Patients were asked to give the total number of times
the balls in one picture need to be moved in order to
make the arrangement of balls identical to that of the
other, opposing picture. There were 20 trials. The
items were of variable difficulty, with a general
tendency for later items to be more difficult. The
test was discontinued if patients made five consecu-
tive incorrect responses. If patients responded cor-
rectly to all 20 trials, two additional trials of greater
difficulty were administered. There were two alter-
nate forms. Measure: number of correct responses
(range: 0 – 22).
2.3. Data analyses
For all subtests and composite scores, test – retest
reliability was measured with intra-class correlations
(ICC) in the patient and control groups separately. The
ICC is a conservative estimate of test –retest reliabil-
ity, as it is sensitive to group mean changes over time
in addition to intra-subject variability. Practice effects
were measured by comparing data collected at test
session 1 to those collected at test session 2 with
within-group t-tests. These were determined in the
patient and control groups separately. Sensitivity to
between-group impairment on all measures was de-
termined with independent t-tests.
The primary measure from each test of the BACS
was standardized by creating z-scores whereby the
test session 1 healthy control mean was set to zero
and the standard deviation set to one. Tests with
287
alternate forms were standardized separately for each
version. A composite score was calculated by aver-
aging all of the six standardized primary measures
from the BACS, and then calculating a z-score of the
composite. Composite scores were calculated for all
subjects using the a priori criterion that they must
have successfully completed all or all but one of the
measures that comprised the composite score. This
criterion included the entire sample, as no subjects
were missing more than one BACS measure.
The relationship among the BACS measures was
determined by calculating Pearson correlations among
the scores. The factor structure of the scores was
determined by performing a principal components
analysis with oblique rotation.
For the standard battery, a composite score was
calculated by adding together the primary measures
from each test after they had been standardized.
Standardized scores for each construct (e.g. verbal
memory) were determined by calculating the mean
of the standardized scores for each of the measures
that comprised the construct. A z-score for each
measure was calculated based upon the healthy
control mean and standard deviation, and the average
of the z-scores was calculated to determine the con-
struct score. The Dot Test working memory deficit
score, Trailmaking A and B scores, and WCST per-
severative error scores were multiplied by 1 so that
good performance was associated with scores in a
positive direction. The following constructs were cal-
culated using the measures listed in parentheses:
Verbal memory (RVLT total scores, WMS-III logical
memory score), Attention (WAIS-III digit symbol,
Mean CPT d-prime), Working Memory (Dot test
working memory error, number of correct items on
Letter –Number Sequencing), Motor Speed (Mean of
dominant and nondominant hand Pegboard perfor-
mance, Trailmaking A), Verbal Fluency (COWAT,
Category instances), Executive functions (Mean of
WCST categories and perseverative error z-scores,
Trailmaking B).
The relative sensitivity of the composite scores
derived from the standard battery and the BACS was
determined by comparing each of their between-
group differences with t-tests. In addition, multiple
logistic regression analyses were performed to deter-
mine the amount of unique between-group variance
that could be accounted for by each battery.
288 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297

Finally, to compare the relation between the
BACS measures and the measures from the standard
battery, the neurocognitive construct summary scores
from the standard battery were correlated with the
BACS measures using Pearson correlations.
3. Results
3.1. Description of BACS data
3.1.1. Test –retest reliability, sensitivity and practice
effects of tests without alternate forms
Table 2 lists the means and standard deviations for
all of the measures from the BACS that do not have
alternate forms. These measures were repeated with
the same form regardless of which version of the
BACS was administered. All tests demonstrated sig-
nificant differences ( P < 0.001) between controls and
patients. The intra-class correlations (ICC) between
performance from test session 1 and test session 2 for
each measure are also included. Each test had one
measure that produced an ICC of 0.79 or greater for
the patient group and healthy control group: total
number of correct responses from the digit sequencing
task; correct responses from the symbol coding task;
and total number of tokens for 60 s from the token
motor task. The longest correct sequence measure for
the digit sequencing tests and the 30-s measures for
the token motor task produced ICCs that were inferior
to the primary measures, and were thus not used in
subsequent analyses. The effect of practice on the
primary measures was small. Only the symbol coding
test showed significant practice effects in patients
(0.25 S.D.) and controls (0.34 S.D.).
3.1.2. Comparability of versions: test – retest reliabil-
ity, sensitivity, and practice effects of tests with alter-
nate forms
Table 3 lists the means and standard deviations for
the BACS measures derived from tests with alternate
forms. The data are organized by the sequence of the
BACS versions subjects received in test sessions 1
and 2. The ICCs between performance for test session
1 and test session 2 for each measure are also
included. Subjects who received the same version
twice (AA and BB) allow for an assessment of
maximal potential practice effects and test – retest
reliability. Subjects who received a different version
on consecutive test sessions (AB and BA) allow an
assessment of general practice effects that are not
dependent upon test version, as well as the compara-
bility of the test forms.
3.1.2.1. Verbal memory. The word lists from each
version were very similar in difficulty in patients and
controls. The final versions of the lists were the result
of a reorganization of the original word lists accom-
plished by switching six of the words between the A
list and the B list, keeping the words in the same or
similar ordinal position as the original list. Since the
samples were smaller for the comparison of the final
lists, we collected data on an additional 30 controls
from Mount Sinai Medical Center not included in the

Table 2
Mean performance and reliability coefficients of BACS tests with no alternate forms in patients with schizophrenia and healthy controls
Measure Schizophrenia Controls P1 P2
Test session 1 Test session 2 ICC Test session 1 Test session 2 ICC
N Mean S.D. N Mean S.D. N Mean S.D. N Mean S.D.
Digit sequencing, total correct 140 15.01 4.65 140 15.71 4.60 0.79 50 19.08 5.27 50 19.26 4.88 0.81 0.007 0.686
Digit sequencing, longest 140 5.76 1.40 140 5.89 1.36 0.66 50 6.58 1.34 50 6.72 1.28 0.61 0.183 0.398
correct sequence
Token motor task (1st 30 s) 142 28.89 8.66 142 29.96 8.23 0.76 50 36.86 8.55 50 38.40 8.10 0.81 0.030 0.038
Token motor task (2nd 30 s) 142 29.37 8.73 142 29.18 8.59 0.71 50 37.40 7.75 50 38.52 7.61 0.54 0.723 0.287
Token motor task total 142 58.27 16.57 142 59.13 16.10 0.79 50 74.26 14.50 50 76.92 14.91 0.80 0.333 0.050
Symbol coding 143 36.63 14.23 143 40.48 16.15 0.90 50 54.86 16.55 50 60.80 18.25 0.83 0.000 0.000
P1 = Significance value for patients day 1 vs. patients day 2, by t-test; P2 = Significance value for controls day 1 vs. controls day 2, by t-test.
Significance value for all patients day 1 vs. controls day 1 comparisons, P < 0.001, by t-test; Significance value for all patients day 2 vs. controls
day 2 comparisons, P < 0.001, by t-test.
 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297 289

original sample. In this separate sample, the word list
difficulty was almost identical, and not significantly
different (A list: mean = 50.33, S.D. = 9.77; B list
mean = 51.00, S.D. = 9.48; t = 0.19, df = 28, n.s.).
The ICCs for this measure ranged between 0.78
and 0.93 in patients and 0.40 and 0.90 in controls.
Due to a randomization procedure that resulted in
fewer than expected controls with an AB or BA
sequence of tests using the final word list, we
administered the word lists only to an additional 11
controls not otherwise included in the study to
increase the sample size for assessing the test – retest
reliability in controls receiving different versions.
The practice effect was significant in patients and
controls who were administered the same version,
but not in subjects who received different versions
on consecutive test sessions. Controls performed
significantly better than patients for each version of
the test. These results support the need for alternate
forms for this test.
3.1.2.2. Verbal fluency. Letter Fluency: In subjects
receiving the same version on consecutive test ses-
sions, the ICCs were good for the patient group,
ranging between 0.69 and 0.78, and lower for the
controls, ranging between 0.53 and 0.65. In subjects

Table 3
Mean performance and reliability coefficients of BACS tests with alternate forms in patients with schizophrenia and healthy controls
Condition Schizophrenia Controls Significance
Test session 1 Test session 2 ICC Test session 1 Test session 2 ICC P1 P2 P3 P4
N Mean S.D. N Mean S.D. N Mean S.D. N Mean S.D.
Verbal memory
A/A 18 33.28 9.38 17 38.65 9.68 0.78 7 51.14 8.11 7 57.71 9.21 0.80 0.003 0.019 0.000 0.000
B/B 16 33.13 12.84 15 40.67 16.35 0.93 9 50.89 11.49 9 64.44 9.14 0.68 0.000 0.001 0.002 0.001
A/B 6 32.50 10.48 6 33.83 14.43 0.92 8 58.40 2.61 8 56.20 2.28 0.40 0.549 0.149 0.074 0.252
B/A 11 33.91 11.87 11 33.00 10.83 0.86 5 51.40 9.02 5 52.40 9.94 0.90 0.584 0.655 0.634 0.550

Verbal fluency
A/A
F 50 10.62 4.02 48 12.00 4.58 0.69 18 14.22 3.90 18 16.17 3.85 0.56 0.004 0.037 0.002 0.001
S 50 11.46 4.36 48 12.00 4.86 0.69 18 16.61 4.73 18 17.72 3.77 0.53 0.292 0.271 0.000 0.000
Supermarkt 50 20.26 7.44 48 20.44 6.73 0.71 18 26.06 6.45 18 26.83 5.88 0.90 0.597 0.240 0.005 0.001
B/B
P 51 10.49 5.08 48 11.56 5.23 0.77 16 14.00 5.94 16 15.69 6.18 0.65 0.031 0.206 0.024 0.011
R 51 8.37 4.40 48 9.31 4.67 0.78 16 14.38 6.53 16 14.00 6.08 0.58 0.035 0.798 0.000 0.002
Tools 51 9.43 3.92 48 9.98 3.83 0.82 16 12.38 6.59 16 15.13 9.13 0.82 0.175 0.038 0.032 0.002
A/B
F/P 24 9.58 4.58 24 10.13 4.41 0.57 8 12.38 3.66 8 15.88 4.52 0.15 0.530 0.108 0.129 0.003
S/R 24 9.50 3.84 24 8.21 4.04 0.60 8 14.63 2.97 8 10.13 4.58 0.13 0.084 0.041 0.002 0.270
Sup/Tools 24 18.54 5.77 24 9.63 3.89 0.46 8 28.75 6.34 8 13.13 5.36 0.81 0.000 0.000 0.000 0.054
B/A
P/F 25 9.76 2.92 24 11.42 4.51 0.12 8 14.00 4.28 8 12.63 3.29 0.61 0.164 0.287 0.003 0.492
R/S 25 8.40 3.32 24 10.88 4.40 0.35 8 10.75 5.04 8 13.38 4.14 0.53 0.017 0.141 0.135 0.168
Tools/Sup 25 9.20 4.88 24 17.75 5.82 0.51 8 11.5 2.73 8 26.50 7.05 0.38 0.000 0.000 0.216 0.001

Tower of London
A/A 50 12.46 4.72 48 13.75 4.80 0.66 18 15.39 3.18 18 16.83 2.75 0.83 0.017 0.002 0.018 0.013
B/B 51 10.90 5.52 46 11.78 5.72 0.77 16 17.19 2.93 16 18.69 2.50 0.73 0.385 0.009 0.000 0.000
A/B 23 14.00 3.30 23 14.00 4.34 0.81 8 16.38 4.31 8 17.13 3.72 0.97 1.00 0.080 0.116 0.080
B/A 25 13.76 5.54 24 14.50 6.41 0.89 8 14.75 3.99 8 14.38 4.21 0.93 0.330 0.504 0.644 0.959
P1 = Significance value for patients test session 1 vs. patients test session 2, by t-test; P2 = significance value for controls test session 1 vs.
controls test session 2, by t-test; P3 = significance value for patients test session 1 vs. controls test session 1, by t-test; P4 = significance value for
patients test session 2 vs. controls test session 2, by t-test.
290 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297

Table 4
Mean performance and reliability coefficients of BACS composite scores in patients with schizophrenia and healthy controls
Condition Schizophrenia Controls Significance
Test session 1 ICC Test session 2 Test session 1 ICC Test session 2 P1 P2 P3 P4
N Mean S.D. N Mean S.D. N Mean S.D. N Mean S.D.
A/A 50 1.70 1.14 0.88 48 1.20 1.25 18 0.08 1.04 0.94 18 0.43 1.19 0.000 0.000 0.000 0.000
B/B 51 1.36 1.01 0.86 48 1.00 0.90 16 0.16 1.08 0.95 16 0.71 1.23 0.000 0.000 0.000 0.000
A/B 24 1.80 1.15 0.92 24 2.03 1.31 8 0.17 0.94 0.87 8 0.23 1.24 0.036 0.080 0.000 0.002
B/A 25 1.06 0.94 0.88 24 0.60 0.81 8 0.32 0.80 0.87 8 0.00 0.90 0.000 0.077 0.054 0.091
P1 = Significance value for patients test session 1 vs. patients test session 2, by t-test; P2 = significance value for controls test session 1 vs.
controls test session 2, by t-test; P3 = significance value for patients test session 1 vs. controls test session 1, by t-test; P4 = significance value for
patients test session 2 vs. controls test session 2, by t-test.

receiving different versions, the ICCs were variable,
but poor overall, with three ICCs below 0.17 and none
higher than 0.60.
The practice effect was small in those receiving the
same version on consecutive test sessions (less than
0.33 S.D. for all letters), and was not statistically
significant in controls and patients for F and S words.
Performance for patients was significantly different
from controls for each of the letters.
Category Instances: The ICCs for subjects receiv-
ing the same version on consecutive test sessions
were good, ranging from 0.71 to 0.90, but poor in
subjects receiving different versions on consecutive
test sessions (0.46 –0.51 in patients; 0.38– 0.81 in
controls).
Patients and controls performed about 2 S.D.
better on Version A (supermarket items) than Version
B (tools). The practice effect was statistically signif-
icant for the tools category, but not for supermarket
items in controls. The pattern of between-group
results suggested that the category of supermarket
items was more sensitive to the differences between
patients and controls than tools. These results sug-
gest that the category of supermarket items is a
superior measure.
3.1.2.3. Tower of London. Versions A and B were
very similar in patients and controls, with no signif-
icant differences between versions. The ICCs ranged
between 0.66 and 0.89 in patients and between 0.73
and 0.96 in controls. The practice effect was small in
patients ( < 0.25 S.D.) and medium in controls (0.4 –
0.6 S.D.). The sensitivity of the test to the differences
between patients and controls was weaker than the
other measures, although they were statistically sig-
nificant in the larger AA and BB samples.
3.1.3. BACS composite scores and BACS profile
Table 4 lists the composite scores for controls and
patients receiving the various combinations of the two
versions of the BACS. Note that all data were stan-
dardized using test session 1 means and standard

Fig. 1. Performance of patients with schizophrenia on the BACS subtests and composite score standardized to healthy controls. All differences
between patients and controls were statistically significant ( P < 0.001).
 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297 291

Table 5 composite score compared to the healthy control

Correlations among BACS measures for healthy controls and mean. In this figure, the data from versions A and B
schizophrenia patients
were collapsed together to include all subjects. All
VM DS VF SC TM TL Comp
differences between patients and controls were statis-
Composite score 0.67 0.73 0.66 0.85 0.84 0.58 – tically significant ( P < 0.001).
Tower of London 0.17 0.44 0.14 0.38 0.41 – 0.59
The BACS composite scores did not differ signif-
Token motor 0.59 0.48 0.45 0.70 – 0.26 0.69
Symbol coding 0.59 0.50 0.51 – 0.47 0.18 0.72 icantly between males (mean = 1.52 F 1.09) and
Verbal fluency 0.28 0.49 – 0.49 0.41 0.08 0.62 females ( 1.37 F 1.10) with schizophrenia. Differ-
Digit sequencing 0.29 – 0.33 0.50 0.29 0.41 0.70
Verbal memory – 0.45 0.29 0.38 0.32 0.29 0.71 Table 7
Correlations in controls above diagonal; correlations in patients Performance of patients with schizophrenia and healthy controls on
below diagonal. Among patients, correlations greater than or equal standard battery of tests
to 0.29, P < 0.001; 0.26 – 0.28, P < 0.01; 0.18 – 0.24, P < 0.05. Schizophrenia Controls
Among controls, correlations greater than or equal to 0.45,
P < 0.001; 0.38 – 0.44, P < 0.01; 0.29 – 0.37, P < 0.05. N Mean S.D. Z score N Mean S.D.
VM = Verbal memory; DS = Digit sequencing; VF = Verbal fluency; RAVLT total 150 33.78 13.01 1.42 50 49.78 11.29
SC = Symbol coding; TM = Token motor task; TL = Tower of Dot average 149 1.85 1.5 1.72 48 0.61 0.72
London; Comp = Composite score. Pegboard 146 105.5 38.11 1.66 50 76.02 17.73
dominant
Pegboard 145 119.16 45.68 2.03 49 84.07 17.25
deviations, therefore a slight practice-related improve- non-dominant
ment for test session 2 data was expected. Mean pegboard 149 112.23 38.34 1.86 50 80.53 17.00
Test – retest reliability was very high, even in sub- Trail making A 147 44.95 22.37 1.45 50 28.54 11.32
jects receiving different versions between test ses- Trail making B 137 131.5 69.33 1.26 50 79.32 41.39
sions. The ICCs ranged between 0.86 and 0.92 for Letter number 137 10.53 4.16 0.79 50 14.40 4.88
sequencing—
patients, and between 0.87 and 0.95 for healthy total
controls. The differences between patients and con- Letter number 137 4.33 1.31 0.81 50 5.36 1.27
trols were statistically significant for all versions and sequencing—
groups with the exception of those who received the longest
tests in the B/A order. The between-group difference Logical 148 7.79 3.94 1.33 50 12.90 3.85
memory—
of the composite score for this group ( P = 0.054) mean A and B
appears to have been smaller than the others due to COWAT—mean 142 10.21 3.84 1.12 50 14.51 3.83
a combination of factors, including poor overall Category 142 11.84 3.55 1.13 50 16.50 4.11
performance in this group of eight controls and better fluency—mean
overall performance in this patient group. WAIS-III 142 12.61 5.53 0.47 50 16.02 7.31
information
Fig. 1 describes the performance of patients on WAIS-III blocks 140 27.75 13.58 0.47 50 35.46 16.44
each of the primary measures of the BACS and the WAIS-III 143 9.50 3.23 0.74 50 12.76 4.43
arithmetic
WAIS-III digit 143 41.78 15.89 1.34 50 67.60 19.24
Table 6 symbol
Factor loadings of BACS measures in patients with schizophrenia CPT A-X 114 2.62 1.27 0.57 47 3.22 1.06
Component CPT 2 digits 113 2.06 1.01 0.89 47 2.86 0.90
CPT 4 digits 110 0.92 0.76 0.44 46 1.47 1.25
1 2 3
CPT – mean 115 1.86 0.88 0.71 48 2.51 0.92
Digit sequencing 0.357 0.522 0.186 WCST – 117 14.95 10.65 0.48 47 10.83 8.57
Symbol coding 0.653 0.444 0.104 perseverative
Tower of London 0.076 0.093 0.945 errors
Token motor task 0.551 0.064 0.443 WCST-categories 117 2.03 1.59 0.61 47 3.03 1.65
Verbal fluency 0.892 0.110 0.027 RAVLT = Rey Auditory Verbal Learning Test, COWAT = Controlled
Verbal memory 0.128 0.924 0.058 Oral Word Association Test, WAIS-III = Wechsler Adult Intelli-
Eigenvalue 2.736 0.958 0.760 gence Test, third edition, CPT = Continuous Performance Test,
Percentage of variance 45.6 16.0 12.7 WCST = Wisconsin Card Sorting Test.
292 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297

Fig. 2. Composite scores for the BACS and standard battery in schizophrenia patients standardized to healthy controls.

ences between males (0.27 F 1.05) and females tests. The table includes the group mean and
( 0.45 F 0.71) (t = 2.65, df = 48, P = 0.011) in the standard deviation for the primary measure of each
healthy control group were consistent with the signif- test and the z-scores for the patients. The least
icantly higher education and WRAT-Reading scores sensitive measures in the battery were letter –num-
for the males in this sample. ber sequencing and WCST categories, coinciding
with the two least sensitive measures in the BACS,
3.1.4. Correlations among BACS measures and factor digit sequencing (measuring verbal working memo-
analysis ry) and the Tower of London test (measuring ex-
Table 5 presents the correlations among the prima- ecutive function).
ry BACS measures for patients (below the diagonal)
and controls (above the diagonal). All but 3 of the 42 3.3. Relationship of standard battery and BACS
correlations were significant at the P < 0.05 level. The
correlations were of slightly greater magnitude in the 3.3.1. Testing duration
controls, yet the pattern of correlations between the The BACS required a mean of 34.2 min for
groups were very similar. The pattern of correlations patients (S.D. = 8.95) and 30.4 (S.D. = 3.88) min for
between each of the measures and the composite
scores, although greater in the controls, was strikingly
Table 8
similar between groups. Pearson correlations for standard battery domains and BACS
Principal components analysis with oblique rota- measures in healthy controls
tion was completed to determine the factor structure Long battery BACS measures
of the BACS. The factor loadings are presented in domains VM DS VF SC TM TL Comp
Table 6. The factor structure suggests a three-factor
Verbal memory 0.57 0.77 0.53 0.66 0.61 0.35 0.79
solution. Measures that emphasize motor speed and Attention 0.33 0.72 0.56 0.70 0.60 0.63 0.79
general cognitive functions load on the first factor; the Working memory 0.41 0.61 0.48 0.71 0.58 0.52 0.76
memory and working memory measures load on the Motor speed 0.20 0.58 0.32 0.52 0.38 0.40 0.56
second factor, and executive function loads on the Verbal fluency 0.45 0.65 0.75 0.63 0.60 0.40 0.80
third factor. Executive function 0.35 0.67 0.57 0.67 0.60 0.45 0.75
Composite score 0.48 0.63 0.65 0.78 0.68 0.55 0.90
BACS measures: VM = Verbal memory; DS = Digit sequencing;
3.2. Description of standard battery data
VF = Verbal fluency; SC = Symbol coding; TM = Token motor task;
TL = Tower of London; Comp = Composite score.
Table 7 presents the performance of schizophren- Correlations greater than and or equal to 0.48, P < 0.001; 0.40 –
ic patients and controls on the standard battery of 0.47, P < 0.01; 0.32 – 0.39, P < 0.05.
 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297 293

Table 9 completed the CPT and 117 patients completed the

Pearson correlations for standard battery domains and BACS WCST, primarily due to patient refusal or inability to
measures in schizophrenia patients
follow test instructions; both of these tests were
Long battery BACS measures
computerized.
domains VM DS VF SC TM TL Comp
Verbal memory 0.61 0.57 0.29 0.45 0.24 0.32 0.60 3.3.3. Sensitivity to deficits
Attention 0.38 0.44 0.28 0.66 0.36 0.31 0.58 Fig. 2 demonstrates the relative sensitivity of the
Working memory 0.36 0.39 0.39 0.60 0.49 0.38 0.62
composite scores from the standard battery and the
Motor speed 0.34 0.51 0.26 0.40 0.29 0.39 0.52
Verbal fluency 0.26 0.42 0.74 0.39 0.32 0.14 0.53 BACS during test session 1. BACS data from test
Executive function 0.31 0.59 0.27 0.56 0.36 0.45 0.62 session 2 were not included to minimize any effects
Composite score 0.47 0.62 0.49 0.69 0.49 0.43 0.76 of practice. The difference between the composite
BACS measures: VM = Verbal memory; DS = Digit sequencing; scores from the BACS and the standard battery was
VF = Verbal fluency; SC = Symbol coding; TM = Token motor task; not significantly different.
TL = Tower of London; Comp = Composite score. Multivariate logistic regression analyses were per-
Correlations greater than or equal to 0.29, P < 0.001; 0.24 – 0.28,
P < 0.01.
formed to determine the sensitivity of the standard
battery beyond what could be provided by the BACS.
controls on the first day of testing. Although data BACS composite scores accounted for 27.1% of the
were not collected in a standard manner for all between-group variance in diagnosis ( F = 73.61;
subjects, time estimates for a small sample of sub- df = 1,198; P < 0.001). The standard battery accounted
jects (N = 10 for each group) suggested that the for 3.3% additional variance, which was a small but
standard battery required an average of 133 min statistically significant variance component ( F = 9.46;
(S.D. = 43.1) for patients and 109 min (S.D. = 20.0) df = 1, 197; P < 0.01). However, when the standard
for controls. battery composite score was entered as the first step
in the equation, and the BACS composite scores were
3.3.2. Completion rate entered second, the BACS composite scores also
A total of 140 of the 150 patients completed each accounted for a small but statistically significant
of the subtests of the BACS and the pencil- and 1.5% of additional variance ( F = 4.31; df = 1, 197;
-paper tests of the standard battery. Only 115 patients P < 0.05).

Fig. 3. Scatterplot of BACS and standard battery composite scores for patients and controls.
294 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297
When only those subjects who received version A
at test session 1 were included in the analysis, BACS
composite scores accounted for 32.6% of the between-
group variance in diagnosis ( F = 37.38; df = 1,98;
P < 0.001). The standard battery composite score
accounted for only 0.1% additional variance, which
was not statistically significant. When the order of
entry was reversed, BACS version A composite scores
accounted for 5.3% additional variance ( F = 36.95;
df = 1,98; P < 0.01) beyond the standard battery com-
posite scores.
When only subjects who received version B at test
session 1 were included in the analysis, BACS com-
posite scores accounted for 23.0% of the between-
group variance in diagnosis ( F = 29.22; df = 1,98;
P < 0.001). The standard battery composite score
accounted for 7.6% additional variance ( F = 10.65;
df = 1,97; P < 0.01). When the order of entry was
reversed, BACS version B composite scores
accounted for only 0.1% additional variance, which
was not statistically significant.
3.3.4. Correlations between BACS measures and
standard battery constructs
The correlations between the standard battery con-
structs and the BACS measures (Test session 1 data
only) are presented in Table 8 for controls and Table 9
for patients. The correlations were slightly higher in
the controls, yet the pattern of correlation magnitude
was similar between groups. The highest correlation
in each matrix was between the standard battery and
BACS composite scores, which was 0.76 for patients
and 0.90 for controls. The individual data points from
this correlation are presented in Fig. 3.
4. Discussion
The Brief Assessment of Cognition in Schizophre-
nia, or BACS, an easily administered pen- and -paper
battery of neurocognitive tests, demonstrated high
reliability and concurrent validity with a standard
battery of tests in schizophrenia patients and healthy
controls with similar ages, racial backgrounds and
parental education. The BACS was as sensitive to the
cognitive deficits of schizophrenia as a standard
battery of neuropsychological tests that took nearly
four times as long to administer. Further, a greater
percentage of patients were able to complete the tests
in the BACS compared to the standard measures,
especially those administered with computerized
methods. The BACS offers promise as a tool for
assessing cognition repeatedly over the course of
treatment in patients with schizophrenia.
4.1. Psychometrics
The intraclass correlation coefficients calculated
from the data in this study suggested that the composite
scores derived from each subtest had very good test –
retest reliability in patients with schizophrenia and
healthy controls. These reliability coefficients were
high not only when the same version of the test was
administered on consecutive test sessions, but also
when different versions (alternate forms) of the test
were administered. The benefit of high reliability on the
BACS composite score is the increased likelihood that
change over time on this measure, such as with treat-
ment, can be interpreted as due to a nonrandom effect.
The primary measures from the subtests without
alternate versions, including tests of motor speed,
working memory, verbal fluency and information pro-
cessing speed, were highly reliable, with ICCs equal to
or greater than 0.79. The practice effects of these tests
were minimal, with none of the improvements exceed-
ing 0.25 standard deviations, despite the potential
effect of practice being maximized by testing twice
within 5 days.
The comparisons of the subtests from the alternate
forms suggested that alternate forms for the verbal
memory and Tower of London tests are necessary, but
a single version measuring verbal fluency is sufficient.
Regarding verbal memory, the lists proved to be
very similar in difficulty and highly reliable. The
practice effect across these versions was very small
compared to the practice effect using the same ver-
sions. Thus, the use of alternate forms for the verbal
memory test is necessary, and will facilitate the
assessment of changes in verbal memory abilities that
are independent from learning the words in a previous
administration.
The reliability of the verbal fluency measures was
found to be much higher when the same versions
were used on consecutive test sessions. The effect of
practice on these measures was very small, even
when the same version was administered on consec-
 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297
utive test sessions. In addition, there was variable
sensitivity of the different versions, particularly in
the domain of category fluency, with supermarket
items being the most sensitive. This pattern of results
suggests that with regard to category instances,
supermarket items will be the most reliable and
sensitive category to use for both versions of the
BACS. There was little differentiation in reliability,
practice effects, and sensitivity to group differences
among the various letter categories for the Controlled
Oral Word Association Test. It appears as though any
combination of letters is satisfactory as long as it is
consistent on consecutive testing periods.
The reliability of the Tower of London was high,
even when different versions were administered on
consecutive test sessions. The practice effects were
small in patients and medium in controls when the
same versions were used on consecutive test sessions.
However, these practice effects were diminished when
a different version was used on consecutive test
sessions. These data suggest that using an alternate
form for this test is helpful to reduce practice effects,
and recommended.
The pattern of results regarding the reliability,
practice effects, and between-group sensitivity of tests
that had alternate forms in this study suggest that the
final BACS should include alternate forms for verbal
memory and Tower of London tests only. The tests
without alternate forms—digit sequencing, symbol
coding, and the token-motor task—had minimal prac-
tice effects. Due to the reduced reliability that results
from alternate forms of verbal fluency measures, and
due to the minimal practice effects that result from
administering the same version consecutively, the
verbal fluency tests should not have alternate forms.
Based upon the slightly higher reliability, smaller
practice effect, and increased sensitivity to between-
group differences, the best verbal fluency categories
to use in the BACS are F and S for letter fluency and
supermarket items for category fluency. Thus, the
final version of the BACS that is recommended is
one in which alternate forms are used for verbal
memory and Tower of London subtests only, and a
single version of verbal fluency is used that includes
supermarket items, F-words and S-words. It should
also be noted that all of the final measures of the
BACS have distributional properties suggesting min-
imal ceiling- and floor-effects. These properties are
295
important for assessing change over time, such as in
clinical trials.
4.2. Validity
The composite scores from the BACS and the
standard battery were highly correlated in patients
and in controls, and neither battery was more sensitive
to the overall deficits found in patients with schizo-
phrenia. The magnitude of these deficits, which was
about 1.5 standard deviations below the healthy con-
trols in this study, were consistent with those reported in
meta-analyses of studies of neurocognitive impairment
in schizophrenia (Heinrichs and Zakzanis, 1998), yet
not as severe as estimated from a more selective review
of the literature (Harvey and Keefe, 1997). Studies such
as this one that pay special attention to matching groups
on age and parental education may yield differences
that are less robust than those that do not attend to
these factors (Heinrichs and Zakzanis, 1998).
The differences between patients and controls may
appear to be smaller than expected on the Tower of
London test, measuring executive functions, and the
digit sequencing test, which measures a verbal form of
working memory. While these measures may initially
appear to be less sensitive to the neurocognitive deficits
of schizophrenia, it is possible that the between-group
differences on these measures were relatively small due
to the particular cohorts tested, as the measures from
the standard battery used to assess these cognitive
domains were similarly small in their differences be-
tween patients and controls. Furthermore, the magni-
tude of these deficits are consistent with the effect sizes
reported in the meta-analysis by Heinrichs and Zakza-
nis (1998).
The factor analyses and correlations among the
BACS measures suggests that while there is a single
general factor of cognitive performance that can be
derived from the BACS raw scores, there are two
other relatively dissociable domains of performance
measured by the BACS. The pattern of correlations
among the BACS measures was very similar in
patients and controls. In both groups, each of the
individual measures demonstrated high correlations
with the composite scores. Factor analysis conducted
on the data collected only from patients suggests that
the first factor, which accounted for the largest
amount of variance, is a factor of general cognitive
296 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297
performance with an emphasis on speed and the
generation of action. The other two factors appeared
to reflect more discrete functions, with the second
factor including measures of memory and the third
including executive functions. The BACS thus ena-
bles an assessment of overall cognitive function as
well as scores on individual cognitive domains.
The definitive validation of the BACS will require
further study. First, the comparisons of the final word
lists for the verbal memory test had sample sizes that
were reduced, which resulted in less statistical power
available for these analyses. However, the final lists
appear to have remarkable similarity in their sensitiv-
ity to group differences. This similarity was also
found in a separate group of controls that was not a
part of this study sample.
Second, we chose to test subjects twice within 1
week to minimize the impact of changes in clinical
state and medication, reduce drop out, and maximize
practice effects. However, the assessment of a treat-
ment effect in research studies or clinical purposes is
likely to be measured with longer time between
assessments. Thus, the reliability of these measures
in actual practice may be lower and the practice effects
of the tests in the BACS are likely to be smaller.
Third, one of the drawbacks of a focus on com-
posite scores in the evaluation of cognition in schizo-
phrenia is that isolated yet important cognitive effects
may be missed. It is possible that a new medication
may improve an important aspect of cognition that is
not assessed by the BACS. However, the importance
of general cognitive effects can be seen in the relative
size of the correlations between functional outcome
and different aspects of cognition. The Pearson corre-
lations between outcome and individual aspects of
cognition such as memory, attention, and executive
functions are relatively small, ranging between 0.20
and 0.34 (Green, 2002) The few studies that have
enabled the determination of the correlation of out-
come with overall summary scores of cognition have
demonstrated far larger correlations, exceeding 0.60
(Green, 2002). These data suggest that the determi-
nation of cognition with summary data, such as the
BACS composite scores reported here, may be the
most powerful indicator of functional outcome, and
has the most promise as a target for cognitive en-
hancement that will produce real-world changes in
patients lives outside the laboratory.
Finally, a complete validation of the BACS will
require assessments in various populations of patients
with schizophrenia, such as first episode populations,
treatment refractory schizophrenia patients, and geri-
atric patients. It will also be important to determine
whether BACS scores can predict changes in func-
tional outcome, such as activities of daily life, and
whether the BACS is sensitive to cognitive changes
during clinical trials. Studies are underway to deter-
mine the validity of the BACS in these areas of
inquiry.
In sum, the BACS assesses the major constructs of
cognition that have been found to be most impaired
and most strongly correlated with outcome in patients
with schizophrenia. The BACS takes less than 35 min
to complete in patients with schizophrenia, yields a
high completion rate in these patients, and has high
test – retest reliability over a period of days. It is as
sensitive to cognitive impairment in patients with
schizophrenia as a standard battery of tests that
required over 2 h to complete. While the sensitivity
of the BACS to change over time can only be
determined through longitudinal treatment studies,
10 of which are currently underway, its psychometric
properties make it a promising tool for assessing
cognition repeatedly in patients with schizophrenia.
Acknowledgements
This work was supported by a grant from Eli Lilly.
Mark Appelbaum provided statistical consultation.
Neurocognitive data were collected, in part, by Adam
Vaughn, Susan Shortel, Matthew Dukes, Trina Walker
and Joseph Kang. Healthy controls at Mount Sinai
were assessed by Adam Brickman and Julie Kim.
References
Cogtest plc. Cogtest(tm): Computerised Cognitive Battery for Clin-
ical Trials. (2002). Retrieved November 8, 2002, from http://
www.cogtest.com.
Cornblatt, B.A., Keilp, J.G., 1994. Impaired attention, genetics,
and the pathophysiology of schizophrenia. Schizophr. Bull.
20, 31 – 46.
Evins, A.E., Fitzgerald, S.M., Wine, L., Rosselli, R., Goff, D.C.,
2000. Placebo-controlled trial of glycine added to clozapine in
schizophrenia. Am. J. Psychiatry 157 (5), 826 – 828.
Folstein, M.F., Folstein, S.E., McHugh, P.R., 1975. ‘‘Mini-mental
 R.S.E. Keefe et al. / Schizophrenia Research 68 (2004) 283–297
state’’. A practical method for grading the cognitive state of
patients for the clinician. J. Psychiatr. Res. 12 (3), 189 – 198.
Friedman, J.I., Adler, D.N., Temporini, H.D., Kemether, E., Harvey,
P.D., White, L., Parrella, M., Davis, K.L., 2001. Guanfacine
treatment of cognitive impairment in schizophrenia. Neuropsy-
chopharmacology 25 (3), 402 – 409.
Friedman, J.I., Adler, D.N., Evelyn, H., Harvey, P.D., Brenner, G.,
Temporini, H., White, L., Parrella, M., Davis, K.L., 2002. A
double-blind placebo controlled trial of donepezil adjunctive
treatment to risperidone for the cognitive impairment of schiz-
ophrenia. Biol. Psychiatry 51 (5), 349 – 357.
Gardner Jr., R., Oliver-Munoz, S., Fisher, L., Empting, L. 1981.
Mattis Dementia Rating Scale: internal reliability study using a
diffusely impaired population. J. Clin. Neuropsychol. 3 (3),
271 – 275.
Geffen, G.M., Butterworth, P., Geffen, L.B., 1994. Test – retest reli-
ability of a new form of the Auditory Verbal Learning Test
(AVLT). Arch. Clin. Neuropsychol. 9, 303 – 316.
Gold, J.M., Carpenter, C., Randolph, C., Goldberg, T.E., Weinberg-
er, D.R., 1997. Auditory working memory and Wisconsin Card
Sorting Test performance in schizophrenia. Arch. Gen. Psychia-
try 54 (2), 159 – 165.
Gold, J.M., Queern, C., Iannone, V.N., Buchanan, R.W., 1999.
Repeatable battery for the assessment of neuropsychological
status as a screening test in schizophrenia: I. Sensitivity, reli-
ability, and validity. Am. J. Psychiatry 156 (12), 1944 – 1950.
Green, M.F., 1996. What are the functional consequences of neuro-
cognitive deficits in schizophrenia? Am. J. Psychiatry 153,
321 – 330.
Green, M.F., 2002. Mediators between neurocognitive deficits and
community outcome in schizophrenia. J. Adv. Schizophr. Brain
Res. 4, 3 – 7.
Green, M.F., Kern, R.S., Braff, D.L., Mintz, J., 2000. Neurocogni-
tive deficits and functional outcome in schizophrenia: are we
measuring the ‘‘right stuff’’. Schizophr. Bull. 26 (1), 119 – 136.
Harvey, P.D., Keefe, R.S.E., 1997. Cognitive impairment in schiz-
ophrenia and implications atypical neuroleptic treatment. CNS
Spectr. 2, 1 – 11.
Harvey, P.D., Keefe, R.S.E., 2001. Studies of cognitive change with
treatment in schizophrenia. Am. J. Psychiatry 158, 176 – 184.
Heaton, R.K., 1993. Wisconsin Card Sorting Test Manual. Psycho-
logical Assessment Resources, Odessa, FL.
Heinrichs, R.W., Zakzanis, K.K., 1998. Neurocognitive deficit in
schizophrenia: a quantitative review of the evidence. Neuropsy-
chology 12 (3), 426 – 445.
Heresco-Levy, U., Silipo, G., Javitt, D.C., 1996. Glycinergic aug-
mentation of NMDA receptor-mediated neurotransmission in
the treatment of schizophrenia. Psychopharmacol. Bull. 32 (4),
731 – 740.
297
Hobart, M.P., Goldberg, R., Bartko, J.J., Gold, J.M., 1999. Repeat-
able battery for the assessment of neuropsychological status as a
screening test in schizophrenia: II. Convergent/discriminant val-
idity and diagnostic group comparisons. Am. J. Psychiatry 156
(12), 1951 – 1957.
Hunter, R., Cameron, S., Perks, S., Wesnes, K., 1997. The cognitive
profile of unmedicated schizophrenic patients in relation to con-
trols. J. Psychopharmacol. 11, A74 (Suppl.).
Keefe, R.S.E., 1995. The contribution of neuropsychology to psy-
chiatry. Am. J. Psychiatry 152, 6 – 15.
Keefe, R.S.E., Lees-Roitman, S., Dupre, R.L., 1997. Performance
of patients with schizophrenia on a pen and paper visuospatial
working memory task with short delay. Schizophr. Res. 26 (1),
9 – 14.
Lezak, M., 1995. Neuropsychological Assessment, 3rd ed. Oxford
Univ. Press, New York, NY.
Randolph, C., 1998. Repeatable Battery for the Assessment of
Neuropsychological Status. Psychological, San Antonio.
Reitan, R.M., 1979. Manual for the Administration of Neuropsy-
choligcal Test Batteries for Adults and Children. Ralph M. Re-
itan, Tucson, AZ.
Robbins, T.W., James, M., Owen, A.M., Sahakian, B.J., McInnes,
L., Rabbitt, P.M., 1996. A neural systems approach to the cog-
nitive psychology of aging: studies with CANTAB on a large
sample of the normal elderly population. In: Rabbitt, P.M. (Ed.),
Methodology of Frontal and Executive Function. Lawrence Erl-
baum Associates, Hove, pp. 215 – 238.
Rosen, W.G., Mohs, R.C., Davis, K.L., 1984. A new rating scale for
Alzheimer’s disease. Am. J. Psychiatry 141 (11), 1356 – 1364.
Ruff, R.M., Parker, S.B., 1993. Gender- and age-specific changes in
motor speed and eye – hand coordination in adults: normative
values for the Finger Tapping and Grooved Pegboard Tests.
Percept. Mot. Skills 76, 1219 – 1230.
Saykin, A.J., Gur, R.C., Gur, R.E., Mozley, P.D., Mozley, L.H.,
Resnick, S.M., Kester, D.B., Stafiniak, P., 1991. Neuropsycho-
logical function in schizophrenia. Selective impairment in mem-
ory and learning. Arch. Gen. Psychiatry 48 (7), 618 – 624.
Wechsler, D., 1997a. Wechsler Adult Intelligence Scale, 3rd ed.
Psychological, San Antonio, TX. WAIS-III.
Wechsler, D., 1997b. Wechsler Memory Scale, 3rd ed. The Psycho-
logical, San Antonio, TX. WMS-III.
Wilk, C.M., Gold, J.M., Bartko, J.J., Dickerson, F., Fenton, W.S.,
Knable, M., Randolph, C., Buchanan, R.W., 2002. Test – retest
stability of the repeatable battery for the assessment of neuro-
psychological status in schizophrenia. Am. J. Psychiatry 59 (5),
838 – 844.
Wilkinson, G.S., 1993. The Wide Range Achievement Test 3 Ad-
ministration Manual. Wide Range, Wilmington, DE.