Behavioural Processes 54 (2001) 111– 126

www.elsevier.com/locate/behavproc

The critical dimensions of the response-reinforcer

contingency
B.A. Williams *
Department of Psychology, Uni6ersity of California, San Diego, CA 92093 -0109, USA

Received 28 June 2000; received in revised form 18 September 2000; accepted 5 January 2001

Abstract

Two major dimensions of any contingency of reinforcement are the temporal relation between a response and its
reinforcer, and the relative frequency of the reinforcer given the response versus when the response has not occurred.
Previous data demonstrate that time, per se, is not sufficient to explain the effects of delay-of-reinforcement
procedures; needed in addition is some account of the events occurring in the delay interval. Moreover, the effects of
the same absolute time values vary greatly across situations, such that any notion of a standard delay-of-reinforce-
ment gradient is simplistic. The effects of reinforcers occurring in the absence of a response depend critically upon the
stimulus conditions paired with those reinforcers, in much the same manner as has been shown with Pavlovian
contingency effects. However, it is unclear whether the underlying basis of such effects is response competition or
changes in the calculus of causation. © 2001 Elsevier Science B.V. All rights reserved.

Keywords: Delay of reinforcement; Reinforcement contingency; Contingency

1. Introduction behavioral situations — which is the function a

The principle of reinforcement is psychology’s
most important concept, at least as assessed by
the potency and diversity of its real-world applica-
tions. Yet at the same time it is not clear that the
principle of reinforcement qualifies as a true law
of behavior. Its basic formula, the three term
contingency, is only a framework for analyzing
behavior, and must be fleshed out in details before
it can make meaningful predictions in specific
* Corresponding author. Tel.: + 1-858-5343938; fax: + 1-
858-5347190.
E-mail address: bawilliams@ucsd.edu (B.A. Williams).
true law of behavior should serve (see Timberlake,
1988 for a more extended discussion of this issue
from a very different perspective from that pre-
sented here).
2. Role of the response-reinforcer temporal
relation
The simple statement that responding is
strengthened when followed by positive events
(reward) is clearly too broad. It is unlikely that a
food pellet delivered today because a rat pressed a
bar last week will have the effect of making the

0376-6357/01/$ - see front matter © 2001 Elsevier Science B.V. All rights reserved.
PII: S 0 3 7 6 - 6 3 5 7 ( 0 1 ) 0 0 1 5 3 - X
112 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
rat press the bar more in the future. There has
always been the assumption that the temporal
relation between the response and reinforcer is
important, although how and why it is important
continues to be the subject of debate.
Skinner believed that the temporal relation was
the defining feature of reinforcement in that tem-
poral contiguity (or at least an approximation to
contiguity) was assumed to be both necessary and
sufficient: necessary in the sense that whenever a
reinforcer not temporally contiguous did result in
response strengthening Skinner then looked for
events intervening in the delay-of-reinforcement
interval which served as conditioned reinforcers,
which themselves were immediately contingent on
the response. For example, in Science and Human
Beha6ior (p. 76) he writes: ‘‘Although it is charac-
teristic of human behavior that primary rein-
forcers may be effective after a long delay, this is
presumably only because intervening events be-
come conditioned reinforcers’’.
2.1. Superstitious conditioning
The evidence Skinner provided for the suffi-
ciency of temporal contiguity was his demonstra-
tion of superstitious conditioning. The procedure
of Skinner (1948) was simply to present food
every 15 s regardless of what the bird was doing.
Six of eight birds exhibited responses sufficiently
clearly defined that two observers could agree
perfectly in counting instances. One turned coun-
ter-clockwise about the cage. Another repeatedly
thrust its head into one of the upper corners of
the cage. A third developed a tossing response, as
if placing its head beneath an invisible bar and
lifting it. Two birds developed a pendulum mo-
tion of the head and body, in which the head was
extended forward and swung from right to left
with a sharp movement followed by a somewhat
slower return. Another bird made incomplete
pecking or brushing movements directed toward
but not touching the floor.
While Skinner assumed that adventitious re-
sponse-reinforcer pairings were the basis of the
idiosyncratic behaviors that he observed, others
have offered alternative interpretations. Both
Staddon and Simmelhag (1971) and Timberlake
and Lucas (1985) have argued that the behaviors
seen in the superstition procedure are due to
Pavlovian contingencies whereby the discrimina-
tion of the reinforcer’s temporal periodicity
evokes responses that are part of the bird’s natu-
ral repertoire of food-related behaviors. The most
widely cited rendition of this perspective is the
distinction between interim and terminal behav-
iors proposed by Staddon and Simmelhag (1971),
who argued that early in the temporal interval
elicited behaviors other than the keypeck were
most probable, and that the nature of that interim
behavior changed systematically as the temporal
interval progressed, with the terminal behavior (in
this case keypecking) becoming most frequent at
the time of maximum probability of food delivery.
(But see Reid et al., 1993). It is important to
recognize that neither of these investigations was
an exact replication of Skinner’s original proce-
dure, in part perhaps because discerning the exact
details of the procedure is impossible from Skin-
ner’s original report. But it does appear there was
one very fundamental difference. Skinner seems to
have trained his subjects for a single session, of
indeterminant duration, whereas the subsequent
two reports used multiple sessions. In neither
report is there an informative analysis of the first
session of training. Staddon and Simmelhag did
report the data for one subject that included
separate plotting of the behavior on the first
session, but that subject apparently kept its head
in the food magazine for most of the session and
is not helpful. The only other behavior reported
for it was 1/4 turns, which is consistent with
Skinner’s own account. This does not mean that
elicited behaviors are not important because there
is no reason to believe that the adventitious re-
sponse-reinforcer relation is the only factor oper-
ating in the superstition procedure. There seems
to be no doubt that pigeons eventually discrimi-
nate the periodicity of the food delivery and that
the time of maximum food probability does evoke
species-characteristic behavior. This in no way
challenges the validity of the superstition concept,
but shows only that the effects of adventitious
response-reinforcer contiguities can be overridden
by the Pavlovian contingency.
 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
The results of Neuringer (1970) provide an
empirical reason for taking the concept of super-
stitious conditioning seriously. Neuringer pre-
sented two groups of pigeons a variable-time (VT)
30-s schedule for 20 sessions of training. The two
groups differed only with respect to the experi-
mental group first receiving three reinforced
keypecks. These subjects maintained their re-
sponding at low levels of 2– 5 responses/min
throughout the entire duration of training,
whereas subjects without the history of reinforced
keypecking emitted only a minimal number of
responses. The most plausible interpretation of
Neuringer’s results is that the group with the
initially reinforced keypecking had their level of
keypecking raised to the extent that keypecking
was the dominant response in the situation and
therefore was the behavior most likely to be
picked up adventitiously by the response-indepen-
dent reinforcement.
2.2. The gradient of delayed reinforcement
If Skinner was correct in his statement that the
ability of a reinforcer to strengthen a response
depends only on the response-reinforcer temporal
relation, it should be possible to establish a quan-
titative function describing the delay-of-reinforce-
ment gradient. Various people have attempted
this over the years (e.g. Hull, 1943; Logan, 1960;
Mowrer, 1960). By far the most sophisticated
attempt in terms of supplying evidence for a
quantitative description of delayed reinforcement
effects has been Mazur’s work using his titration
procedure (see Mazur, 1997, for a review), the
result of which has been the well known hyper-
bolic equation. Note that there are important
implications of the equation being hyperbolic
rather than exponential as most early theorists
assumed, in that it allows for the phenomenon of
preference reversal, which underlies the analysis
of Ainslee, Rachlin and others of self-control
procedures (see Mazur, 1998).
Unfortunately, it is no longer clear how
Mazur’s equation should be interpreted. Mazur
(1997) argues that it is a description of the value
of the conditioned reinforcers intervening in the
delay-of-reinforcement interval. What then would
113
the equation be if there were no conditioned
reinforcers present in the delay interval? This is an
intractable problem experimentally because the
only way the actual time values between a re-
sponse and reinforcer can be controlled in a pre-
cise way is to preclude the response from
occurring while the delay of reinforcement inter-
val is in operation. There are two ways one might
accomplish this. One is to remove the response
opportunity, but this necessarily introduces a
stimulus change that might assume conditioned
reinforcement properties. The second is to use a
dro contingency, in which the delay is reset after
each response, but this means that the schedule of
reinforcement is changing, and also means that
one is likely to be reinforcing competing behavior.
There is a third alternative — to use an
unsignaled delay of reinforcement procedure. This
procedure has the obvious difficulty that re-
sponses are allowed during the delay interval, so
that the actual time values involved are indetermi-
nant, although they of course can be measured.
The second problem is the same as the dro proce-
dure, in that competing behaviors may also occur
during unsignaled delays. Nevertheless, it now
seems that the use of the unsignaled delay proce-
dure can be instructive.
Williams (1976) (Expt. 2) trained pigeons on a
VI 2 min schedule of immediate reinforcement
and then a 5-s unsignaled delay contingency was
added to the end of the 2-min schedule. The delay
contingency entailed that the first peck after a VI
interval had elapsed started a 5-s timer where
food was delivered independently of the pigeon’s
behavior at the timer’s offset. No stimulus change
occurred as a function of the timer’s onset (techni-
cally a tandem VI-2 min FT 5-s schedule). For
each subject receiving the delay contingency a
second subject was yoked, such that it received
response-independent reinforcement whenever a
delayed reinforcer occurred for the subject trained
with the delay contingency. Midway through
training the roles of the two subjects was reversed
so that the yoked subject received the delay con-
tingency and vice versa.
Fig. 1 shows the results in terms of normalized
response rate relative to the baseline rate when
immediate reinforcement was delivered. A great
114 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126

deal of variability is evident for both the delay With continued training, however, the delay sub-
and yoked subjects during the first ten or so jects, with one exception, responded at a higher
sessions so it is difficult to see any consistent rate than did their paired yoked subjects, indicat-
difference between the delay and yoked subjects. ing that the delayed reinforcement contingency

Fig. 1. Response rate maintained by a 5-s unsignaled delay, along with that of yoked subjects receiving response-independent
reinforcement. The contingency for individual subjects within a pair was switched midway through training. (Reprinted from
Williams, 1976, Experiment 2, Journal of the Experimental Analysis of Behavior.)
 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
did exert some small amount of control over
responding. But the most salient aspect of the
figure is the level of responding eventually reached
under the delay contingency. For four of the eight
subjects, it is less than 10% of the baseline level,
and these rates were sufficiently low that the
subjects lost a significant fraction of the available
reinforcers. The remaining four subjects had re-
sponse rates between 20 and 40% of their baseline
rates and these were highly variable across
sessions.
The apparent inference to be drawn from Fig. 1
is that delay of reinforcement is a very powerful
variable, and even short delays can wipe out an
enormous amount of behavior. This inference
concurs with the conclusion of the classic delay of
reinforcement experiment of Grice (1948) who
used a discrimination procedure without differen-
tial cues during the delay interval, and reported
almost no learning, with rats as subjects, when
delays reached approximately 5 s.
But there are complications to be resolved be-
fore this inference can be accepted. First is the
fact that the delay gradient is nonmonotonic.
When very short delays are used in the unsignaled
delay procedure, in the range of 0.5– 2 s, ex-
tremely high response rates are produced with the
short delays. The apparent reason for this is that
pigeons respond in bursts, so that the first re-
sponse of the burst initiates the delay, while the
end of the burst corresponds with the end of the
delay and thus is immediately reinforced. When
delays are gradually increased, these response
bursts may increase in length, producing very high
response rates. This implies that response rate is a
problematic measure of response strength, in that
the response unit may change systematically with
the delay value (Arbuckle and Lattal, 1988).
A second interpretative problem for the
unsignaled delay procedure is that much of the
precipitous fall-off in response rate seen in Fig. 1
could be due to the development of competing
behavior. The unsignaled delay procedure allows
many forms of behavior other than keypecking to
be present at the time of reinforcer delivery, so it
is possible that these competed with keypecking.
If such competing behavior could be eliminated, it
seems plausible that the delay gradient might have
been not nearly so steep.
115
The third and most important complication for
interpreting the unsignaled delayed reinforcement
procedure is that it is now known that pigeons
can acquire keypecking in an unsignaled delay
procedure, de novo, with delays up to 30-s. This
was shown in an important paper by Lattal and
Gleeson (1990). Moreover, acquisition occurred
even when a dro was used, which ensured that the
scheduled delay was the minimum time between
the response and reinforcer. Similar experiments
have been conducted in my laboratory with rats,
without the dro, and have found acquisition with
most rats with 30-s unsignaled delays, but inter-
estingly, complete failure of acquisition when us-
ing 60-s delays.
2.3. Associati6e competition as a second 6ariable
The question is how to reconcile the great
sensitivity to delays seen in Fig. 1 with the acqui-
sition data showing that pigeons and rats can
learn with delays that are an order of magnitude
greater. One clue to this apparent conflict is a
feature of the Lattal and Gleason paper that
differs from most other experiments. Their sub-
jects were placed in the chamber and kept there
until they responded. In contrast other experi-
ments have used fixed session durations which
were independent of when the subject starts
responding.
Why should long waits in the chamber be an
important variable? One possible answer comes
from Dickinson et al. (1992), who used rats with
60-s delays. Like my own results, the rats failed to
acquire barpressing when the procedure was be-
gun with the onset of the session. But in a second
condition the subjects waited 30 min in the exper-
imental chamber before the bar was inserted into
the chamber, at which time the 60-s delay contin-
gency was in effect. Here the subjects did acquire
responding. Why might this be? What likely hap-
pened is events other than barpressing competed
as predictors of the food delivery. As described by
Killeen and Bizo (1998), rats engage in a variety
of behavior when first exposed to the experimen-
tal chamber, including circling, sniffing the walls,
rearing in the corners, etc, and these activities
gradually decrease in frequency. Each of these
116 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
activities, and their accompanying stimuli, poten-
tially could be associated with the food delivery if
a lever press were to occur at the beginning of the
session. But with the 30-min wait in the chamber
before the lever was inserted, attention to these
potential competitors was essentially extinguished,
much in the same way as latent inhibition has
been shown to occur to pre-exposed CSs in Pavlo-
vian conditioning procedures. Then when the first
food pellet produced by a lever press occurred,
only the memory of the bar press was a salient
event because its potential competitors had lost
their salience due to their prior exposure.
The foregoing analysis implies that it isn’t sim-
ply the temporal relation between the response
and reinforcer that is critical. One must also con-
sider the how the response-reinforcer relationship
compares to the relation between potential asso-
ciative competitors and the reinforcer. The possi-
bility that associative competition occurs with
respect to operant conditioning in the same man-
ner as it occurs for overshadowing and blocking
effects in Pavlovian conditioning (e.g. Kamin,
1988) was first investigated by Williams (1975).
Pigeons were presented a discrete-trial procedure
in which a keylight was on for 5 s, and then food
was delivered 10 s after the offset of the keylight
if at least one peck had occurred during the 5-s
keylight. In the control condition the stimulus
condition during the delay-of-reinforcement inter-
val was the same as during the intertrial interval.
In the ‘blocking’ condition the last 3– 5 s of the
delay were filled by a second keylight, or, depend-
ing on the experiment some other kind of stimu-
lus. The results were that the second stimulus in
the latter portion of the delay greatly reduced the
level of responding maintained by the delay con-
tingency. I interpreted the result as showing that
stimulus-reinforcer associations could block re-
sponse-reinforcer associations.
The procedure of Williams (1975) was contested
as a method of demonstrating associative compe-
tition in operant conditioning because of its use of
keypecking by pigeons as the putative operant
response. Because keypecking has been shown to
be controlled by Pavlovian contingencies in au-
toshaping experiments, the possibility thus arises
that the pecking behavior studied by Williams
(1975) was also under the control of Pavlovian
rather than operant contingencies. This criticism
has been addressed in several subsequent studies
(Williams, 1978, 1982), most recently by Williams
(1999). This most recent study is also most ger-
mane to the present discussion because it studied
response acquisition de novo, in a manner similar
to Lattal and Gleeson (1990). In the first experi-
ment naive rats were placed in the chamber with
4– 5 food pellets in the magazine with a 30-s delay
of reinforcement contingency in effect. A barpress
started a timer, without any feedback that the
delay was operating, and food was delivered auto-
matically at the end of the 30-s delay. When the
timer was running, further responses had no ef-
fect, so the maximum rate of reinforcement that
could be obtained was 2/min or 120/h. The usual
result of this procedure is that 75–80% of the rats
learn to barpress on a consistent basis, although
with highly variable response rates. There were
two other conditions in the experiment, both in-
volving presentations of a 5-s houselight. In the
‘marking’ condition (see Lieberman et al., 1979,
for the development of the concept of marking),
this houselight occurred immediately after the bar
press that started the-delay of-reinforcement inter-
val. For the ‘blocking’ condition, the houselight
occurred during the last 5 s of the delay.
Fig. 2 shows that the different placements of
the houselight had opposite effects. For the mark-
ing condition, it substantially facilitated the learn-
ing; for the blocking condition, it completely
prevented acquisition from occurring. Note that
for all three conditions, the response-reinforcer
relation was identical.
In a second experiment published in the same
paper, the blocking effect was studied in a choice
procedure. Here each of two responses started its
own 30-s timer, which ended in food regardless of
what occurred during the delay interval. For one
response the delay was completely unsignaled; for
the other response, the last 5 s of its delay had a
5-s houselight present. All subjects strongly pre-
ferred the response alternative without the signal
during its delay.
Both of the effects seen in Fig. 2 are important
for understanding delay-of-reinforcement contin-
gencies. The marking effect indicates that one
 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126 117

Fig. 2. Acquisition of barpressing with a 30-s unsignaled delay of reinforcement. Different groups of subjects received different signal
conditions during the delay intervals, either no signals (No Sig), a houselight at the start of the delay (Marking), or a houselight at
the end of the delay (blocking). (Reprinted from Williams, 1999, Experiment 1, Psychonomic Bulletin & Review.)

needs to take into consideration the memory of
the response upon which the reinforcer is contin-
gent. When that response is highlighted by some
distinct event, that response is more strongly af-
fected by the delayed reinforcer. A general impli-
cation of the marking effect is that a critical
consideration is the content of the animal’s mem-
ory buffer at the time the reinforcer is delivered.
This implies that any of a variety of procedures
should be potentially valuable for increasing the
effectiveness of delayed reinforcement. In Pavlo-
vian conditioning, Peter Holland and his collabo-
rators have demonstrated that evoking the
memory of a CS can substitute for the CS in
several situations, ranging from conditional dis-
criminations to extinction. For example, Holland
and Forbes (1982) conditioned a taste aversion to
a specific flavor, then paired a tone with that
flavor, and then presented the tone repeatedly
alone. The result was that the conditioned aver-
sion to the flavor was attenuated. What this sug-
gests for operant reinforcement is that conjuring
up the memory of a response just prior to rein-
forcer delivery might cause that response to be
strengthened.
Of primary interest for the present discussion is
the effect of the houselight signal at the end of the
delay interval, which appears to have blocked the
rat’s association between its lever press and the
food delivery. This blocking effect demonstrates
that ‘predictiveness’ is a factor in operant condi-
tioning apparently in the same manner as it is in
Pavlovian conditioning. Thus, a given response-
reinforcer temporal relation can have very differ-
ent effects depending on whether the reinforcer
occurring at the delay-of reinforcement-interval is
already predicted by events occurring during the
delay.
At a more general level, the potency of both
marking and blocking says any quantitative
118 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
model of delay of reinforcement will be problem-
atic. Only when those effects are absent, or at
least equal for the different responses being com-
pared, can a general equation be applied, and
those restrictions suggest that any such equation
will be highly situation specific. For example,
based on the results of Dickinson et al. (1992), the
equation for delayed reinforcement effects should
be substantially different if the subject is habitu-
ated to the experimental chamber prior to the
onset of the experimental session (also see Dickin-
son et al., 1996).
2.4. Self-reinforcement as an example
At a yet more general level, the concept of
predictiveness may shed considerable light on var-
ious enigmatic concepts in behavior analysis. One
that generated considerable attention 15– 20 years
ago was ‘self-reinforcement’: can one supply one’s
own reinforcement contingencies, and have those
contingencies meaningfully affect behavior? For
example, can I better sustain my writing behavior
by consuming M&M’s on a variable interval
schedule, given that I am working at the time the
timer’s offset is signaled. In Science and Human
Beha6ior (1953) Skinner appears to be of two
minds about whether self-reinforcement should be
an effective procedure. On the one hand he recog-
nizes that any notion of reinforcement based
solely on response-reinforcer temporal contiguity
implies that self-reinforcement would work. But
he also recognizes that the concept seems implau-
sible: (p. 238: ‘‘The ultimate question is whether
the consequence has any strengthening effect
upon the behavior which precedes it. Is the indi-
vidual more likely to do a similar piece of work in
the future? It would not be surprising if he were
not, although one must agree that he has ar-
ranged a sequence of events in which certain
behavior has been followed by a reinforcing
event’’).
Once it is recognized that predictiveness is a
critical part of the response-reinforcer contin-
gency, failure of self-reinforcement should not be
surprising. In order for a reinforcer to be effec-
tive, the response must not be redundant with
other events in the same situation that predict
when a reinforcer will be delivered. But it is the
compliance with the rules of self-reinforcement,
rather than the response of writing that deter-
mines when the M&M’s will be consumed. Be-
cause writing is less predictive than the behavior
of compliance, writing therefore should not be
strengthened.
3. The role of alternative reinforcement
In addition to the temporal relation between
the response and reinforcer, and the complexities
posed by the concepts of predictiveness and mem-
ory as modulators of the effects of the temporal
relation, there is a second dimension of the con-
tingency of reinforcement that must be consid-
ered. It is not only the response-reinforcer
pairings that are important, but also the occur-
rence of the reinforcer when the response has not
occurred. That is to say, the response rate is
governed by the relative rate of reinforcement
contingent on a response, not simply the rein-
forcement for that response considered in
isolation.
The concept of relative rate of reinforcement
has played a huge role in theories of operant
behavior over the past 40 years, both for multiple
and concurrent schedules, and also for behavior
maintained by simple schedules when additional
reinforcement is presented alongside the response-
contingent reinforcement. Much of this theorizing
was inspired by the relative law of effect of Herrn-
stein (1970), which was intended to apply to all
schedule situations:
B1 = kR1/(R1 +mR2 + Ro) (1)
B1 refers to the behavior being counted, R1 to its
reinforcement rate, R2 to the reinforcement rate
from other identifiable behavior (B2), and Ro to
the hypothetical reinforcement rate of other be-
havior not directly observed.
It is important to recognize that Herrnstein
himself came to believe this formula did not actu-
ally reflect the conditioning process at its most
basic level, which was instead the process of me-
lioration (see Williams, 1988, for a discussion).
This change is critically important because Eq. (1)
 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
suggests that a molar definition of the reinforce-
ment contingency is appropriate, whereby there is
some comparison of the rate of response-contin-
gent reinforcement to the rates of reinforcement
from other sources in the same situation. Meliora-
tion theory, on the other hand, reduces to the
very simple idea that choice is governed by which
response alternative has the highest recent local
rate of reinforcement, which, in the case of dis-
crete responses, is equivalent to which response
alternative has had the higher recent probability
of reinforcement. Probability of reinforcement is
of course the variable that Skinner invoked as the
fundamental determinant of the strength of
behavior.
Contrary to melioration theory, there are data
that appear to support the concept of relative rate
of reinforcement as a basic determinant of how
reinforcement operates to strengthen behavior.
An illustrative experiment (also see Catania, 1963)
was provided by Rachlin and Baum (1969), who
presented pigeons two response keys, one with a
constant VI schedule, the second with a second VI
of similar value but with a signal contingency,
such that the second keylight was illuminated only
when a reinforcer had been scheduled by its VI
timer. The duration of reinforcement associated
with the signaled VI was then varied. Response
rate maintained by the unsignaled VI varied in-
versely as a function of the duration of the sig-
naled reinforcer. Note that this relation held
despite very little time being devoted to the sig-
naled VI key, because a single peck was all that
was required to obtained its reinforcer. Moreover,
the time allocation presumably should not vary
with the duration of the food for that key. In a
related paper Rachlin and Baum (1972) reported
that the reduction in responding to a constant VI
was the same regardless of whether additional
reinforcement was available from working on a
second key on a VI schedule, or whether the same
amount of food was presented on a VT schedule
of free food.
These aged data are important because meliora-
tion theory has no account for why response rate
for the unchanged schedule should depend on the
rate of reinforcement from alternative sources
without concomitant changes in the amount of
119
response competition engendered by those other
sources. For example, in the study of Rachlin and
Baum (1969), it should be impossible for the
subject to equate the local rates of reinforcement
across the two choice alternatives, because the
local rate in the absence of the signal was zero,
while that when the signal was present was ex-
tremely high. Why then should response rate
maintained by the unchanged VI be a regular
function of the amount of reinforcement associ-
ated with the signaled VI? Such a finding suggests
strongly that relative rate of reinforcement is the
fundamental variable controlling response rate,
and that the effects of relative rate of reinforce-
ment cannot be reduced to the probabilities of
reinforcement for the individual response alterna-
tives.
3.1. Similarities between Pa6lo6ian and operant
conditioning
The concept of relative rate of reinforcement as
a controlling variable in operant conditioning is
similar to the concept of contingency as employed
in Pavlovian conditioning. Rescorla (1967, 1968)
reported that the strength of fear conditioning
depended not on the probability of shock during
the CS, but instead upon the ratio of the CS
probability of shock to the probability of shock in
the absence of the CS. This suggested that contin-
gency was a primitive variable with a fundamental
status similar to the CS-US temporal relationship.
It is interesting to note that a parallel argument
was being advanced at approximately the same
time in the operant literature (e.g. Baum, 1973),
where various theorists argued that animals are
sensitive to response-reinforcer correlations, or
had their behavior governed by schedule-feedback
functions.
In the operant literature the contingency/corre-
lation view of how reinforcement makes contact
with behavior is still seriously maintained, at least
in the sense that no one has advanced a theoreti-
cal account showing how the effects of relative
rate of reinforcement can be explained in terms of
more basic principles. In Pavlovian conditioning,
by comparison, very soon after Rescorla demon-
strated the effect of contingency at an empirical
120 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
level he went on to explain the effect of contin-
gency in reductionistic terms (for a review see
Papini and Bitterman, 1990). Contingency effects
occurred, said Rescorla, because of ‘context
blocking’. In the random-control procedure in
which the probability of the US was the same
during the CS and during its absence, US presen-
tations in the absence of the CS were assumed to
cause the background cues to become associated
with the US. Because these cues were also present
when the CS was paired with the US, they
blocked the CS-US association.
The critical test of the context blocking account
of contingency effects has been to observe
whether signalling the reinforcers otherwise paired
only with the background cues would allow con-
ditioning to occur in the random control proce-
dure. The rationale is that excitatory conditioning
to the background cues should be blocked by the
signal, because the signal has greater predictive
validity than the background. That is, the signal
always is paired with the US presentations, while
the background only occasionally was paired.
Then, because the background cues had no excita-
tory value, blocking of conditioning to the CS in
the random control procedure should not occur.
The actual results of signalling the background
reinforcers have been conflicting. Jenkins et al.
(1981) were the first to test the effects of signalling
the background reinforcers and their initial find-
ings were that signalling the background rein-
forcers had no effect: conditioning to the CS
failed to occur regardless of where the back-
ground reinforcers were paired with the signal.
However, subsequent studies by Durlach and Res-
corla (e.g. Durlach, 1983) produced the opposite
findings, as signalling the background reinforcers
did cause conditioning to the CS to emerge, al-
though much more slowly than when no rein-
forcers were presented during the ITI.
The probable explanation of the conflict be-
tween these results is the duration of the signal for
the ITI reinforcers. Williams (1994) used a sign-
tracking procedure with rats as subjects, in which
the CS was a localized pilot light at either end of
the elongated chamber, and the measure of condi-
tioning was whether the rat went to the side of the
chamber on which the pilot light appeared on a
given trial. The food magazine was in the center
of the chamber, which meant that sign tracking to
the CS caused the rat to move away from the
food magazine.
Fig. 3 shows the acquisition of the sign-tracking
behavior for four groups of subjects: (1) the con-
trols for which no reinforcers were presented dur-
ing the ITI (CS-only); (2) subjects that received
unsignaled reinforcers during the ITI (No Signal);
(3) subjects that received ITI reinforcers signaled
by a 5-s noise (Short Signal); (4) subjects that
received ITI reinforcers signaled by a 15-s noise
(Long Signal). When the ITI reinforcers were
unsignaled, no conditioning emerged. Condition-
ing also failed to occur when a 5-s noise preceded
the US presentations in the absence of the CS, in
apparent agreement with the earlier results of
Jenkins et al. (1981). But when the signal was 15-s
in duration, conditioning to the CS did occur,
although obviously much more slowly than when
no ITI USs were presented.
These results shown in Fig. 3 from a Pavlovian
conditioning procedure are quite similar to those
obtained with a standard operant conditioning
procedure (Williams, 1989a) similar in many re-
spects to the procedures used by Rachlin and
Baum (1969, 1972). Pigeons were trained to peck
on a VI 3 min and then a VT 30-s schedule was
superimposed. The VT reinforcers were either
unsignaled, or preceded by a blackout signal. In
one condition the signal was 4 s in duration. In a
second condition the signal was 12 s in duration.
A within-subject design was used such that each
subject received each condition for 25 sessions.
Results for individual subjects are shown in Fig.
4. It is apparent that the 4-s signal produced only
marginally less suppression than the unsignaled
condition, while the 12-s signal produced little
response suppression relative to the baseline.
As in the case of blocking considered earlier,
Pavlovian and operant conditioning seem to be
quite similar in the laws of reinforcement, despite
the differences in the types of contingencies in-
volved. This suggests that the associative analysis
that has been given for Pavlovian conditioning
might also be used for understanding operant
conditioning. Certainly the predictive validity of
the response for the reinforcer seems to be an
 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126 121

important determinant of operant conditioning,
just as the predictive validity of the CS for the US
seems to be important for Pavlovian conditioning.
But it is less clear that the kind of analysis that
Rescorla has given for contingency effects can be
extended to the concept of relative rate of rein-
forcement in operant conditioning.
3.2. Context 6alue in operant conditioning
The effects of context conditioning on the rate
of operant response rate were assessed by Pearce
and Hall (1979) by various manipulations of con-
text value after responding had been established
on a VI schedule of reinforcement. With the levers
removed from the chamber, one group of subjects
received VT food presentations; a second group
received extinction, while a third group had a
corresponding period of time in their home cages.
The effects of the context manipulation were then
assessed during an extinction test phase with the
levers re-inserted into the chamber. Response
rates were lowest in subjects that received no food
during their exposure to the context, while re-
sponse rates for the remaining two groups were
not significantly different. In a second experiment
the nonreinforced exposure and VT reinforcement
conditions were compared to a third condition in
which response-independent food was preceded
by a 30-s signal. Here the nonreinforced exposure
and signaled VT conditions produced similar re-
sponse rates, with both significantly less than the
unsignaled VT reinforcement condition. The re-
sults of Pearce and Hall thus suggest that context
value is positively correlated with response rate,
not negatively as appears to be the case with the
acquisition of Pavlovian conditioning. Such ef-
fects are consistent with ‘two-factor’ theories of
instrumental conditioning (e.g. Rescorla and
Solomon, 1967) which postulate a positive inter-
action between the response-reinforcer operant
contingency and the incentive process generated
by Pavlovian signal-reinforcer and/or context-re-
inforcer associations.
But the opposite pattern of results was obtained
by Reed and Reilly (1990), who pretrained their

Fig. 3. Acquisition of sign tracking as a function of different reinforcement conditions during the ITI: all groups except CS-only
received reinforcers during the ITI while the groups differed as a function of whether these ITI reinforcers were signaled and the
duration of that signal. (Reprinted from Williams, 1994, Psychonomic Bulletin & Review.)
122 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126

Fig. 4. Response rate maintained by a VI-3 min schedule of reinforcement as a function of the signal condition of reinforcers
delivered on a VT-30-s schedule. (Reprinted from Williams, 1989a, Learning and Motivation.)

subjects on a VI schedule but with a 5-s resetting
delay of reinforcement. After this pretraining,
subjects received either nonreinforced exposure to
the context in the absence of the response lever or
an equivalent time in their home cages. Effects of
context manipulation were then assessed during
an extinction test for both conditions. Here re-
sponse rate was higher when the subject received
nonreinforced exposure to the context, pre-
sumably because competing responses had been
extinguished. In a second experiment the pretrain-
ing involved a signaled resetting delay of rein-
forcement. Response rate during baseline was
much higher with the signaled delay than with the
unsignaled delay. The effects of the context ma-
nipulation were also different. With signaled de-
lays, context extinction decreased the rate of
responding on the probe extinction test, relative
to the home-cage controls. In a third experiment
Reed and Reilly pretrained with an unsignaled
delay of reinforcement, then compared the effects
of signaled versus unsignaled VT reinforcers dur-
ing the context exposure. Here the signaled VT
reinforcers increased response rate relative to the
home cage controls, and produced higher re-
sponse rates than when the context exposure in-
volved unsignaled VT reinforcers. The latter
condition produced response rates slightly lower
than the home-cage controls.
Dickinson et al. (1996) studied similar manipu-
lations of context value but prior to acquisition
training under extended delays of reinforcement,
much like those first used by Lattal and Gleeson
(1990). Prior to each acquisition session with a
32-s delay of reinforcement contingency, context
exposure was presented with the response lever
withdrawn. Subjects receiving nonreinforced con-
text pre-exposure produced higher response rates
than subjects without context pre-exposure, con-
sistent with the prior study of Dickinson et al.
(1992). In addition, subjects receiving free food
during the pre-exposure initially had a similar
elevated response rate during the first several ses-
sions, but these decreased over extended training
to be similar to the home-cage controls. In their
Experiment 2 response rates were also higher than
the home-cage controls when pre-exposure in-
cluded signaled VT reinforcers.
 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
The experiments just described yield a complex
picture of how context value affects operant be-
havior. When acquisition under unsignaled delays
of reinforcement is studied, decreasing the value
of the context appears to produce faster acquisi-
tion. But when asymptotic response rates main-
tained by schedules of immediate reinforcement
are studied, increases in the value of the context
appears to augment response rate. Finally when
the asymptotic response rate is maintained by
unsignaled delays of reinforcement extinguishing
the context increases response rate, but decreases
response rate when the operant response is main-
tained by signaled delays of value.
The forgoing results imply that unsignaled de-
lays of reinforcement have differential sensitivity
to context manipulations, presumably because the
contingency allows many different kinds of behav-
ior oriented toward the contextual stimuli to be-
come competitors with the operant response,
either in terms of the response-reinforcer associa-
tion, or in terms of response competition. Unclear
from the studies just described is whether there is
a systematic difference between acquisition and
asymptotic responding in their sensitivity to the
context manipulation, given that the acquisition
was always studied in combination with a delay
contingency. Whether pretraining the context
would help or hinder response acquisition using
immediate reinforcement thus remains an impor-
tant unanswered question.
The effect of context-reinforcer associations on
operant response rate apparently depends on the
complex dynamics of the translation of arousal
into the specific channels specified by the contin-
gency of reinforcement. Killeen (1994) has sum-
marized the evidence that arousal grows
monotonically with the total rate of reinforcement
in the situation. Whether this growth in arousal
will increase the rate of the operant response then
depends on whether this arousal is allocated to
the operant response or to response forms that
compete with the operant response. Because
adding a VT schedule of free reinforcement does
not differentially allocate its arousal, behaviors
competitive with the operant response should in-
crease in frequency and thus reduce the response
rate maintained by the VI schedule. Signaling the
123
VT reinforcers presumably reduces the response
competition by restricting the competing behavior
to the time the signal is present. The reason short
signals 1– 4 s in duration are ineffective in pre-
venting the suppression of operant responding
due to the VT schedule is that the signal duration
must exceed the size of the memory buffer, the
contents of which at the time of reinforcer deliv-
ery determines what is strengthened. One implica-
tion of such an account is that assessment of the
size of the memory buffer (see Killeen, 1994, for a
method for making such an assessment), should
predict the duration of the signal that is needed in
order to eliminate the suppression of operant
responding caused by superimposed VT reinforce-
ment.
But at least one aspect of contingency manipu-
lations on operant responding cannot be ex-
plained by response competition due to
generalized arousal. Several different studies
(Dickinson and Charnock, 1985; Colwill and Res-
corla, 1986; Williams, 1989b) have show that the
degree of response suppression caused by the
superimposition of schedules of free reinforcement
is significantly greater when the response-indepen-
dent reinforcer is identical to the response-contin-
gent reinforcer than when its identity is different.
Given that the value of the different reinforcers is
similar, it is not obvious why they should produce
differential amounts of competing behavior. The
results are further complicated by the finding that
this effect of reinforcer identity is decreased when
the alternative response-independent reinforcer is
preceded by a signal or contingent on a second
response (Williams, 1989b)
4. Controlling variables for choice behavior
The experimental results I have discussed so far
are based primarily on experiments in which the
strength of a single response has been studied,
while much of the experimental literature on con-
tingencies of reinforcement during that past 30
years involves choice behavior, which has spon-
sored a theoretical emphasis on the determinants
of relative rates of responding. These two enter-
prises often seem entirely separate, with major
124 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
differences in the conceptual frameworks that
have been constructed to explain the two sets of
findings.
A major part of the problem is that it is
difficult to know which independent variables are
fundamental and which are derivative. For exam-
ple, is rate of reinforcement derivable from proba-
bility of reinforcement, or is probability of
reinforcement derivable from rate of reinforce-
ment? Skinner believed probability of reinforce-
ment was fundamental, but most subsequent
investigators in the operant tradition have favored
rate of reinforcement, or, in some cases, local rate
of reinforcement. The reason that rate of rein-
forcement has been favored is that the concept of
probability implies a discrete response occurrence,
whereas responses can often be continuous in
duration.
There is an important distinction between mo-
lar rate of reinforcement versus local rate of rein-
forcement as explanatory concepts. Local rate of
reinforcement is equivalent to probability of rein-
forcement if the time spent responding is consid-
ered to be segmented into time units, each of
which is associated with a given reinforcement
probability. Then by summing over the time units,
a local rate of reinforcement can be calculated. In
contrast the molar rate of reinforcement, which
disregards the time actually spent responding, ex-
cept as a dependent variable, is not commensurate
in any obvious way with probability of reinforce-
ment.
For the theoretical analysis of reinforcement to
advance it is imperative to gain a deeper under-
standing of the basic units of analysis. An exam-
ple from my own laboratory illustrates the
fundamental nature of the issue. Williams (1993)
presented pigeons a multiple schedule of discrete
trials in which probability of reinforcement for
the different response alternatives was the inde-
pendent variable. The schedule in one component
was designed to mimic a concurrent VI VI sched-
ule: when a reinforcer had been scheduled by the
probability gate, the reinforcer was held until the
next response to that alternative. This means that
the longer the time since the last response to a
given choice alternative, the greater the probabil-
ity that a reinforcer will have been scheduled. The
scheduled probabilities for the choice component
of the schedule were 0.20 versus 0.05. Note that if
matching occurs, there should be a 4:1 preference
in favor of the 0.20 probability, and the obtained
probabilities of reinforcement for the two alterna-
tives should be equal, somewhere in the range of
0.2– 0.25. In the second component of the multi-
ple schedule a single response alternative was
available, with a scheduled probability of 0.10.
Because subjects responded on almost every trial,
the obtained probability was also 0.10.
After behavior had stabilized in each compo-
nent, choice probe tests were presented in which
the 0.20 alternative was paired with the 0.10 alter-
native, and in which the 0.05 alternative was also
paired with the 0.10 alternative. The results were
that the 0.20 was preferred over the 0.10, but the
0.10 was preferred over the 0.05. Given that the
obtained probabilities of reinforcement for the
0.05 and 0.20 alternatives were approximately
equal, the fact that preference went in opposite
directions seems paradoxical. The apparent infer-
ence is that the obtained probability of reinforce-
ment is not the controlling variable.
The results become even more challenging when
they are considered along side of those of subjects
that were yoked to those just described. Here
there were never any choice trials, but the ‘0.20’
and ‘0.05’ alternatives were presented separately
with a probability of reinforcement equivalent to
those actually obtained by the master subjects for
each alternative. The yoking procedure also en-
tailed that the distribution of trials for the 0.20
and 0.05 alternatives was similar to those ob-
tained by the subjects as a result of their choice
distribution. This meant there were four times as
many trials with the 0.20 alternative as with the
0.05 alternative. The 0.10 alternative continued to
be presented on 1/2 of all of the trial presenta-
tions. Here the probe results were entirely differ-
ent. Now both the ‘0.20’ and ‘0.05’ were strongly
preferred over the 0.10 alternative. This of course
is not surprising given that both had higher ob-
tained probabilities of reinforcement.
The issue posed is why obtained probability of
reinforcement was the controlling variable in the
yoked procedure but not in the choice procedure.
This simple question is, I believe, diagnostic of the
 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
future of understanding reinforcement at a more
fundamental level. Two recent theoretical papers,
Dragoi and Staddon (1999) and Gallistel and
Gibbon (2000), offer accounts of the results just
described, along with related findings (Williams
and Royalty, 1989; Belke, 1992; Gibbon, 1995), as
part of a more general framework. The assump-
tions underlying the two accounts are fundamen-
tally different, and it remains to be seen which, if
either, account will prevail. Both are similar in
offering a comprehensive account that encom-
passes not only choice but response acquisition,
and asymptotic response rates. At the minimum
the development of such comprehensive compet-
ing theories promise renewed enthusiasm for the
experimental analysis of behavior, and an in-
creased focus on defining the concept of reinforce-
ment at its most fundamental level.
5. Uncited reference
Skinner, 1953
References
Arbuckle, J.L., Lattal, K.A., 1988. Changes in the functional
response units with briefly delayed reinforcement. J. Exp.
Anal. Behav. 49, 249 –263.
Baum, W.M., 1973. The correlation-based law of effect. J.
Exp. Anal. Behav. 20, 137 –143.
Belke, T.W., 1992. Stimulus preference and the transitivity of
preference. Anim. Learn. Behav. 20, 401 – 406.
Catania, A.C., 1963. Concurrent performances: reinforcement
interaction and response independence. J. Exp. Anal. Be-
hav. 6, 299 – 300.
Colwill, R., Rescorla, R., 1986. Associative structures in in-
strumental learning. In: G.H. Bower (Editor). Psychol.
Learn. Motivat. 20, 55 –104.
Dickinson, A., Charnock, D.J., 1985. Contingency effects with
maintained instrumental reinforcement. Q. J. Exp. Psychol.
37B, 397 – 416.
Dickinson, A., Watt, A., Griffiths, W.J.H., 1992. Free-operant
acquisition with delayed reinforcement. Q. J. Exp. Psychol.
45B, 241 – 258.
Dickinson, A., Watt, A., Varga, Z.I., 1996. Context condition-
ing and free-operant acquisition under delayed reinforce-
ment. Q. J. Exp. Psychol. 49B, 97 – 110.
Dragoi, V., Staddon, J.E.R., 1999. The dynamics of operant
conditioning. Psychol. Rev. 106, 20 – 61.
125
Durlach, P.J., 1983. The effect of signalling intertrial USs in
autoshaping. J. Exp. Psychol. Anim. Behav. Process. 9,
374 – 389.
Gallistel, C.R., Gibbon, J., 2000. Time, rate, and conditioning.
Psychol. Rev. 107, 289 – 344.
Gibbon, J., 1995. Dynamics of time matching: arousal makes
better seem worse. Psychon. Bull. Rev. 2, 208 – 215.
Grice, R., 1948. The relation of secondary reinforcement to
delayed reward in visual discrimination learning. J. Exp.
Psychol. 38, 1 – 16.
Herrnstein, R.J., 1970. On the law of effect. J. Exp. Anal.
Behav. 13, 243 – 266.
Hull, C.L., 1943. Principles of Behavior. Appleton-Century,
New York.
Holland, P.C., Forbes, D.T., 1982. Representation-mediated
extinction of conditioned flavor aversions. Learn. Motivat.
13, 454 – 471.
Jenkins, H.M., Barnes, R.A., Barrera, F.J., 1981. Why au-
toshaping depends on trial spacing. In: Locurto, C.M.,
Terrace, H.S., Gibbon, J. (Eds.), Autoshaping and Condi-
tioning Theory. Academic Press, New York, pp. 225 – 284.
Kamin, L.J., 1988. ‘Attention-like’ processes in classical condi-
tioning. In: Jones, M.R. (Ed.), Miami Symposium on the
Prediction of Behavior: Aversive Stimulation. University of
Miami Press, Coral Gables, FL, pp. 9 – 33.
Killeen, P.R., 1994. Mathematical principles of reinforcement.
Behav. Brain Sci. 17, 105 – 135.
Killeen, P.R., Bizo, L.A., 1998. Psychon. Bull. Rev. 5, 221 –
238.
Lattal, K.A., Gleeson, S., 1990. Response acquisition with
delayed reinforcement. J. Exp. Psychol. Anim. Behav. Pro-
cess. 16, 27 – 39.
Lieberman, D.A., McIntosh, D.C., Thomas, G.V., 1979.
Learning when reward is delayed: a marking hypothesis. J.
Exp. Psychol. Anim. Behav. Process. 5, 224 – 242.
Logan, F., 1960. Incentive: How the Conditions of Reinforce-
ment Affect the Performance of Rats. Yale University
Press, New Haven, CT.
Mazur, J.E., 1997. Choice, delay, probability, and conditioned
reinforcement. Anim. Learn. Behav. 25, 131 – 147.
Mazur, J.E., 1998. Choice and self-control. In: Lattal, K.A.,
Perone, M. (Eds.), Handbook of Research Methods in
Human Operant Behavior. Plenum Press, New York, pp.
131 – 161.
Mowrer, O.H., 1960. Learning Theory and Behavior. Wiley,
New York.
Neuringer, A.J., 1970. Superstitious key pecking after three
peck-produced reinforcements. J. Exp. Anal. Behav. 13,
127 – 134.
Papini, M.R., Bitterman, M.E., 1990. The role of contingency
in classical conditioning. Psychol. Rev. 97, 396 – 403.
Pearce, J.M., Hall, G., 1979. The influence of context-rein-
forcer associations on instrumental performance. Anim.
Learn. Behav. 7, 504 – 508.
Rachlin, H., Baum, W.M., 1969. Response rate as a function
of the amount of reinforcement for a signalled concurrent
response. J. Exp. Anal. Behav. 12, 11 – 16.
126 B.A. Williams / Beha6ioural Processes 54 (2001) 111–126
Rachlin, H., Baum, W.M., 1972. Effects of alternative rein-
forcement: does the source matter? J. Exp. Anal. Behav.
18, 232 – 241.
Reed, P., Reilly, S., 1990. Context extinction following condi-
tioning with delayed reward enhances subsequent instru-
mental responding. J. Exp. Psychol. Anim. Behav. Process.
16, 48 – 55.
Reid, A.K., Bacha, G., Moran, C., 1993. The temporal organi-
zation of behavior on periodic food schedules. J. Exp.
Anal. Behav. 59, 1 – 27.
Rescorla, R.A., 1967. Pavlovian conditioning and its proper
control procedures. Psychol. Rev. 74, 71 – 80.
Rescorla, R.A., 1968. Probability of shock in the presence and
absence of CS in fear conditioning. J. Comp. Physiol.
Psychol. 66, 1 – 5.
Rescorla, R.A., Solomon, R.L., 1967. Two-process learning
theory: relationships between Pavlovian conditioning and
instrumental learning. Psychol. Rev. 74, 151 –182.
Skinner, B.F., 1948. Superstition in the pigeon. J. Exp. Psy-
chol. 38, 168 – 172.
Skinner, B.F., 1953. Science and Human Behavior. Free Press,
New York.
Staddon, J.E.R., Simmelhag, V.G., 1971. The ‘superstition’
experiment: a reexamination of its implications for the
principles of adaptive behavior. Psychol. Rev. 78, 3 –43.
Timberlake, W., 1988. The behavior of organisms: purposive
behavior as a type of reflex. J. Exp. Anal. Behav. 50,
305 – 317.
Timberlake, W., Lucas, G.A., 1985. The basis of superstitious
behavior: chance contingency, stimulus substitution, or
appetitive behavior? J. Exp. Anal. Behav. 44, 279 –299.
.
Williams, B.A., 1975. The blocking of reinforcement control.
J. Exp. Anal. Behav. 24, 215 – 225.
Williams, B.A., 1976. The effects of unsignalled delayed rein-
forcement. J. Exp. Anal. Behav. 26, 441 – 449.
Williams, B.A., 1978. Information effects on the response-rein-
forcer association. Anim. Learn. Behav. 6, 371 – 379.
Williams, B.A., 1982. Blocking the response-reinforcer associa-
tion. In: Commons, M., Herrnstein, R., Wagner, A. (Eds.),
Quantitative Analyses of Behavior: Acquisition, vol. 3.
Ballinger Publishing Company, Cambridge, MA, pp. 427 –
445.
Williams, B.A., 1988. Reinforcement, choice, and response
strength. In: Atkinson, R.C., Herrnstein, R.J., Lindzey, G.,
Luce, R.D. (Eds.), Stevens’ Handbook of Experimental
Psychology, 2nd edition. Wiley, New York, pp. 167 – 244.
Williams, B.A., 1989a. Signal duration and suppression of
operant responding by free reinforcement. Learn. Motivat.
20, 335 – 357.
Williams, B.A., 1989b. The effects of response contingency
and reinforcement identity on response suppression by
alternative reinforcement. Learn. Motivat. 20, 204 – 224.
Williams, B.A., 1993. Molar versus local reinforcement proba-
bility as determinants of stimulus value. J. Exp. Anal.
Behav. 59, 163 – 172.
Williams, B.A., 1994. Contingency theory and the effect of the
duration of signals for noncontingent reinforcement. Psy-
chon. Bull. Rev. 1, 111 – 114.
Williams, B.A., 1999. Blocking the response-reinforcer associa-
tion: theoretical implications. Psychon. Bull. Rev. 6, 618 –
623.
Williams, B.A., Royalty, P., 1989. A test of the melioration
theory of matching. J. Exp. Psychol. Anim. Behav. Pro-
cess. 15, 99 – 113.
.