VOL. 11, NO.

WATER

RESOURCES

RESEARCH

DECEMBER

1975

Reservoir Management Models

MATTHEW

J. SOBEL

Schoolof Organizationand Management, Yale University,New Haven, Connecticut 06520

The optimal policiesfor severaldiscretetime control modelsof reservoirstorageare characterized.

Each characterizationis exploitedto simplify the numericalsolutionof heretoforeformidableproblems.
Some of the resultsexploit analogiesbetweenmulti-item inventorytheory and systemsof multiple reservoirs. In particular, it is observedthat the 'linear decisionrule' label in the water resourceliterature ordinarily arisesin contextswhereeither(1) a myopicpolicyis optimal, or (2) a multiple-reservoiroptimal
releasepolicy,as a functionof the vectorof reservoircontents,possesses
a Jacobianmatrix whosevalues
are betweenzero and one. Either (1) or (2) impressivelyreducesthe computationalburden that would
otherwise

be carried.

This paper analyzesthe structure of optimal policies for

severaldiscretetime control modelsof reservoirstorage.Most
of the models are stochasticand they are prompted by
operating problems of regulating the amounts of water discharged from reservoirs.However, the design problem of
selectingthe capacityof a reservoirinducestwo modelsin the
paper and, of these,one is deterministic.The form of an optimal policyis characterizedfor eachmodeland thenexploited
to simplify the numerical solution of heretofore formidable
problems. Several sectionsof the paper exploit analogies
betweenmultiple-reservoirsystemsand multi-item inventory
systems.As a consequence,the form of some of the results
belowshouldbe familiar to readersacquaintedwith inventory
theory.
Severalpurposesare servedby proliferatingthe number of
available reservoirmodels.Actual reservoirsystemsdisplay
heterogeneouscharacteristics,and so a richer set of models
permits adequate representationof more of these systems.
Furthermore, it permits the same reservoir system to be
modeled in several ways. Thus dichotomies in this paper
such as deterministic/stochastic,independent/correlated,and
chance constraints/explicit costs are not presented as a taxonomy of actual reservoirs.Instead,they are differentways of
idealizing the same problem of managing a water storage
process.

tor) that receivesdifferent labelsin differentliteratures:'linear

decisionrule' (water resources),'basestockagepolicy' (operations research), and 'single critical number' (operations
research).In the subsequentsectionthe structureof optimal
policies is explored for multiple-reservoirsystemsthat may
lack a myopic solution.Optimal releasepolicies,as functions
of reservoircontents,are shown to possessa Jacobianmatrix
whose values are between zero and one. This property is
shown to acceleratecomputations impressively.
For expository convenience, the previously mentioned
stochasticmodels assumethat inflows in different periods are
indendent.This assumptionis relaxedin the final sectionof
the paper whereit is shownthat the precedingalgori.thmsand
policy structuresextend to generalstochasticprocessesof inflows and demands.
NOTATION

AND CONVENTIONS

Vector notation is used in the sectionsof the paper concerned with multiple-reservoir systems. However, inner
productsare the only multiplicationsneeded.If a = (a, ...,
ai) and = (, ..., i) are real I vectors,then a. or .a is

writtento denotetheinnerproduct-'t--[ottlt.
The notation
ej is used for the I vector having all zeros except one in the
jth component,j = 1, ..., I.
All functions that arise are assumed to attain their minima

SVMMA}

The following sectioninvestigatesthe reservoirdesignproblem of computingthe smallestreservoircapacitythat will accommodatea sequenceof predictedinflows. It is shownthat a
hand calculation, simpler than previously proposed
algorithms,will solvethis problem. Moreover, somestochastic
versionsof the reservoircapacityproblemcan be solvedequally easily.
The paper continuesby developingan analogy between
modelsof multiple-reservoirsystemsand of 'multi-item inventory' models. Cost minimization (or net revenue maximization) reservoirmodelsare very closelyrelated to inventory modelsabout whoseoptimization so much is known. The
analogy is exploited in a stochastic model of a class of
multiple-reservoirsystems.This model has a simple myopic
solution, and so the sequentialproblem degeneratesto a sequenceof single-periodproblemswhosenumericalsolutionis
easilyobtained.The 'myopic' solutionspecifiesa point (or vecCopyright(D 1975by the AmericanGeophysicalUnion.
767

when they are constrainedto compactdomains.It is sufficient

that the functions be lower semicontinuous,and this restriction poses no practical difficulties. Whatever functions are
differentiated will possessone-sided derivatives. For definiteness, therefore, derivatives are assumed to be taken from
the left.

When I' is a finite set of real numbers or a real-valued

func-

tion that attains a constrained minimum, the notation min I'

denotes the smallest member of the set or lowest value of the

function. Similarly, if I' is a finite set of real numbers, then

max I' denotesthe largestmemberof the set. It is convenient
to write(a) + for max {a, 0} whena is a real number.Ifa = (a,
", at) is a real I vector, then (a) + denotesthe I vectorwhose
ith component is (at)+. The notation P{A} indicates the
probability of an event A.
M any quantitiesin the followingmodelswill vary with time,
and the subscriptt denotesthe time period to which a variable
pertains. If the variable refers to the ith reservoir in a collection of I reservoirs,then it carriesa superscriptof i. For example, st denotesthe quantityof water storedin reservoiri at the
end of period t. If the superscripti is absentfrom such a

768

SOBEL:RESERVOIRMANAGEMENTMODELS

variable,thenit denotesthe relevantI vectorin periodt, e.g.,st

=
..., sD.
The notationlist presentsmostof the symbolsusedin the

constraints

are

paper.Generally,capitallettersdenotethe randomcounterpartsto lowercase

letters,andthesecapitalsareabsentfrom
thelist.Exceptions
to thisrule,suchasI, K, and T, areon the
list. The latterpart of the papermakessomeuseof dynamic
programingin generaland inventorytheoryin particular.
Somequantities
whichare scalarsuntil that pointthereafter
are functions.For example,the randomdrawdownin periodt
is denotedXt(u) asa functionof freeboardu at the beginning
of the period.Asterisksare sometimes
usedto denotean optimal policy,Xt*( ) beingan example.Suchvariantsof a
lowercase
symbol,xt in thiscase,are not shownin the list.
One last notationalpeculiarityattendsthe mixingof water
resourcemodelswith inventorytheory. The maximal draw-

If discharge
is constrained
by storageat the beginning
of the
periodinstead
of at theend,i.e.,xt < st- insteadof xt < st +

0 <_St <--C

0 _<Xt <--St- + rt

(3)

rt, thenthe followingdevelopment

is essentially
unchanged
exceptthat rt- replacesrt throughout.
Additional constraints on freeboards and dischargequan-

titiesare sometimesdictatedby the multiple usesof a reservoir:

m < st

q < xt <f

(4)

In (4), rnis minimalstorage,and q andf areminimalandmaximal discharges.

A planninghorizon of finite length,say, T periods,is assumed.For giveninitial storageSothe problem is to find the
down has been labeledf in past water models,and the ex- smallestcapacityc (includingminimal freeboard)and a sepectedcostof an optimalinventorypolicyhasbeendenoted quencex, ..., xr satisfying(1), (4), andst < c for all t = 0,
ft( ) in inventorytheory.Boththeseusages
occurin the se- .., c. Hereafter,it is assumedthat this problem hasa feasible

quel,butat mostoneof themisusedin anygivensection.

In

solution.

fact,thefirstoccurrence
offt( ) iswellbeyondthelastuseoff

Given So,if s, ..., s. is the sequenceof storedquantities

generatedby a feasiblepolicy, then the smallestfeasible
capacityis the maximum of So, "', s.. Thus the original
problem(for givenSo)is equivalentto minimizingthe maximumamong{So,', s.}suchthat therearex, ., x. satisfy-

to denote maximal

drawdown.

SMALLEST STORAGE CAPACITY FOR PREDICTED NEEDS:

A CHEBYSHEV PROBLEM

This section concerns a deterministic model, namely, a

storageprocess
in whichinflowsanddemandaretreatedasbeing knownin advance.The problem,concerning
a planning
horizonof T periods,is to determine(1) the smalleststorage
capacitythat accommodates
the inflowsand permitsthe projecteddemands
to be metand(2) the corresponding
schedule
of discharges.
Suchdeterministic
modelsarisein subtasks
in
simulationprogramshavingrandomelements.
The optimization has been posed by ReVelle et al. [1969] as a linear
programing
problem.It will be shownherethat simplehand

ing (1) and (4).

First Result,

This 'minimax capacity problem' is an example of a

Chebyshevoptimization problem, namely, a searchfor the
minimum possiblevalue of a constrainedmaximum. Such
problems have a long honorable history in applied
mathematics.Intuitively, reasonabIeproceduresoften lead to
optimal solutionsof Chebyshevproblems.As occurredin the
water qualityproblemconsideredby Sobel[1971],the followcalculations lead to a solution.
Thomas' 'sequent peak' algorithm [Harvard Water ing result is verified with an argumentbasedon the idea of
Resources
Group,1963](seeFiering[1967]or Loucks[1970]) alwaysdischargingas muchas possible.A simplecaseis consolvesa problemrelatedto ours.It isthespecialcaseof (1)-(4) sideredfirst: supposert -->q for all t = 1, ..., T. Then thereis
belowwhenft = coandmt= 0 for all t andit is 'assumed
that an optimal solution of the minimax capacityproblem such
that
the inflowand draftswill be repeatedin successive
setsof cy-

clesof T-yearseach'[HarvardWaterResources
Group,1963,
(]- xt)(st- m) = 0
t = 1, ..., T
(5)
pp. 1-6]. This lastassumption
is riot madebelow.In someapof (5)
plicationsit is an attractivefeature,whilein othersit is not The proofis basedon the directlyverifiableequivalence
sought.In somecases,theassumption
forcesan increase
in the with
requisitecapacity.

xt = min {f, st_ + rt -- m}

(6)
Let st denotethe quantityof waterin storageat the endof
periodt and let rt be the predictedinflowduringthe period. Then the assumptionthat somefeasiblepolicy existsand also
rt >--q for all t impliesthat this policysatisfies(1) and (4). OpThen st_ and st are connectedby
timality is shownby taking the storagelevelss', ., s.'deter,

St = St- + rt -- Xt

(1)

wherext denotesthe drawdown,the quantitydischargeddur-

minedby any other policy,evenif it is optimal, and comparing

them with s, ..., s.generatedby (1) and (6). It is seenthat st

< st' for t = 1, -.., T.

ing period t.
More realistically,supposethat insteadof (4) thereare timeIn reservoirswhosesurfacearea doesnot vary significantly
with storagevolume,leakageand evaporationalsoare essen- varying constraints

tially independent
of storageolume.Thenrt canbe regarded
as inflow net of leakageand evaporation,and the algebraic
sum is not a function of storagevolume.
Of course(1) is invalidif the rightsideexceeds
thecapacityc
of the reservoir,and so an elaboration of (1) is

mt <--st

qt <--xt <--Jt

t = 1,"

', T

(7)

withrt < qtperhaps

inatleastoneperiodt. Thenthepolicy(6)

of discharging as much as possible may be infeasible.

Although(5) may no longerbe feasible,the ideaof discharging
as much aspossibleis still worth pursuing.'As muchaspossist = min {c, st_ + rt -- xt}
(2)
ble' in any period t is the minimum of ft and st_ + rt -- mt
the overflow,if any, being(st_ + rt -- xt -- c)+. The physical minus any water required to attain xT > qT or sT> rn in

SOBEL:RESERVOIRMANAGEMENTMOVERS

future periodsr > t. Let Lt denotethe sum of mt and water

requiredto be held back in t for future use. A policy for
whichxt > st-1 + rt -- Lt in a period t leadsto infeasibilityin
some period r > t. Then

Lr = mr

andsuppose
that the valuesst-i, mr,rt, qt, andft takenby St-i,
Mr, Rt, Qt, and Ft are known when the value of Xt is selected.
A stochasticversion of rt -- qt and mt -- mt+l for all t is

P{Rt- Qt

Lt = mt + (Lt+l + qt+l - rt+l - mr)+

t = 1, ...,

(8)

T-

769

Mt - Mt+l

There is some decision rule X1, "',

t=

1,'..,

T} = 1 (13)

Xr such that

PlMt+l < St

Qt < xt < Ft

t = 1,...,

T} = 1

(14)

is verified inductively startingwith t = T- 1 and proceeding

from t + 1 to t. The minimax capacity problem with (7) in Assumptions(13) and (14)guarantee that a feasiblerule exists
(with probability one). Let X denote the set of rules (X1, "',
place of (4) has an optimal solution satisfying
Xr) satisfying(14). The stochasticversionof (6) is
(ft - xt)(st - Lt) = 0
t = 1,..., T
(9)
't* = min {Ft, St_i* q- Rt - Mt}
(15)
The proofs of (9) and (5) are the sameafter recognizingthat (9)
where So* -- So and
is equivalent to

xt = min {ft, st-1 + rt -- Lt}

The computationsunderlying(8) and (10) (or (6) if rt -- qt

and mt -->mt+l for all t) are trivial by hand. They are followed
by the recursion(1) for sl, "', st. Then the minimally sufficient capacity is c = max {So,'", st}.
Table 1 presentsa numerical exampleof the applicationsof
(1), (8), and (10). The data, the computationsfor (8), (10), and
(1), and the end products, the draft and storagedecisions,are
given.
STOCHASTIC MINIMAX

St* : St-i*

t = 1," ', T (10)

CAPACITY

An inspectionof (5) and (6) showsthat the valuetaken by an

optimal dischargequantity xt doesnot dependon the data for
any periodlater than t. In fact, if qt -< xt --<ft and mt --<st are
time-varying restrictions,then rt -- qt for all t and rnl _>rn2>
" > mr imply that

xt = min Ift, st-1 + rt - mr}

(11)

q- Rt - Xt*

t = 1, "',

Second Result

Then the optimality and myopia of (11) imply that (15) is

feasible and induces minimax capacity under assumptions
(13) and (14). This claim is true with probability one. As a
consequence,among all rules satisfying(14) the one given by
(15) minimizes the expectation of the maximum stored
quantity.
Let ht be an outcome(or 'realization' or 'point in the sample
space') of the history
Ht -- (So, Q1, F1, M1, X1, R1, "',

St-2, Qt-1, Ft-1,

Mt_l, )'t-1, Rt-1, St-1, Qt, Ft, Mt)

(16)

It is physicallyimpossiblefor policy X1, '", Xr to haveXt depend on more information than is contained in Hr. Suppose
(14) is replaced by the chance constraint

P{mt St "-St_l q-rt -- mr(mr)

(17a)
is optimal. Here the decisionxt in period t is unaffectedby and
knowledgeof eventsin periodslater than t. Thus the myopic
qt --<X't(ht) ft} - Olt
(175)
characterof (6) is retained by (11) which implies that a variety
of stochasticminimax capacity problems have an easy solu- for all ht and t = 1, ..., T. The values of mr, rt, qt, and st-1
are includedin ht and the valuesof at are assumedto satisfy0
tion. Supposethat a stochasticprocess(Q1, F1, M1, R0, "',
(Qr, Fr, Mr, Rr) determines(qt, ft, mr, rt) for t = 1, ..., T. (at ( 1, t = 1, ...,T.
The restrictiveness
of (17) comparedto (14) is more illusory
The storagelevels S1, "', Sr and discharges,Y1, '", ,Yr are
than real. Suppose(13) is valid and that X1, '", Xv is a policy
controlled random variables. Let
different from (.15) and let So, "', St be the storage levels
St = St-1 + Rt-Xt
(12) generated by the Xt. It will be shown that maxt {St*t
TABLE 1. A Simple Minimax Capacity Problem
Data

Decisions

Month

Maximum
Draft

Minimum
Draft

ft

qt

10

10

Inflow
rt

Minimum
Storage
mt

10
10
15
21

1'
1'
1
1

17
13

Computations
Dratt

Lt

$t- i + rt - Lt

Xt

-1
-3
16

1'
1'
4
17'

9
10
12
10

9
10
10
10

1
1
6
17

I
3

23
22

24*
25*

10
12

10
12

24
25t

9
6

2
1

16
6

18
7

16
18

15
17

19
8

12

2*

14

14

10

10
12
11

1'
0
1

1'
0
1

11
13
12

11
11
10

1
2
3

10'

10

11

13

11

15'

13

17'

15

14
12
11'
10'

11
12

St

10

Storage

Lt + + qt+ - rt +i

8
8

*Binding constraint.
'Minimal c = s6 = 25 = max {s0,'",

s.}.

-4
-2

770

SOBEL:
RESERVOIR
MANAGEMENT
MODELS

_< maxt {St} with probability one. If Xt(ht) < Xt*(ht) for
all ht and t, then St 2 St* for all ht and t; so

P{max {St*} max {St}} = 1

t

and X* is superiorto X. The only other possibilityis that there

is the smallestinteger t and history ht such that

Xt(ht) > Xt*(ht) = min {ft, st- * + rt - mr}

Third Result

Then either (1) Xt(ht) ft, and Xt is infeasible,or (2) Xt(ht)

St-* + rt -- mt = st- + rt -- mr; so mt > st = st- + rt --

X(h), P{M(h) _<&(h, X ), and Q(h) _<X(h) _<F(h)} =

0, which violates(17); so Xt is again infeasible.Therefore(13)
implies that X* in (15) is optimal for the stochasticminimax
problem with chance constraints.
COMMENT

reservoirreleaseshas developed.Little [1955], Gessfordand

Karlin [1958], and Amir [1967] significantlyadvancedMass6's
results.As a generalizationof their models,consideran inter-

connectedsystemof I reservoirshavingcapacitiesC, ..., CI

and let C = (0, ..., CI).
Elaborating on the previous notation, let sd denote the
amount of water in the ith reservoirat the end of period t, let
xd denote the amount of water released from it, and let
denote the random amount of water that flows into it. Let st =

(St, ''', Stl), Rt = (Rt1, ''', Rtl), and xt = (xt', '", xtI) so
that (2) is valid with the presentvectornotionif rt is replaced
by Rt and the min in (2) is takenseparatelyfor eachcomponent i = 1, ...,

discharge
beforeinflow,as the decisionvariableinsteadof xtt.
Let

Yt = St-1i -- Xt

ON CHANCE CONSTRAINTS

AND POLICY STRUCTURE

'Chance-constrained'programing combines dynamic cost

minimization with probability constraintson some operating
characteristicsthat would result from a chosen policy. Inequality (17) is an exampleof a chanceconstraint.Severalrecent reservoir management models (see LeClerc and Marks
[1973] and its references)have included chance constraints,
but their authors have ignored the well-known [Derman and
Klein, 1965] possibility that a randomized policy dominates
every unrandomized policy. For a trivial example, suppose
there are two actionslabeled0 and 1. Exactly one of thesetwo
must be taken every period subjectto the constraintthat action I be taken at least 100% of the time. Action j incurs a
cost of $j each time that it is taken, and the objective is to
minimize the averagecost per period. Action I taken every
period is the only unrandomizedstationarypolicythat satisfies
the 100% constraint, and the associatedaverage cost is $1.
However, the randomizedst.ationary policy that takesaction 0
with probability I - and action 1 with probability also
satisfiesthe constraint, and its average cost is $ < $1.
Water resource models have not yet applied chance constraints to joint events. For example, if et is the event being
constrainedin (17), then one may seeka solution that satisfies
P{ UI . U/ '"

UI r} > a0

in addition to (17). This observationis pertinentin mostwater

resourcecontexts;methods for solving the resultingoptimization problemsare discussedby Miller and Wagner [1965].
FORMULATION
APPLICABILITY

OF OPERATING
OF INVENTORY

PROBLEMS:
THEORY

The benefitsand costsassociatedwith a particular reservoir

designdependpartly on the modesof operationthat the design
permits. In other words, design problems and operating
problems are interrelated becausethe design of a reservoir
affectsthe compositionof the classof feasibleoperatingrules.
However, this interdependence will not be made explicit
henceforth,and the remainderof the paper is devotedto determining the size of drafts in successiveperiods. This sectionof
the paper examinesthe formulation of operatingproblemsand
concludesthat their structure is sharedby models found in
'inventory theory.'
Beginning with the seminal work by Massd [1946] (which
also originated dynamic programing), a sizableliterature on
the determination of optimal sequentialdecision rules for

I.

It is convenientto usest_ - xd, namely,the storageafter

Yt = (Yt1, '' ', YtI)

so that (2) is equivalent to

st - min {C, Yt q- Rt}

(18)

If water cannotbe pumpedfrom one reservoirto another,then

the constraint on xt is 0 < xt < st_x or

0 __Yt -- St-

(19)

If water can be pumpedfrom any reservoirto any other during

the same period then the constraint is weaker, namely,

st-t i _ E Yti
i

0 _<yt C

(20)

If water can be pumped only from somereservoirsto others,

then the constraintswill be intermediatebetween(19) and (20).
For specificityin the sequelthe 'no pumping'assumption(19)
will be made. However, the essentialpropertiesof the following resultsare preservedunder (20) or any mixture of (19) and
(20). For easeof expositionthe random vectorsR, Ro.,... are
assumedto be independent.This assumptionwill be relaxedat
the end of the paper;if this werenot possible,the resultswould
be valueless.

There

is a one-to-one

relation

between

the contents

of a

reservoir and its freeboard. Let ud = C - s,_d denote the

freeboardin reservoiri at the beginningof period t and let u, =

(ut, "',

ud). Similarly, let v,' = C - yd which can be in-

terpreted as the freeboard after dischargingxd before the inflow Rt occurs:

l)? = Ci -- yti = Ci -- St_ + Xtt = Ut + Xt

Now (18) and (19) are equivalent to

ut+ = (vt - R)+

u < v < C

(21)

The purposeof the transformation (s, y) --, (u, v) is to observe

that (21) is formally the recurrenceequation for successive
inventory levelsin a 'lost sales'inventory system.If vd is the
amount of 'goods' availableto satisfy'demand'Rt, then the
subsequent'inventorylevel' ut+ i is -- R? if that difference is
nonnegative.If Rtt ttt, then the excessdemand is lost (lost
sales)and ut+ = 0 asa result. There is a hugeliteratureon optimal sequentialorderingrules(the 'order' is vd - ut) in inventory systems(as evidencedby the surveysby Veinott [1966]
and Clark [1972]). Therefore the appearanceof (21) suggests
that inventory theoretic resultsand methodologymay be applicable to reservoirmanagement.That conjectureis verified
in following sections.

SOBEL:
RESERVOIR
MANAGEMENT
MODELS
COSTS AND BENEFITS DEPENDING
ON RESERVOIR

ONLY

771

Example1. The problems(21) and (22) encompass

costs
and revenuesas linear functionsof dischargequantitiesx,
' ', xr. This fact was first exploitedby Veinottand Wagner
[1965](who mentionthat Martin Beckmannhad recognizedit
earlier)in an inventoryproblem.SupposeAt' is the cost(net of
benefits)of a unit quantityreleasedfrom reservoiri during
periodt. Let At = (At, '' ', At')sothat At'xt (innerproduct)
isthe netcostof xt. Let bt(ut)denotethenetcostof havingst_
= C - ut in storageat the startof periodt. Then the total cost

LEVELS

Continuingfrom (21), let Gt(u,v) denotethe expectedcostin

periodt (net of revenues)of a vectorxt = v - u of discharges
if
storageat the beginningof the period is st_ = C - u. Detailed
modelsfor the constructionof Gt( , ) are presentedby
many authorssuchasMassd [1946],Little [ 1955],Gessfordand
Karlin [1958], Amir [1967], and Roefs [1968]. Su and Deininger
[1972] presentmodels in which the expectednet costsin each
period t depend only on the reservoir level, and so there are K during T periods, a random variable, is
T
functionsg ( ), '", gr( ) such that
gt(v) Gt(u, tg)

t = 1, 2,''

K=

', T

[At'xt -'[-Ot(ut+)l

t--1

whereT is the planninghorizon.The problemin thiscaseis to

choosev, ..-, vr sequentiallysubjectto (21) so as to

Letting Ar+ -= 0 and using(21) and ut = (vt_ - Rt_O+,

minE gt(vt)

(22)

t----1

----

where E denotesexpectation.

'

)+

t=2

If the constraints(21) could be ignored,then the problem

would havethe followingtrivial myopicsolution.Let tit be the
biggest(lexicographically)nonnegativevector in [0, C] that
minimizesgt( ); an optimal unconstrainedsolutionis vt = tit
(so xt = tit - C + st-O for all t = l, ...,

t--.1

T. The constraints

(21)jeopardizethissolutionwith thepossibilitythat ut ; tit for

somet. That eventualityis precludedby the assumption

tit-dr _< tit+ t = 1, ...,

T-

1,(23)

....

t--.--1

Therefore
T

EK = Y'. gt(vt)-- A' u

t=l
where dt= (dr, "', dt), dt beingthe minimum possibleinflow to reservoiri in period t, and'so dt is the largestnumber where A-u is uninfluencedby the decisionsv, ..., vr and
satisfyingP{Qt _>dtt} = 1. Of course,u < ti < ti2< < tit
gt(v) = At.v - At+.E(v - Rt)+ + Ebt[(v - Rt)+] (25)
is sufficient for (23).

To seethat (23) implies that vt = tit for all t is feasible,first Then the problemof feasiblyminimizingEK is equivalentto
observethat u _<ti impliesthat v = ti satisfies(21) for t = 1. (22).
Supposethat vt = tit is feasible; then

Example2. As a specialcaseof (25), supposeonly a single

reservoiris beingregulatedsothat I = 1 and alsoAt = A, bt(u)
with probabilityoneby definitionof dr. For the ith compo- = B(u - 3,)2, and P{Rt _<r} = r/C (0 _<r < C) for all t = 1,
nent in (24), eitherut- - dt < 0 = (ut- - cltt)+ < ut+- .., T. It is assumedthat A > 0, B > 0, 3, > 0, and 2B(C - 3,)

ut+ = (vt - Rt)+ = (tit - Rt)+ < (tit - dr)+

(24)

(becauseut+-t is nonnegative
by definition)or 0 < (ut- -

< A. For t < T, (25) takes the form

dt)+ = ut- -dt <

_ Ut+-t from (23). The inductiveconclusion,with probability one, is that (23) ensuresut < tit, and sovt
Cgt(v)= --A (v-- r) dr--[-B (v-- r --)2 dr
= at is feasiblefor all t = 1, ..., T. Optimality of vt = tit for all
t followsfrom the fact that the impositionof constraints(21)
-'l- B3'2(C-- v) --I-CAr
(25')
cannotreducethe costof an optimal solutionto a previously
for 0 < v < C. Straightforwardcalculusshowsthat this funcunconstrainedproblem.
tion is concaveon [0, C] (becauseof the inequalityabove).
Fourth Result
Thereforeits minimum on [0, C] occurseither at v = 0 or at v
In summary,(23) impliesthatvt = tit for t = 1, ..., T is op- = C. The assumptionA > 0, B > 0, and 3, > 0 ensuresthat
timal for the problemof (21) and (22). The key stepis to gt( ) is minimizedat v = 0 if t < T. BecauseA r+ -= O, subchangethe decisionvariablefromxt to eitherYt or t9t.In either stitutionin (25) for t = T yields(25') exceptthat the first term
case,givenst_ or ut, there is a one-to-onerelation with xt. is absent.It is straightforwardto showthat gr( ) is convexon
When (23) is valid,changingthe decisionvariablereplacesa [0, C] and minimizedat 0. Then in the notationof (23), tit = 0
burdensomeT-perioddynamicprogramingrecursionin an l- for all t; so (23) is satisfiedif u = 0, i.e., So= C. Therefore if
dimensionalstate space with T separate /-dimensional the reservoiris full at the outset,then an optimalpolicyconminimizationproblems.If gt( ) is the samefor all t, i.e., the sistsof dischargingxt = C - st_ - tit = C - C - 0 = 0 at the
cost structure is 'stationary,' then just one /-dimensional beginningof each period t.
minimizationis needed.The wholeproblemis saidto have a

fo

myopic solution becausevt = tit does not stem from a recur-

sion. The resultsabovefor problems(21) and (22) with assumption(23) makeup a specialcaseof propertiesthat Veinott
[1965]derivedfor multi-iteminventorysystems.
Clark [1972]
surveysrecentresearchon multi-item inventory systems.

fo

Example3. The problemanalyzedby Su and Deininger

[1972]may exemplify(25). They considera singlereservoir
(LakeSuperior)in whichthedecision
variableis thequantity
discharged
andtheexpected
costs(netof benefits)
depend
only
on the levelof the reservoirat the beginningof a period.

772

SOBEL: RESERVOIR MANAGEMENT MODELS

FORMULATION

OF OPERATING

DYNAMIC

PROBLEMS:

It is useful to introduce

the notation

PROGRAMING

(28)

Jt(u, 1))= Gt(u, l)) q- Eft+xLo(/),Rt) ]

Recall the notation Gt(u, o) for the single-periodexpected

costof a vectorx = v - u of discharges
if initial storagesare
st_ = C - u. The precedingsectionexplainedthat the computation of an optimal policyis often easyif Gt dependsessentially only on v. This sectionconcernsmorecomplexmodelsin
which that assumptioncannot be made. It is convenientto let
p(a, fi) = (a - fi)+ for I vectorsa and fl. Then from (21),
Ut+x = P(t;t, Rt)

so (27) can be written

it(u) = min Jt(ll,t))

u_v _ c

(0 _<U _<C)

(29)

It is assumedthat Gt( , ) is a weakly subadditivefunction

on its domain {(u, v): 0 _ u _ r _ C} A, namely,that crosspartial derivativesof Gt, with respectto uj and ra, are nonpositive. Specifically,henceforthit is assumedfor each t that

At the beginning of each period t an outcome ht of the 0 _>Gt(u + bey,v + 'Yen)- Gt(u, l; + 'Yen)
historyHt is observed,and then vt.is chosen(xt = vt - ut). A
-- Gt(u + 6el, V) + Gt(u, v)
reservoirreleasepolicy for period t is thus an I vector-valued

(30)

functionVt( ) ofht suchthat 0 < ut < Vt(h)< C for all possi- whereb, 'Y > 0 suchthat all four argumentsin (30) lie in A.
Example 1 above is a special case of the usual situation
ble outcomesht of Ht (ut is specifiedin ht). Let Vt = (Vt, '",
where Gt( , ) containsone term dependingon reservoir
Vr) denotean arbitrary releasepolicy in periodst, t + 1, ...
T andletft(Vt[ht) betheconditional
expected
netcostin those levelsand another term dependingon dischargequantities.
periodsfrom followingpolicy Vt giventhat the historyHt = Specifically,if
hr. That is
Gt(u, v) = at(v - u) + bt(u)
(31)
T

lb,)=

then nonnegativityof the cross-partialderivativesof at( ) ensures(30). The crosspartials are nonnegative,for instance,if

i=t

The criterionof optimalitywill beft( [ ). Moreprecisely,

at(u- v) -=at*Y][v'- u] withat* beingconvex.

The secondterm in (28), Eft+[o(v, Rt)], is trivially weakly

subadditivein (u, v), and a sum of weakly subadditivefuncft(Vt*lht) < ft(Vt ht)
(26) tionsis itselfweaklysubadditive.Therefore(30) and (28) imply
for eacht that Jt( , ) is weakly subadditive.It followsfrom
for all ht and Vt, t = 1, ..., T, whereVt* = (Vt*, "', Vr*). A (29) and Topkis [1968] that there is an optimal policy Vt*( )
straightforwardinductiveproof (starting with t: T) shows for (29) such that

a policy V* = (V*, ..., Vr*) is'definedto be optimal if

that if thereexistsan optimalpolicy,thenft(Vt*lht) depends

on ht onlythroughut, t = 1, ..., T, andft(Vt*lht) -=ft(ut),the
{ft} satisfying

0_u_C,t

0 _( u _( u' _( C

(32)

To interpret (32), let s and s' be I vectorssuchthat 0 < s' < s

ft(u) = min {G,(u, v) '-k EJ,+,[p(v,R,)]},

u_v_

Vt'(u) Vt(u')

(27)

= 1,"',T(fr+(

)=0).Conversely, ifthereisa

sequence
{ft( )} satisfying(27), then thereexistsan optimal
policyV* = (V*,. ., Vr*), Vt*(ht) -- Vt*(ut) beinganyvalue
of v at which the minimum in (27) is attained.
The recursiveequation (27) will be used to characterizean
optimal policy. A virtue of the dynamicprograming formulation thus far is the absenceof any heuristicdevicessuchas the
'principle of optimality.'
There is a considerableliteratur,e on optimal economic
growth under uncertaintyand optimal consumptionand savingsby an individual during a lifetime. Equation (27) with 0 <
v _( u replacing u _( v _( C is formally equivalentto the recursive equationsof 'capital accumulation'which pervade that
literature.However,it is reasonableto assumethat p( , r) is a
concavefunction (for eachr) in the capital accumulationcontext whereas(v - r) + lacksthat usefulproperty. Referencesto
the capital accumulation literature and resultsfor (27) when
p( , r) is concavecan be found in part 1 of the volume edited
by Szegb'and Shell [1972].
MONOTONE

POLICIES

<Candletu=C-sandu'=C-s'sothat0<u<u'<C.

NOW VtVa(u)-- C-s q- X't(s) VtV(u

') - C- st+ X't(s
')
from (32) where Xt*(s) is an optimal dischargepolicy when
period t initial storagelevelsare given by s. Therefore

Xt*(s) - Xt*(s') < s - s'

0 < s' < s < C

(33)

is equivalentto (32), and so the optimal amount dischargedincreasesno faster than the stored quantity. Also if Xt*( ) is

differentiablefrom the left (provedbelow), then (33) implies

aXt*(s)/as < 1, i, j = 1, ..., I, 0 < s < C.
To proveXt*( ) nondecreasing
so that aXt*(s)/as > O,it
is convenient,followingVeinott[1972],to rewrite(29) with the
decision variable

as x = v -

u instead of v and the state

variableass = C - u insteadof u. Let wt(s)---ft(C - s) sothat

wt(s) = min {Gt(C-- s, C-- s Jr-x)

O_z_x

+ EWt+l(min{C, s-

x nt- Rt})}

(34)

Assumption
(30)istooweakto ensure
fhedesired
propertyfor
Xt*(

). Supposealsothat separation(31) appliesto Gt( , )

so that

Gt(C - s, C-

s + x) = at(x) + bt(C-

s)

The objective of this sectionis to demonstratethat there is It is assumedfor eacht that at and bt have nonnegativecrossan optimal dischargepolicy Xt*(s), as a function of reservoir partial derivativesor, more generally,that
levelss, suchthat 0 < aXt*'(s)/asj < 1, i,j = 1, ...,I. Then a
at(x + be:) - at(x)
bt(x + be:) - bt(x)
(35)
higherstoragelevel inducesa largerdischarge,but the incrementin thedischarge
doesnotexceed
theincrement
in storage. are nondecreasing
functionsof xn for any k j, j - 1, ., I,
This propertywill be exploitedcomputationallyin the follow- b > 0, and 0 < x < x + bej < C. It followsfrom (34), (35), and
ing section.
Topkis[1968] that there is an optimal policy Xt*( ) for (34)

SOBEL:
RESERVOIR
MANAGEMENT
MODELS
such that

Xt*(s') < Xt(s)

773

angle[0, C] canbe generatedsequentiallyby startingat u = C

unit vectorsej.
(36) and then repeatedlysubtracting/-dimensional

0 _<s' _<s _< C

There are

II (C q-- 1)-- 1

Fifth Result

i--1

Thereforeoptimal dischargequantitiesare increasingfunctions of the storagequantities.Inequality (36) impliesthat latticepointsin [0, C] not includingthe point C. From (40), at
Xt*( ) is-continuousexceptat upwardjumps and that these each of thesepoints at most 2 alternativesneed to be comare the onlykind ofjumpspossible.However,(33) impliesthat pared; so for each t, Vt*( ) is determinedwith fewer than

only downwardjumpsare possible.Taken together,(33) and

(36) imply that Xt*( ) is continuous.Also (36) asserts

monotonicity,
and so ,It*(

2r II (C' -3-1)

(41)

i--1

) is left-differentiable
on (0, C], comparisonsof alternative decisions.A straightforward com-

and all the elementsof the Jacobianmatrix of Xt*(

values between 0 and 1, i.e..

) have putation using(29), by comparison,requiressearchingthe lat-

tice points in [u, C] for each u. The rectangle [u, C] contains

I

0<
_ OXt*(x)
Os
i _<1

0 < s <_C

i,j = 1, ... , I

II (c'i=l

u' +

(37) points, so summing over u [0., C], there are

Someversionof (33), (36), or (37) when I = 1, i.e., a system
I
with just one reservoir,was shown by Mass [1946], Little
Z

[1955],and Gessford
andKarlin [1958].As Figure2 by Eisel
[1970]and Figure I by Russell[1972]illustrate,(37) is char-

u [0, tY]

i=l

= 2-' II (C+ 2)(C+ 1)

acteristic of most reservoir release rules in the literature. Amir

[1967] usedan intricateproof and more restrictiveassumptionsto prove(37) for multiple-reservoir

systems.
Burt [1964]
conjectureddXt*(s)/ds _>0 whenI = 1 in groundwatermanagementmodels.The mathematicalstructureunderlyingmany
of Burt'spapers(seeBurr [1970]for a bibliography)is closely
related to (27) and thereforeto capital accumulationmodels.
A numerical example that illustrates the computational
benefitsof monotoneoptimal policiesis presentedafter the
following section.
ACCELERATED COMPUTATIONS

FOR MONOTONE

POLICIES

This sectionexplainshow (33), (36), and (37) canbe exploitedto accelerate

thecomputation
of an optimalpolicy.It
is assumed
thatvariables
havebeendiscretized
for computa-

(42)

comparisonswhen the monotone structureof Vt*( ) is not

exploited.Somevaluesof the ratio of (41) to (42) are presented
in Table 2. The computational improvement of (40) with

respectto (29) becomes

morepronounced
asI and C increase.
Supposethat (30) is valid but (31) is invalid, or (31) applies
but the crosspartials of at and bt are not all nonnegative.Then
in the discretecase, Vt*(u) < Vt*(u + e), but Vt*'(u + e) >
Vt*(u) + 1 is possiblefor someu, i, andj. The problem is still
simpler than (29) but computationally much more burdensome than (40).
EXAMPLE

The followingexampleillustratesthe useof (40). Suppose

tionsand that Ci is an integralmultipleof the commonunit I = 1;soa singlereservoiris beingcontrolled,
itscapacityis C =
storageand releasequantityfor reservoiri, i = 1, ..., I. Then 3, and the planninghorizon has T = 3 periods.Benefitsand

(37) is equivalent to

costsin a period t are assumedto derivefrom a demandDt for

Xt*'(s + ej) {Xt*'(s),Xt*'(s) + 1}

(38)

downstream

use of water and the level of the reservoir at the

beginning of each period. Suppose that the net cost of

i = 1, ...,I

j = 1, ...,I

downstream

use is

In words,suppose
that Xt* is knownat somelatticepoints, 0
< s < C. Let s' = s + ej be an adjacentlatticepoint suchthat

4(Dr - xt) + + 2(xt - Dr) +

and that

S'

k .i

s' = s-'[' 1

k = j

Now ,Yt*(s+ e;) is an optimalquantityto releasefrom reservoir i if the contentsof thereservoirs

aregivenbys + e;. Thus
(38) asserts
that eitherXt*(s + e;) is the sameasXt*(s) or else
it is Xt*(s) + 1. ThereforetheI vectorXt*(s + e;) mustlie on
oneof the2 latticepointsx adjacentto Xt*(s) suchthat x _>
Xt*(s). Formally, let M denotethe set of I vectors,each of
whose componentsis zero or one. Thus M contains 2 elements.Now (38) is equivalentto

Xt*(s + e.) {Xt*(s) + m' rn M}

the net cost associated

with

reservoir

levels is

(st-1 - 2)'.The demands D1, Do.,and Ds are assumedto be

independentrandom variableswith the same distribution:

(39)

By usingv - u = x in (29), the effectof (39) is

ft(u- ej) = min lJr(u- ej, Vt*(u) - m): rn M} (40)

for 0 < u < C andj = 1, ..., I. For eacht, (40) suggests
a
recursionin u startingwith u = C. Necessarily,Vt*(C) - C
because
u < v < C. The latticepointsin the/-dimensionalrect-

P{Dt = d}

0.1

1
2

0.4
0.3

3
4

0.1
0.1

TABLE 2. Ratio of Number of AlternativeDecisionsper Stagein

AcceleratedVersusStraightforwardAlgorithms,
O=C

.....

Ct=c

Number
of

Size of Each Reservoir c

Reservoirs
I

1
2
5

10

50

0.571
0.327
0.187

0.333
0.111
0.037

0.078
0.006
2.6 X 10-6

774

SOBEL:
RESERVOIR
MANAGEMENT
MODELS
TABLE 3.

For example,Xa(2) = 2 + Va(3 - 2) - 3 = -1 + Va(1)= -1 +

Values of G(u, v) in the Example

2-

v=O

0
1
2
3

7.8

v=l

v=2

4.4
6.8

v=3

3.4
3.4
7.8

1.

It is necessaryto evaluateEf[(v - Rs)+] because

4.2
2.4
4.4
10.8

Jr(u, v) = St(u, v) + Eft+[(v - Rt)+]

(45)

It is assumedthat R, R., and Rs are independent random

variableswith the following common distribution'

Then Gt(u, v) takes the following form:

Gt(u, t;) = 4E(Dt - t; + u)+ + 2E(v - u - Dr)+

+ (3 - u - 2)' (43)

P{Rt = r}

0
1
2
3
4

0.3
o. 1
0.1
0.3
0.2

Using the distributionof Dt to evaluatethe expectations

resultsin the valuesfor G(u, v) (seeTable 3). To illustratethe
computationof G(u, v), considerG(2, 3):

Use of this distributionand the valuesoff(u) resultsin the fol-

G(2, 3) = 4[P{Ot = 0}(0 -- 3 + 2)+ + P{Ot = 1}(1 -- 3 '-I-2)+ + P{O = 2}(2 -- 3 + 2)+

+ P{ D = 3}(3 -- 3 + 2)+ + P{ D = 4}(4 -- 3 + 2)+1+ 2[P{D, = 0}(3 - 2 - 0)+

+ P{Dt = 1}(3- 2-

1)+ + P{Dt = 2}(3-

2-- 2)+ + P{D = 3}(3-

2-

3)+

q- P{Dt = 4}(3 -- 2-- 4)+1q- (3-- 2-- 2)'

= 410.1(0)q- 0.4(0)q- 0.3(1)-3-0.1(2)q- 0.1(3)1q- 210.1(1)q- 0.4(0)q- 0.3(0)q- 0.1(0)q-0.1(0)1q- 1'

=

3.4 q- 1 = 4.4

The weak subadditivityof G(u, v) can be verified either with

(43) or Table 3. For example,

lowing tabulation of Ef[(v - RO+]:

[G(2, 3) - G(2, 2)] - [G(1, 3) - G(1, 2)]

= -3.4(-1) = -2.4 N 0

BecauseT = 3 andfr+( ) -- 0, we haveJ3(u,v) = G(u, v).

Therefore the recursionspecifiedbeneath (40) takes the form
in Table 4 for t - 3. To illustrate the computation, consider

3.4

3.4

3.7

5.7

For example, for v = 3,

/a(1)= min{Ja(1,Va(2)),
Ja(1,Va(2)- 1)}

Era[(3- Rs)+]

= min { Ja(1,3), Ja(1,2)}

= fa(3)P{Rs= 0} q- /a(2)P{Rs= 1}

= min {G(1, 3), G(1, 2)}

'-I-fa(1)P{Rs= 2} + fa(0)P{Rs
> 2}

= min {4.2, 3.4} = 3.4

= 10.8(0.3)+ 4.4(0.1)+ 3.4(0.1)+'3.4(0.5) = 5.7

The minimum is attainedat J3(1, 2) = G(1, 2), and so Va(1) = 2.

In orderto find thecorresponding

optimaldischarge
policy,
namely,Xa(s2),we use the identity
t)t = C -- st- + Xt
so

Xt(st-O = st_ + Vt(C- st_) - C

(44)

mined with (44) and follows:

In the presentcase,this yieldsthe followingpolicy:

TABLE 4.

Using this equation and the above tabulationin (45) for

t = 2 resultsin valuesfor J.(u,v) (seeTable 5). For example,
J.(1,2) = G(1, 2) + Ef[(2 - R0 +] = 3.4 + 3.7 = 7..1.Table 5
is usedto computef(u) in the samefashionthat Table 1 was
usedto computef(u). SeeTable 6 for the specificrecursion.
The optimal dischargepolicy associatedwith V.(u)is deter-

s,.

Xs(s,)

s,

X4s,)

0
I
2
3

0
1
1
2

0
I
2

0
1
1

TABLE 5.

RecursiveComputation off(u)

Computationoffa(u)

f(u)

3
2
1
0

G(3, 3) = 10.8
min {G(2, 3), G(2, 2)} - min {4.4, 7.8}
min {G(1, 3), G(1, 2)} = min {4.2, 3.4}
min {G(10, 2), G(0, 1)} = min {3.4, 4.4}

10.8
4.4
3.4
3.4

v=0

V3(u)
3
3
2
2

0
1
2
3

11.2

Valisesof d,.(u, v) in the Example

v=l
7.8
10.2

v=2
7.1
7.1
11.5

v=3
9.9
8.1
10.1
16.5

SOBEL:RESERVOIR
MANAGEMENT
MODELS
TABLE 6.

RecursiveComputation off(u)

TABLE 8.

Computation off(u)

f(u)

Vs(u)

ds(3, 3) = 16.5
rain {ds(2,3), d:(2, 2)} = rain {10.1, 11.5}

16.5
10.1

3
3

min {ds(l,3), ds(l, 2)} = rain {.8.12,7.1}

7.1

rain {ds(O,2), ds(O,1)} = rain {7.1, 7.8}

7.1

Tables 7-9 present the remaining computations in the same

order in which they are performed.
v

A[(v -

7.1

1
2
3

7.1

775
Recursive Computation of f(u)

Computation off(u)

f(u)

V(u)

21.0
14.6
11.4
11.4

3
3
2
2

d(3, 3) = 21.0
min {d(2, 3), d(2, 2)} = min {14.6, 15.8}
rain {d;(1,3), d(1,2)} = rain {12.6, 11.4}
rain {dffO,2), d(O, 1)} = rain {11.4, 11.5}

Let at(w) be the (lexicographically)

biggestnonnegativevector
that minimizesgt( ; w) and let dti(w) be the minimum possible inflow to reservoiri in periodt if Wt = w. In placeof (23), if

P{tit(w)- dr(w) < tit+x(Wt+x)lWt= w} = 1

for all o: ft, t = 1, ..., T-- 1, and ux < fix(Wx),then an optimal policy is vt = tit(Wt) for all t.

8.0
10.2

Returningnowto thegeneralcasein (28) and(29), suppose

for each t that Gt( ,

; w) is weakly subadditivefor each
o: ft. It follows that there is an optimal dischargepolicy
This section is concernedwith the impact of conditional Xt*(x; w)such that
dependencein the inflow sequenceof random I vectorsRx, R2,
Xt*(s; oo)- Xt*(s'; ) < s - s'
0 < s' < s < C
", Rr. The precedingstochasticmodelsalready encompass
interdependence
of the I componentsof Rt = (Rtx, ''', Rtt). thus generalizing (33) for each w ft. If, further, for each t,
Also, the stochastic minimax capacity model allowed
Gt(u, u + x; w) = atix; w) + bt(u; w)
dependenceamong Rx, "', Rr. In general, the impact of
dependenceis to expand the dimensionof the state spacebut with at( ; w) and bt( ; w) satisfying(35) for eacho: ft, then
to leaveunchangedthe generalform of an optimal policy. This also
assertionwill be illustrated for the stochasticmodels in prex*(s'; o.,)_< x*(s; o.,)
0 _<s' _<s _< C
ceding sections.
Let Wt(Rx, '' ', Rt-x) denote a statistic of Rx, "', Rt-x Hence for each t and co,Xt*( ; w) is left-differentiable on
that is sufficient for Rt, namely,
(0, C], and (by analogy with (46)) for each w e fit,
CORRELATED

INFLOWS

P{Rt < alWt} = P{Rt < alRx, ..., Rt_x}

(46)

Such a statistic always existsbecause(Rx, "', Rt-O is itself

sufficient.Let ft be the set of all possiblevalues of Wt. Then
instead of (27) or (28) and (29) the generic problem is

u _<v _< C

Jr(u, v; w) = Gt(u, v; w) + Eft+x(p[v,Rt], t[w,Rt])

t=l,...,T;fr+x(

beenusedby econometricians
for suchdualpurposes.
Estimation in the distributed
lagmodelhasbeenstudiedparticularly
(48) extensively
(seethe surveyby Griliches[1967]).A distributed

) -=0),wherect

lag model here is

Rt = lit + fit

Wt+x(Rx,"', Rt) -: t[Wt(Rx, "', Rt_x), Rt]

(49)

Consider the problem in (22) and (23) having expectednet

costsin each period t dependingonly on reservoirlevels.The
natural extension to (47) and (48) is
gt(v; w) = Gt(u, v; w)

t = 1, "',

i,j=1 ... I

Statisticalestimationproblemsarisein any particularexample of (46) and (50). Thereforeit is importantto selecta model
whoseestimationproblemsare solvableand whoseoptimal
(47) policy is easily computable. Structured Markov chains have

Jr(u;co)= min Jr(u,v; co)

(O_<u<v<C,w,2t,
is defined by

0<
OXt*i(s;w)
<
1 0 < s< C

osi
....

fit - et + /bt_x

0 < / < 1

lit beinga vector of specifiedconstantsand fit beinga random I

vector. The 'error terms' et, et-x, et-2, "', are ordinarily assumedto be independentand identically distributed.It follows
from (51) that

(50)

i=0

i=0

The auxiliary function t in (49) in this caseis specifiedby

An exampleof (50) is obtainedfrom (25):

qbt(w,Rt) = Rt - lit

gt(v; w) = (At - At_i) v + EIAt+x (Rt - v)+

+ bt[(v - Rt)]l Wt = w}
TABLE 9.

0
1

2
3

v=0

14.9

v=l

v=2

Optimal Discharge Policy

Discharge

TABLE 7. Values of d(u, v) in the Example

u

(51)

v=3

11.5

11.4

14.6

13.9

11.4

12.6

15.8

14.6
21.0

Storage
s

X,(s)

Xds)

X(s)

0
1
2
3

0
1
1
2

0
1
1
2

0
1
1
2

776

SOBEL: RESERVOIR MANAGEMENT MODELS

NOTATION

at(x)

At

net cost in period t of drawdown vector x.

net unit cos:of drawdownfrom reservoiri during

period t.

bt(u) netstoragecostof havingC - u in storageat startof

period t.

reservoircapacity as endogenousvariable.
exogenouscapacity of reservoir i.

dt minimumpossible
inflowto reservoiri in periodt.
Dt random downstreamdemand for water during
period t.

expectation(of a random variable).

ft(u) expected
costof an optimalpolicyduringperiodst
throughT if C - u is in storageat thestartof period
t.

ft

maximaldrawdownduringperiodt (sometimes
not

indexedby t).
gt(l)) Gt(u, v) when it doesnot dependon u.
Gt(u, v) expectednet cost in period t of a vectorv - u of
drawdowns
if storagelevelsat the beginningof the
period are C - u.

ht historyof eventsup to the beginning

of periodt.
i genericindexnumberof reservoir
superscripted
on
other variables.

I numberof reservoirsin system.

Jr(u, t)) expectedcostduringperiodst, ..., T of drawdown
u - v in periodt, whentheinitial storagewasC - u,
followedby an optimalpolicy.
K total (random)costduring T periods.

Lt minimalstoragein periodt pluswaterobligedto be

held back for future use.

mt minimalstorageat end of periodt (sometimes

not
indexedby t).

setof all I vectors

eachof whosecomponents
iszero
or

one.

expected
costsirrelevantto the decision
problem.

probability measure.

qt minimaldrawdownduringperiodt (sometimes
not
rt

indexedby t).
inflow during period t.

st quantityof waterin storageat endof periodt.

t
T

genericindex numberof period.

lengthof planninghorizon;

ud freeboard
in reservoir
i a,tbeginning
of periodt.
vt freeboard
in reservoir
i afterdischarge
butbefore
inw(x)
Wt(

xt

flow in period t.
f(cx).
sufficientstatistic.

drawdownduring period t.

ytt quantity
of waterin reservoir
i duringperiodt after
drawdownbeforeinflow,equalto st_x
- xt.
at

minimum probabilitylevel of a chanceconstraint.

p(v,r) storage
at endof periodif inflowisr andstorage
was
C - v just before inflow occurred.

REFERENCES

Amir, R., Optimum operation of a multi-reservoir water supply

system,Ph.D. thesis,StanfordUniv., Stanford,Calif., 1967.

Burt, O. R., Optimal resourceuse over time with an application to
groundwater, Manage. $ci., 11, 80-93, 1964.
Burt, O. R., Groundwater storagecontrol under institutional restrictions, Water Resour. Res., 6(6), 1540-1548, 1970.
Clark, A. J., An informal survey of multi-echeloninventory theory,
Nay. Res. Logist. Quart., 19(4), 621-650, 1972.
Derman, C., and M. Klein, Some remarks on finite horizon Marko-

vjan decisionmodels,Oper. Res., 13, 272-278, 1965.

Eisel, L. M., Comments on 'The linear decision rule in reservoir

managementand design' by Charles ReYelle, Erhard Joeres,and

William Kirby, Water Resour. Res., 6(4), 1239-1241, 1970.
Fiering, M., Streamflow Synthesis, Harvard University Press,
Cambridge, Mass., 1967.
Gessford,J., and S. Karlin, Optimal policy for hydroelectricoperations, in Studies in the Mathematical Theory of Inventory and
Production,chap. 11, editedby K. J. Arrow, S. Karlin, and H. Scarf,
Stanford University Press,Stanford, Calif., 1958.
Griliches, Z., Distributed lags: A survey,Econometrica,35(1), 16-49,
1967.

Harvard Water Resources Group, Operations research in water

quality management,Div. of Eng. and Appl. Phys. Harvard Univ.,
Cambridge, Mass., !963.
Iglehart, D. L., Capital accumulationand productionfor the firm: Optimal dynamicpolicies,Manage. Sci., 12, 193-205, 1965.
LeClerc,G., and D. H. Marks, Determinationof the dischargepolicy
for existing reservoir networks under differing objectives, Water
Resour. Res., 9(5), 1155-1165, 1973.
Little, J. D.C., The use of storagewater in a hydroelectricsystem,J.
Oper. Res. Soc. Amer., 3, 187-197, 1955.
Loucks, D. P., Some comments on linear decision rules and chance

constraints, Water Resour. Res., 6(2), 668-671, 1970.

Mass6, P., Les Rdserveset la Rdgulationde l'Avenir Dans la Vie

Economique,
2 vols., Hermann, Paris, 1946.
Miller, B. L., and H. M. Wagner, Chance constrainedprogramming
with joint constraints, Oper. Res., 13(6), 930-945, 1965.
ReVelle, C., E. Joeres,and W. Kirby, The linear decisionrule in reser-

voir managementand design, 1, Developmentof the stochastic

model, Water Resour. Res., 5(4), 767-777, 1969.
Roefs, T. G., Reservoirmanagement:The state of the art, Rep. 3203508, IBM Sci. Center, Yorktown, N.Y., 1968.
Russell,C. B., An optimal policy for operatinga multipurposereservoir, Oper. Res., 20(6), 1181-1189, 1972.
Sobel,M. J., Chebyshevoptimal wastedischarges,Oper.Res., 9(2),
308-322, 1971.
Su, S. Y., and R. A. Deininger, Modelling the regulation of Lake

Superiorunder uncertaintyof future water supplies,in Proceedings

of the InternationalSymposiumon Uncertaintiesin Hydrologicand
Water Resource Systems, vol. 2, pp. 555-575, University of
Arizona, Tucson, 1972.
SzegiS,G. P., and K. Shell, Mathematical Methods in Investmentand
Finance, North Holland, Amsterdam, 1972.

Topkis, D. M., Ordered optimal solutions, Ph.D. thesis, Stanford

Univ., Stanford, Calif., 1968.

Veinott, A. F., Jr., Optimal policy for a multi-product,dynamic,nonstationary inventory problem, Manage. Sci., 12(3), 206-222, 1965.
Veinott, A. F., Jr., The status of mathematicalinventory theory,
Manage. Sci., 12(11), 745-777, 1966.

Veinott, A. F., Jr., Sub-additivefunctionson a latticein inventory

theory, paper presentedat 19th InternationalMeeting, Inst. of
Manage. Sci., Houston, Tex., April 1972.
Veinott, A. F., Jr., and H. M. Wagner, Computing optimal (s, $) inventory policies, Manage. Sci., (5), 525-552, 1965.

3_ domainof thesingle-period
costfunctionGt( , ).
Acknowledgments.
This manuscriptwas partially supportedby
NSF grantGK-38121 and waspresentedat the PuertoRico meetingof
the Instituteof ManagementSciences/Operations
ResearchSocietyof
America

in October

1974.

(Received February 5, 1974;

revisedFebruary 13, 1975;
acceptedMarch 20, 1975.)