Professional Documents
Culture Documents
P
(t) +s
t
c
t
) dt +m
t
P
(t)dZ
t
, (3)
which is a special case of (2). If
P
= 0 then agent can only save (or borrow) at a known
risk-free rate. My model thus generalizes the previous treatments of hidden savings in
the literature. However, as I will see below, my results may be more applicable when
the drift b and/or the diffusion from (2) are strictly concave in m, as they are not here.
Absent this, I require that the agents wealth directly enter the utility function.
13
2.3. More Detail and A Change of Variables
I now provide a bit more technical detail on the basic model. I let C
n
y
and C
n
y
+n
z
be the spaces of continuous functions mapping [0, T] into R
n
y
and R
n
y
+n
z
, respectively. I
adopt the convention of letting a bar over a variable indicate an entire time path on [0, T].
Note then that the time path of output y = {y
t
: t [0, T]} is a (random) element of C
n
y
,
and the principals observation path in the hidden state case ( y,
Z) = {(y
t
, Z
t
) : t [0, T]}
is an element of C
n
y
+n
z
. I dene the ltration {Y
t
} to be the completion of the -algebra
generated by y
t
at each date, and similarly dene {Z
t
} as the ltration generated by
(y
t
, Z
t
). In the hidden action case then, I dene the set of admissible contracts S
a
for
11
Therefore I distinguish between the case of hidden states, as consdiered here, and that of partial information, in
which some random elements are not observed. Detemple, Govindaraj, and Loewenstein (2005) consider an intertem-
poral partial information model.
12
See Liptser and Shiryaev (2000) for a statement of the necessary conditions for the existence of a strong solution
and a discussion of the equivalence of observing the state and the Brownian motion.
13
As noted above, the hidden state case can also capture the dependence of output on past actions as in Fernandes
and Phelan (2000), say by allowing the drift f in (1) to depend on the hidden state m (which now captures a past
effort stock) and to allow the drift b to depend on e. Such generalizations are also used in Williams (2006) for hidden
information problems. My more specialized formulation here allows for a simple comparison between hidden actions
and hidden states.
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 7
the principal to be those Y
t
-predictable functions s
a
: [0, T] C
n
y
S.
14
This means
that the contract may specify an action by the principal at date t that depends on the
whole past history of the observations of the state up to that date (but not on the future).
Similarly, in the hidden state case, admissible contracts S
s
are those Z
t
-predictable func-
tions s
s
: [0, T] C
n
y
+n
z
S. Note that in either case I allow almost arbitrary history
dependence in the contract, where the relevant history is summarized by the informa-
tion in the appropriate ltration. Since the agent optimizes facing the given contract,
dene the set of admissible controls A for the agent as those F
t
-predictable functions
e : [0, T] C
n
y
+n
z
A and c : [0, T] C
n
y
+n
z
B. I assume that the sets A and B can be
written as the countable union of compact sets.
I now focus on the hidden state case, with hidden actions case as a special case. For a
given contract s
s
(t, y,
Z) the evolution of output (1) and wealth (2) can be written:
dy
t
=
f(t, y,
Z, e
t
)dt + (t, y,
Z)dW
t
, (4)
dm
t
=
b(t, y,
Z, m
t
, c
t
)dt +(t, m
t
)dZ
t
,
where I have dened
f(t, y,
Z, e
t
) = f(t, y
t
, e
t
, s
s
(t, y,
Z)), (t, y,
Z) = (t, y
t
, s
s
(t, y,
Z)),
and
b(t, y,
Z, m
t
, c
t
) = b(t, m
t
, c
t
, s
s
(t, y,
Z)). The history dependence in the contract in-
duces history dependence in the state evolution. Thus (4) is a stochastic differential
equation of the functional type, with coefcients depending on elements of C
n
y
+n
z
. This
dependence would complicate a direct approach to the problem.
As in Bismut (1978), I can make the problem tractable by taking the key state variable
to be the density of the output process instead of the output process directly. Different
effort choices by the agent change the distribution of output. Thus I can view the agents
effort choice as a choice of a probability measure over output. Similarly, in the hidden
state case it is convenient to look at the evolution of the agents wealth under this new
measure as well. As I show below, this reformulation of the problem leads to a simple
characterization of the agents optimality conditions, and more importantly to a deriva-
tion of the class of implementable contracts. Details of the change of measure are given
in Appendix A.1.
15
I nowchange my focus fromthe output process y
t
to the distribution over outcomes in
C
n
y
. The key state variable is the relative density
t
for the change of measure associated
with different effort policies. As I show in Appendix A.1, it evolves as:
d
t
=
t
1
(t, y,
Z)
f(t, y,
Z, e
t
)dW
0
t
, (5)
with
0
= 1. Here W
0
t
is a Weiner process on C
n
y
, and is interpretable as the distribution
of output resulting from an effort policy which makes output a martingale. The covari-
ation between output and the agents wealth is a key factor in the model. Thus is it also
14
See Elliott (1982) for a denition of predictability. Note that any left-continuous, adapted process is predictable.
15
Similar ideas are employed by Elliott (1982) and Schattler and Sung (1993), who use a similar change of measure
in their martingale methods. The martingale approach does not apply in the hidden state case however, as the contract
is conditioned directly on Z
t
and thus I cannot change the measure on that portion of the state.
8 NOAH WILLIAMS
useful to take x
t
=
t
m
t
as the relevant wealth state variable. Simple calculations show
that the evolution of this scaled wealth is given by:
dx
t
=
t
b(t, y,
Z, x
t
/
t
, c
t
)dt +x
t
1
(t, y,
Z)
f(t, y,
Z, e
t
)dW
0
t
+
t
(t, x
t
/
t
)dZ
t
, (6)
with x
0
= m
0
. Note that by changing variables from (y, m) in (4) to (, x) in (5)-(6) I
have changed the state evolution from a functional SDE to one with random coefcients.
Instead the key states directly depending on their entire past history, the coefcients of
the transformed state evolution depend on ( y,
Z), which are random elements of the
probability space. This leads to substantial simplications, as I show below.
3. THE AGENTS PROBLEM
In this section I derive optimality conditions for the agent facing a given contract. I
suppose that the agent has a reservation utility V
0
, and so I focus on contracts that satisfy
this participation constraint. As in the previous section, I focus on the more general
hidden state case, with the hidden action case being a simple specialization.
I rst pose the agents decision problem. The agent has standard expected utility
preferences dened over the states, his controls, and the principals actions (including
the agents payment). Preferences take a standard time additive form, with ow utility u
and a terminal utility v dened over the states. In particular, for an arbitrary admissible
control policy ( e, c) A and a given contract s
s
(t, y,
Z), the agents preferences are:
V ( e, c) = E
e
__
T
0
u(t, y
t
, m
t
, c
t
, e
t
, s
t
)dt +v(y
T
, m
T
)
_
= E
e
__
T
0
u(t, y,
Z, m
t
, c
t
, e
t
)dt +v(y
T
, m
T
)
_
= E
__
T
0
t
u(t, y,
Z, m
t
, c
t
, e
t
)dt +
T
v(y
T
, m
T
)
_
.
Here the rst line uses the expectation with respect to the measure P
e
over output in-
duced by the effort policy e, as discussed in Appendix A.1. The second line uses the
notation u(t, y,
Z, m
t
, c
t
, e
t
) = u(t, y
t
, m
t
, c
t
, e
t
, s
s
(t, y,
Z)), and the third uses the density
process dened above. The agents problem is to solve:
sup
( e, c)A
V ( e, c)
subject to (5)-(6). Any admissible control policy (e
, c
, x
).
Under my change of variables, the agents problem is simply a control problem with
random coefcients. I apply a stochastic maximum principle from Bismut (1973)-(1978)
to derive the agents necessary optimality conditions. Analogous to the deterministic
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 9
Pontryagin maximum principle, I dene a Hamiltonian function H as follows:
H(t, y, Z, m, e, c, , p, Q, R) = ( +Qm)
f(t, y, Z, e) + p
b(t, y, Z, m, c) (7)
+R (t, m) + u(t, y, Z, m, c, e).
Here and Q are n
y
dimensional vectors, p is a scalar, and R is a n
z
dimensional vector.
As in the deterministic theory, optimal controls maximize the Hamiltonian.
A key role in my analysis is played by the adjoint (or co-state) variables, whose evolu-
tion is governed by differentials of the Hamiltonian. Thus I introduce the adjoint equa-
tions corresponding to the states (
t
, x
t
), which satisfy the following:
dq
t
=
_
f(t) + (
b(t)
b
m
(t)m
t
) p
t
+ ((t)
m
(t)m
t
) R
t
+ ( u(t) u
m
(t)m
t
)
dt
+
t
(t)dW
0
t
+
t
dZ
t
(8)
q
T
= v(y
T
, m
T
) v
m
(y
T
, m
T
)m
T
.
dp
t
=
_
b
m
(t)p
t
+Q
t
f(t) + R
t
m
(t) + u
m
(t)
dt +Q
t
(t)dW
0
t
+R
t
dZ
t
(9)
p
T
= v
m
(y
T
, m
T
).
For a given ( e, c), I use the shorthand notation
b(t) =
b(t, y,
Z, m
t
, c
t
) and so on. These
adjoint variables followbackward stochastic differential equations (BSDEs), as they have
specied terminal conditions but unknown initial values.
16
BelowI showthat the adjoint
process q
t
associated with the density process
t
corresponds to the agents optimal util-
ity process, while the adjoint process p
t
associated with the scaled wealth x
t
characterizes
the agents shadow value in terms of marginal utility of the hidden state. Moreover,
these adjoint variables play a key role in characterizing the implementability of contracts
below.
In the following, I say that a process X
t
L
2
if E
_
T
0
X
2
t
dt < . My rst result
gives the necessary conditions for optimality. As with all later propositions, required
assumptions and proofs are given in Appendices A.2 and A.3, respectively.
Proposition 3.1. Suppose that Assumptions A.1, A.2, and A.3 hold, and that u also satis-
es Assumption A.2. Let (e
, c
, x
, c
t
, e
t
, c
t
,
t
, p
t
, Q
t
, R
t
) = max
(e,c)AB
H(t, y,
Z, m
t
, e, c,
t
, p
t
, Q
t
, R
t
). (10)
Suppose in addition that Aand B are convex and f and u are continuously differentiable in (e, c).
Then an optimal control (e
, c
t
, e
t
, c
t
,
t
, p
t
, Q
t
, R
t
) (e e
t
) 0 (11)
H
c
(t, y,
Z, m
t
, e
t
, c
t
,
t
, p
t
, Q
t
, R
t
) (c c
t
) 0
16
See El Karoui, Peng, and Quenez (1997) for an overview of BSDEs in nance. In particular, these BSDEs depend
on forward SDEs, as described in Ma and Yong (1999).
10 NOAH WILLIAMS
I stress that these are only necessary conditions for problem. As Mirrlees (1999) and
Rogerson (1985b) have shown, rst order conditions such as (11) may not be sufcient to
characterize an agents incentive constraints, and so the set of implementable contracts
may be smaller than that characterized by the rst order conditions alone. However, I
establish the validity of my rst-order approach in the next section.
Now I consider the simplications which result in the hidden action case. Here
t
is
the sole state variable, and thus I suppose that the agents preferences u and v dont de-
pend on m
t
. I also suppose that the agent just consumes his payment from the principal,
so I eliminate c
t
as well. These assumptions are simply made to make the notation more
concise. In this case, the Hamiltonian (7) simplies to:
H(t, y, e, ) =
f(t, y, e) + u(t, y, e), (12)
and the rst order condition (11) to:
f
e
(t, y, e, s) = u
e
(t, y, e, s). (13)
Thus there is a single adjoint process q
t
which simplies from (8) to:
dq
t
=
_
f(t) + u(t)
dt +
t
(t)dW
0
t
(14)
= u(t)dt +
t
(t)dW
e
t
q
T
= v(y
T
),
where the second line uses the change of measure for the optimal policy. This is a partic-
ularly simple BSDE whose solution is easily seen to be the following:
q
t
= E
e
__
T
t
u(t)dt +v(y
T
)
F
t
_
,
so that q
0
= V (e
_
T
0
u(t, y
t
, e
t
, s
a
t
)dt +
_
T
0
t
(t, y
t
, s
a
t
)dW
e
t
, (16)
for some q
0
V
0
.
Similarly, with the specied volatility
t
the Hamiltonian from (12) can be represented
explicitly in terms of the target effort e:
H
(t, e) = (t, y
t
, e
t
, s
a
t
)f(t, y
t
, e, s
a
t
) + u(t, y
t
, e, s
a
t
). (17)
I say a contract s
a
has the maximization property if for almost every (t, y
t
) [0, T] R
n
y
the
following holds almost surely:
H
(t, e
t
) = max
eA
H
) and ( p,
Q,
R)
of the adjoint equations (8)-(9). Thus, as in (15) above, each of the adjoint processes now
becomes a function of (t, y,
Z, m, e, c) under the contract. Thus I extend my representation
of the Hamiltonian (7) with these particular adjoint processes as in (17):
H
(t, m, e, c) = (
t
+
Q
t
m) f(t, y
t
, e, s
s
t
) + p
t
b(t, m, c, s
s
t
) (19)
+
R
t
(t, m) +u(t, y
t
, m, c, e, s
s
t
).
Then parallel to (16) above, if an acceptable contract respects the adjoint equations (8)-(9)
the terminal utility can be represented:
v(y
T
, m
T
) = q
T
+ p
T
m
T
= q
0
+ p
0
m
0
+
_
T
0
dq
t
+
_
T
0
d(pm)
t
(20)
= q
0
+ p
0
m
0
_
T
0
[(
t
+
Q
t
m
t
) f(t, y
t
, e
t
, s
s
t
) +u(t, y
t
, e
t
, c
t
, s
s
t
)]dt
+
_
T
0
[ p
t
(t, m
t
) + m
t
R
t
+
t
]dZ
t
+
_
T
0
[
t
+
Q
t
m
t
](t, y
t
, s
s
t
)dW
0
t
.
So now V ( e, c) = q
0
+ p
0
m
0
.
As above, I say a contract s
s
has the maximization property if for almost every (t, y
t
)
[0, T] R
n
y
the following holds almost surely:
H
(t, m
t
, e
t
, c
t
) = max
(e,c)AB
H
(t, m
t
, e, c). (21)
Nowthe maximization property effectively states that given the specied adjoint processes,
if the agent has the target wealth level then the target control satises his optimality
14 NOAH WILLIAMS
conditions. However I need an additional assumption to rule out a situation in which
the agent would accumulate a different amount of wealth and choose different actions.
Such possibilities are ruled out by the assumption that the Hamiltonian H
is concave in
(m, e, c). This is analogous to the condition which insures the sufciency of the maximum
principle in Zhou (1996).
Unfortunately, this concavity assumption is somewhat high-level, and is too strong to
handle typical formulations of hidden savings as discussed in Section 2.2 above. This
is driven by the lack of joint concavity in wealth and effort. Assuming the necessary
differentiability, it is easy to see that the Hessian of the H
Q
t
f
e
(t) +u
me
(t) (
t
+
Q
t
m
t
) f
ee
(t) +u
ee
(t)
_
,
where I use the same shorthand notation as above. In a standard hidden savings model, b
and are linear as in (3) above, and u is independent of m. Hence the upper left element
of the matrix is zero. It is clear to see that in this case this matrix typically fails to be
negative semidenite.
18
The only possibility would be to have
Q
t
0, in which case
the savings decision and effort decision would be completely uncoupled. But as long as
the payment to the agent depends on the realized output, as it of course typically will in
order to provide incentives, and so
Q
t
= 0. Thus at a minimum, I require that at least one
of the drift or diffusion of wealth or the preferences be strictly concave in m. Moreover
the curvature of the appropriate functions must be enough so that the determinant of the
Hessian matrix is non-negative.
Thus my results do not directly apply to standard hidden savings formulations, but
may cover other hidden state models. For example, if I suppose that agents have strictly
concave preferences over wealth then my results may apply to the hidden savings for-
mulation of Section 2.2. Alternatively, I may suppose that instead of having access to
hidden savings, the agent has access to home production with a strictly concave produc-
tion function. As discussed above, other applications would be models in which current
output depends on past effort choices, such as through an effort stock. If current out-
put were a strictly concave function of the past effort stock, the concavity requirement
for the Hamiltonian would be more easily satised. However, even when my sufcient
conditions fail, the methods can be used to derive a contract and then one can check ex-
post whether the contract is implementable. Werning (2001a) and Abraham and Pavoni
(2003) do this numerically in their models, and I pursue this strategy in the example
below (where it can be done analytically).
I now state my main result of this section, characterizing implementable contracts in
the hidden state case. The necessary conditions are those from the rst-order approach,
which become sufcient under the concavity assumption.
18
A similar problem in a related setting appears in Kocherlakota (2004a).
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 15
Proposition 4.2. Under the assumptions of Proposition 4.1, an implementable contract
s
s
[ e, c] S
s
in the hidden state case, with an interior target control ( e, c) and target wealth
m is (i) acceptable, (ii) respects the adjoint equations, and (iii) has the maximization property.
If in addition, the Hamiltonian function H
f(t) =
f(t, y, m, s, ,
Q, p) = f(t, y, e(t, y, m, s, ,
Q, p), s).
In (23) y
0
and m
0
are given, and q
0
and p
0
are to be determined subject to the constraints:
q
0
+p
0
m
0
= V ( e, c, s) V
0
, q
T
= v(y
T
, m
T
) v
m
(y
T
, m
T
)m
T
, p
T
= v
m
(y
T
, m
T
). (24)
Thus the principals problem is to solve:
sup
{s
s
S
s
}
J( s, e, c)
subject to (24), with the initial conditions for (y
0
, m
0
), and the evolution (23). In general,
this is a difcult optimization problem, which typically must be solved numerically. In
the next section I study an example which can be solved explicitly.
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 17
6. A FULLY SOLVED EXAMPLE
While my methods can in principle lead to a full characterization of the optimal con-
tracts in relatively general settings, they typically require numerical solutions in practice.
In this section I show that the full information, hidden action, and hidden savings ver-
sions of a model with exponential preferences and linear evolution is explicitly solvable.
This allows us to fully describe the optimal contract and its implementation, as well as
to directly characterize the distortions which are caused by the informational frictions.
6.1. An Example
I now consider a simple model in which the principal hires the agent to manage a
risky project, with the agents effort choice affecting the expected return on the project.
My s is a dynamic version of the model of Holmstrom and Milgrom (1987). In their
environment consumption and payments occur only at the end of the period and output
is i.i.d., while I include intermediate consumption (by both the principal and agent) and
allowfor persistence in underlying processes. In addition, mine is a closed system, where
effort adds to the stock of assets but consumption is drawn from it.
19
I also consider an
extension with hidden savings, in which the agent can save in a risk-free asset with a
constant rate of return. In this case, my results are related to the discrete time model of
Fudenberg, Holmstrom, and Milgrom(1990), who showthat the hidden savings problem
is greatly simplied with exponential preferences. Much of the complications of hidden
savings comes through the interaction of wealth effects and incentive constraints, which I
abstract from here. However my results are not quite as simple as theirs, as the principal
in my model is risk averse and the production technology is persistent.
20
I show that
hidden savings affects the contract by changing the effective rate of return that the agent
can access.
As in Holmstrom and Milgrom (1987), I assume that the principal and agent have
identical exponential utility preferences only over consumption, while the agent has
quadratic nancial costs of effort:
u(c, e) = exp
_
_
c
e
2
2
__
, U(d) = exp(d).
For simplicity, I consider an innite horizon version of the model, thus letting T
in my results above. The evolution of the asset stock or cumulative output is linear with
additive noise:
dy
t
= (Ay
t
+Be
t
c
t
d
t
) dt +dW
t
(25)
19
My preferences are also differ from Holmstrom and Milgrom (1987), as they consider time multiplicatively sepa-
rable preferences while I use time additively separable ones.
20
In their model Fudenberg, Holmstrom, and Milgrom (1990) show that there are no gains to long term contracting,
and that an optimal contract is completely independent of history. The rst result relies on the risk neutrality of the
principal, while the second relies on technology being history independent as well. Neither condition holds in my
model, and I nd that the optimal contract is history dependent and that hidden savings alter the contract.
18 NOAH WILLIAMS
Thus A represents the expected return on assets in the absence of effort (and before any
consumption), while B represents the productivity of effort. Since I consider an innite
horizon problem, it is easiest to work with the agents discounted utility process (still
denoted q
t
), which follows:
dq
t
= [q
t
u(c
t
, e
t
)]dt +dW
t
.
I now successively solve for the optimal contract when the principal has full informa-
tion, when the agents effort choice is hidden, and when the agents savings are hidden
as well.
6.2. Full Information
Although not explicitly considered above, the full information case can be analyzed
using my methods as well. I abuse notation slightly and let J(y, q) denote the principals
value function, where q = V
0
is the agents promised utility. With full information this
state variable only matters because of the participation constraint. Under the current
specication, I can write the principals Hamilton-Jacobi-Bellman (HJB) equation as:
J(y, q) = max
c,d,e,
_
exp(d) +J
y
(y, q) [Ay +Be c d] +J
q
(y, q)[q + exp((c e
2
/2))]
+
1
2
J
yy
(y, q)
2
+J
yq
(y, q)
2
+
1
2
J
qq
(y, q)
2
2
_
(26)
The rst order conditions for (d, c, e, ) are thus:
exp(d) = J
y
exp((c e
2
/2)) = J
y
/J
q
e exp((c e
2
/2)) = BJ
y
/J
q
= J
yq
/J
qq
Taking ratios of the conditions for (c, e) I get e = B.
Due to the exponential preferences and linear of the evolution, it is easy to verify that
the value function is the following:
J(y, q) =
J
0
q
exp(Ay),
where the constant J
0
is given by:
J
0
=
1
A
2
exp
_
2(A )
A
B
2
2
+
2
2
A
4
_
.
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 19
The optimal policies are thus:
e
fi
= B,
fi
(q) =
Aq
2
,
c
fi
(q) =
B
2
2
log A
log(q)
,
d
fi
(y, q) =
log(J
0
A)
+
log(q)
+Ay.
The agents optimal effort is constant and his consumption does not depend directly
on output, but instead is linear in the log of the utility process. The principals consump-
tion is linear in the log of the agents utility process and also linear in current output.
These policies imply that the state variables evolve as follows:
dy
t
=
_
2(A )
A
+
2
A
4
_
dt +dW
t
dq
t
= ( A)q
t
dt
A
2
q
t
dW
t
.
Thus output follows an arithmetic Brownian motion with constant drift, while the utility
process follows a geometric Brownian motion. The expected growth rate of utility is
constant and equal to the difference between the subjective discount rate and the rate
of return parameter A.
6.3. The Hidden Action Case
I now turn to the case where the principal cannot observe the agents effort e
t
. Note
that the agents preferences are not additively separable in consumption and effort, and
thus the sufcient conditions from Corollary 4.1 above are not satised. However it is
straightforward to verify that the agents Hamiltonian is concave in (c, e) so that Propo-
sition 4.1 holds. Since the technology is linear, this is simply a consequence of the con-
cavity of preferences. Therefore the agents effort level is determined by his rst order
condition, which from (13) specialized to this model is:
B = e exp((c e
2
/2)). (27)
The principal must now choose contracts which are consistent with this effort choice by
the agent. The principals HJB equation now becomes:
J(y, q) = max
c,d,e
_
exp(d) +J
y
(y, q) [Ay +Be c d] + J
q
(y, q)[q + exp((c e
2
/2))]
+
1
2
J
yy
(y, q)
2
+J
yq
(y, q)(c, e)
2
+
1
2
J
qq
(y, q)(c, e)
2
2
_
(28)
20 NOAH WILLIAMS
where I substitute = (c, e) using (27). The rst order conditions for (d, c, e) are then:
exp(d) = J
y
,
J
y
J
q
exp((c e
2
/2)) J
yq
2
(c, e) J
qq
2
(c, e)
2
= 0,
J
y
B +J
q
e exp((c 1/2e
2
)) +J
yq
2
1 +e
2
e
(c, e) +J
qq
2
1 +e
2
e
(c, e)
2
= 0.
A special feature of this example is that the value function and the optimal policies
take the same form as the full information case, albeit with different key constants. In
particular, the value function is of the same form as above,
J(y, q) =
J
1
q
exp(Ay)
for some constant J
1
. The optimal policies are thus:
e
ha
= e
,
ha
(q) =
e
kq
B
,
c
ha
(q) =
(e
)
2
2
log k
log(q)
,
d
ha
(y, q) =
log(J
1
A)
+
log(q)
+Ay,
where (e
, k, J
1
) are constants.
Thus effort is again constant, consumption is again linear in the log of the utility
process, and the principals consumption is also linear in output. Note also that u(c(q), e
) =
kq. Using this along with the form of the value function in the rst order conditions for
(e, c) determines the equations that e
k/B 2
2
2
(e
)
2
k
2
/B
2
= 0, (29)
BA +e
k
2
A(1 +(e
)
2
)k/B 2
2
e
(1 +(e
)
2
)k
2
/B
2
= 0.
These can be solved to get e
(k):
e
=
B
3
A +
2
BAk
B
2
A + 2
2
k
2
, (30)
then substituting this back into the rst equation in (29) gives an expression for k. Notice
that for = 0 these collapse to the full information solution e
= B and k = A, which
of course they should since with no noise the agents action is no longer hidden. The
constant J
1
is determined fromthe HJB equation after substituting in the optimal policies,
which implies:
J
1
=
1
kA
exp
_
A +k 2
A
e
_
B
e
2
_
+
2
2
_
A
2
e
k
B
+
(e
)
2
k
2
AB
2
__
.
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 21
Under the optimal contract, the evolution of the states is also similar to the full informa-
tion case, now being:
dy
t
=
_
(A +k 2)
A
+
2
_
A
2
e
k
B
+
(e
)
2
k
2
AB
2
__
dt +dW
t
dq
t
= ( k)q
t
dt
e
k
B
q
t
dW
t
.
So again output follows a Brownian motion with drift, while the utility process follows
a geometric Brownian motion.
While the form of the policy functions is the same as in the full information case, the
constants dening them differ. Solving for the values of the constants is a simple nu-
merical task, but explicit analytic expressions are not available. To gain some additional
insight into the optimal contract, I expand e
and k in
2
around zero. From (29) and (30)
I have the following approximations:
e
= B
2
A
B
+o(
4
)
k = A
2
A
2
2
+o(
4
).
In turn, substituting these approximations into c
ha
(q) gives:
c
ha
(q) = c
fi
(q) + o(
4
).
Thus the rst order effects (in the shock variance) of the information frictions are a re-
duction in effort put forth, but no change in consumption. I also show below that k is
the agents effective rate of return, and thus this return decreases with more volatility.
Moreover, the effect on effort depends on the parameters in a simple way: the higher
the rate of return parameter A or risk aversion parameter the larger is the reduction,
while larger productivity values B lead to smaller reductions in effort. Below I plot the
exact solutions for a parameterized version of the model and show that the results are in
accord with these rst order asymptotics. The information friction leads to a reduction
in effort, but little effect on consumption.
6.4. The Hidden Saving Case
When the agent has access to hidden savings, some of the analysis is altered. In par-
ticular, I must now distinguish between the principals payment s
t
and the agents con-
sumption c
t
. Thus the evolution of the asset stock in the project y
t
and the agents wealth
m
t
are:
dy
t
= (Ay
t
+Be
t
s
t
d
t
) dt +dW
t
dm
t
= (
Am
t
+s
t
c
t
)dt.
Thus I assume that the agent can borrow and lend at a constant risk-free rate
A A. I
mostly focus on the case when
A = A, in which case the agents saving is redundant, as
22 NOAH WILLIAMS
all that matters are total assets y
t
+m
t
. Without loss of generality I can have the principal
do all the saving, and thus have the target m
t
0 and so c
t
= s
t
.
21
The agents Hamiltonian is now:
H = ( +Qm)(Ay +Be s d) +p(
Am+s c) +u(c, e),
and thus his optimality conditions are:
( +Qm)B = e exp((c e
2
/2))
p = exp((c e
2
/2)).
For the same reasons as discussed above, the agents Hamiltonian is not concave in
(m, e, c) and so the conditions of Proposition 4.2 fail, and I cannot be sure the rst-order
approach is valid here. However I will assume that it is valid for the time being, and will
verify below that the optimal contract I derive is indeed implementable. Note that the
agents optimality conditions determine the following functions:
(e, p) =
ep
B
c(e, p) =
e
2
2
log(p/)
.
Thus I also see that u(c(e, p), e) = p/. The evolution of the discounted marginal utility
state, i.e. the discounted version of (9) in this example, is simply:
dp
t
= (
A)p
t
dt +Q
t
dW
t
.
Note that p follows a BSDE which in my innite horizon problem does not have a
binding terminal condition.
22
Thus the principal is free to choose the initial value p
0
under the contract. For a given p, the principals value function J(y, q, p) solves the HJB
equation:
J = max
d,e,Q
_
exp(d) + J
y
[Ay +Be c(e, p) d] +J
q
[q +p/] +J
p
(
A)p
+
2
_
1
2
J
yy
+J
yq
(e, p) +J
yp
Q+
1
2
J
qq
(e, p)
2
+J
qp
(e, p)Q+
1
2
J
pp
Q
2
__
(31)
where I substitute = (e, p) and c = c(e, p) and suppress arguments of J. Then assum-
ing that the value function is concave in p, the optimal choice of the initial condition p
0
is
determined by:
J
p
(y, q, p
0
) = 0.
21
Note that this relies on risk being additive. Otherwise varying m may affect the risk of the total asset stock y +m,
and the principal would face a portfolio problem. Also note that even with additive risk, if
A < A there may be an
incentive for the principal to have the agent accumulate debt, if it were feasible.
22
In nite horizon problem the terminal condition would be pinned down via the marginal utility of a terminal
payoff.
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 23
Based on my results in the previous section and the proportionality of utility and
marginal utility with exponential preferences, I now guess that the value function can be
written in the form:
J(y, q, p) =
F(p/q)
q
exp(Ay)
for some univariate function F. To simplify notation, I dene z = p/q as the argument
of this function, noting that z (, 0). Thus the optimality condition for p
0
translates
to F
(z
0
) = 0, or p
0
= (F
)
1
(0)q
0
. Thus the initial marginal utility is proportional to the
initial promised utility.
Then I use the guess in the HJB equation (31), take the rst order condition for the
dividend, and obtain a policy function similar to the hidden action case:
d
hs
(y, q, z) =
log(F(z)A)
+
log(q)
+Ay.
For the volatility of marginal utility, I nd it convenient to normalize Q
t
=
Q
t
p
t
. Then
the optimal choices of e and
Q are determined by the rst order conditions:
AF(z)(B e) +
_
F
(z)z
2
+ 4F
(z)z + 2F(z)
_
2
z
2
e
B
2
+A(F
(z)z +F(z))
2
z
B
(2F
(z) +F
(z)z)
Q
2
z
2
B
= 0, (32)
F
(z)z
QAF
(z) (2F
(z) +F
(z)z)
ez
B
= 0. (33)
Thus given F, these equations determine the optimal policies as a function of z alone:
e
hs
(z) and
Q(z). Substituting these into (31), I nd that F is the solution of the second
order ODE:
F(z) = AF(z) (F
_
+(
A)zF
(z) + (F
(z)z
2
+ 4F
(z)z + 2F(z))
2
e
hs
(z)
2
z
2
2B
2
+ (F
(z)z +F(z))
2
Ae
hs
(z)z
B
+
2
_
2
A
2
2
F(z) +F
(z)z
2
Q(z)
2
2
AF
(z)z
Q(z) (2F
(z) + F
(z)z
2
)
e
hs
(z)
Q(z)z
2
B
_
,
with the boundary conditions lim
z0
F(z) = and lim
z
F(z) = . Note that these
conditions imply that the principals utility falls without bound as the agents current
consumption or future promised utility increase without bound.
Thus I have reduced the solution of the three dimensional PDE for the value function
to an ODE. The ODE is sufciently complex that an explicit solution for arbitrary z does
not seem feasible. However, I can obtain more explicit results for the value and the
24 NOAH WILLIAMS
policies at the optimal starting value z
0
. In particular, note that at z
0
(33) implies
Q(z
0
) =
e
hs
(z
0
)z
0
B
and thus (32) leads to:
e
0
= e
hs
(z
0
) =
B
3
A
2
BAz
0
B
2
A + 2
2
(z
0
)
2
/
.
Then using these in the ODE we get:
F(z
0
) =
Az
0
exp
_
A z
0
2
A
e
0
_
B
e
0
2
_
+
2
2
_
A
2
+
e
0
z
0
B
+
(e
0
z
0
)
2
AB
2
__
.
Note that if z
0
= k then these results agree with the hidden action case, as e
hs
(k) =
e
0
) and
Q(z
0
) to solve for z
0
. However for
A = k I cannot analytically
solve for the optimal starting value, and so I numerically solve the ODE for F(z), then
choose z
0
as the minimizing value.
23
For the parametrization below, I set
A = A and nd
that z
0
= A to sufcient numerical precision. I have also veried the same result in
alternative parameterizations with
A k. In all of these cases the agent has a higher
return on savings than his effective return from the project. Thus it appears that at least
for these cases, the optimal choice is z
0
=
A.
These results are also signicant, since they imply that z
t
is in fact constant over time,
so that utility and marginal utility remain proportional forever under the contract. This
can be seen by using Itos lemma to get the evolution of z
t
:
dz
t
=
_
(z
t
/ +
A)z
t
+
2
e
hs
(z
t
)
2
z
2
t
B
_
Q(z
t
)
e
hs
(z
t
)z
t
B
__
dt+z
t
_
Q(z
t
)
e
hs
(z
t
)z
t
B
_
dW
t
.
Using the expression for
Q(z
0
) above, note that dz
t
= 0 if z
0
=
A. Thus in this case it
turns out that the contract does not need to condition on the additional marginal utility
state. These results are clearly a consequence of the absence of wealth effects, and thus
special to exponential utility.
6.5. Comparing the Dierent Cases
Table 1 summarizes the optimal contracts under full information, hidden actions, and
hidden savings (assuming z
0
=
A). As weve seen, the policy functions bear a strong
resemblance to each other. In each case, effort is constant, consumption is linear in the log
23
These calculations are straightforward but lengthy and thus are not included here. Details can be furnished upon
request. For
A = k when differentiating the ODE for F there is a term in F
(z
0
) which cancels in the expression,
allowing for explicit solution for z
0
. But for arbitrary
A the term does not cancel.
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 25
TABLE 1.
Comparison of the policies for the optimal contracts under full information, hidden actions, and hidden savings.
Full Information Hidden Action Hidden Saving (z
0
=
A)
Effort, e B e
(k) =
B
3
A+
2
BAk
B
2
A+2
2
k
2
e
(
A) =
B
3
A+
2
BA
A
B
2
A+2
2
A
2
Consumption, c
B
2
2
log A
log(q)
(k)
2
2
log k
log(q)
(
A)
2
2
log
A
log(q)
Dividend, d
log(J
0
A)
+
log(q)
+Ay
log(J
1
A)
+
log(q)
+Ay
log(F(
A)A)
+
log(q)
+Ay
of the promised utility q, and the principals dividend is linear log(q) and the current
assets y. The consumption policies also depend negatively on the agents effective rate
of return on assets, which is A with full information, k with hidden actions, and
A with
hidden savings. As weve already seen, the main effect of the hidden action case relative
to full information is the reduction of effort. Weve also seen that to rst order there is no
effect on consumption. This can be seen in the consumption policies, as effort falls but so
does the effective return as k < A, leading to at least partially offsetting effects.
The main difference between the hidden saving and the hidden action cases is the
effective rate of return on the agents savings. When
A = k then the optimal policies in
the hidden action and hidden savings cases coincide. With
A > k, determining exactly
whether the effort increases or decreases under hidden savings relative to the hidden
action case is complicated by the fact that I do not have an explicit expression for k.
However if A < 4k then effort is decreasing in k, and hence consumption is as well.
24
Since for small shocks weve seen that k is smaller than A by a term proportional to
the shock variance, this will certainly hold for small shocks. Thus for small enough ,
with hidden savings the agent puts forth less effort and consumes less than with hidden
actions alone. This is clear since c
ha
(q; k) is decreasing in k if e
(k), and e
hs
(z
0
) = e
(A)
versus , while the right panel plots c
fi
, c
ha
, and c
hs
(all evaluated at q = 1) versus .
Clearly as 0 the cases all agree, as there is no information friction. Compared to full
information, both effort and consumption fall under hidden actions. Moreover, for small
the approximation results in the hidden action case above are accurate, as effort falls
but consumption is relatively unaffected. When the agent has access to hidden savings,
24
Simple calculations give
de
(k)
dk
=
2
B
3
A(A 4k) 2
4
BA
2
k
(B
2
A+ 2
2
k
2
)
2
,
which is clearly negative if A < 4k.
26 NOAH WILLIAMS
0 1 2 3
0.3
0.35
0.4
0.45
0.5
e
f
i
,
e
h
a
,
e
h
s
Effort
FI
HA
HS
0 1 2 3
0.98
1
1.02
1.04
1.06
1.08
Agent Consumption (q=1)
c
f
i
,
c
h
a
,
c
h
s
FIGURE 1. The agents effort and consumption policies for different noise levels .
consumption and effort fall further. In addition, these effects are all monotone in the
level of noise .
I can also use the policy functions to provide explicit expressions for the inefciency
wedges, discussed by Kocherlakota (2004b) and others. These measure how the infor-
mation frictions lead the consumption and labor allocations to be distorted for incentive
reasons. In particular, suppose that the agent can borrow and lend at the same risk free
rate A as the principal. When the agents saving is observable, the principal is able to
tax it and drive its return down to
A = k. However if the principal cannot observe the
agents savings, then of course he cannot tax it and so
A = A. Thus in the hidden action
(but observable saving) case, the contract introduces an intertemporal wedge
K
, a gap
between the intertemporal marginal rate of substitution and the marginal return on as-
sets. This is simply given by the tax rate which drives the after-tax rate of return down
to k:
K
(k) = 1
k
A
.
Of course, under hidden savings
K
(A) = 0.
By varying the payment to the agent for incentive reasons, the optimal contract also
induces a wedge between the agents marginal productivity of labor and his marginal
rate of substitution between consumption and effort, a labor wedge
L
. In my example,
since the marginal product of effort is simply B and the marginal rate of substitution is
e
(
A), the labor wedge is simply:
L
(
A) = 1
e
(
A)
B
.
Thus both the intertemporal and labor wedges are constant and (negatively) propor-
tional to the key terms discussed above. In Figure 2 I plot the intertemporal and labor
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 27
0 1 2 3
0
0.1
0.2
0.3
0.4
0.5
L
Labor Wedge
0 1 2 3
0
0.05
0.1
0.15
Intertemporal Wedge
K
HA
HS
FIGURE 2. The labor and intertemporal wedges for different noise levels .
0 1 2 3
0
0.005
0.01
0.015
0.02
0.025
0.03
Reduction in Principal Dividend
HA
HS
FIGURE 3. Reduction in the principals dividend relative to the full information case for different noise levels .
wedges in the hidden action and hidden savings cases. The labor wedge is especially sig-
nicant for this parameterization, as for > 1.5 it is comparable to labor income tax rates
of more than 30% under hidden actions and roughly 40% under hidden savings. The in-
tertemporal wedge is smaller, of course being identically zero under hidden savings and
attening out near an effective tax rate of 13% under hidden actions.
Finally, I provide a measure of the cost of the information frictions for the principal.
As Table 1 shows, the different informational assumptions affect the principals dividend
only through the additive constants. Thus for each level of promised utility q and assets y,
the principals consumption differs by a constant amount depending on the information
structure. In Figure 3 I plot the reduction relative to the full information case in the
level of the principals dividend under hidden actions and hidden savings. The costs
are relatively low, attening out near 0.025 units of consumption under hidden actions
28 NOAH WILLIAMS
and 0.03 units under hidden savings. These numbers are somewhat difcult to interpret,
as the levels are somewhat arbitrary since output is growing over time. As an example,
with y = 0 and q = 1 the dividend is exactly equal to the constant term, and these level
reductions imply a 1.5-2% fall in the principals dividend. Of course with greater output
and lower levels of promised utility, the proportional decline is much lower.
6.6. Implementing the Optimal Allocations
Thus far I have focused on the direct implementation of contracts, with the princi-
pal directly assigning consumption to the agent. However in this section I show how
the same outcomes from the optimal contracts can be achieved by giving the agent a
performance-related payment but then allowing him to invest in a risk-free asset, and
thus to choose his own consumption.
As in the hidden state formulation above, I now distinguish between consumption c
t
and the principals payment to the agent s
t
. The payment is set at the agents optimal
consumption under the contract, so s
t
= s(q
t
;
A) = c
ha
(q
t
;
A), where q
t
is the agents
utility process under the contract and
A = k in the hidden action case and
A = A in
the hidden savings case. I now directly verify that the optimal contract, expressed as a
payment s(q
t
;
A) which in turn embodies an associated target effort e
(
A) and evolution
for q
t
and a tax rate
K
(
A) = 1
A/A on saving, is implementable. Of course weve
already shown that contract is directly implementable in the hidden action case. But
recall that in the hidden saving case my sufcient conditions guaranteeing the validity
of the rst order approach failed, and thus it is important to know that the contract
I deduced is indeed implementable. Moreover, this is a natural form of the contract
specication.
For the agent, the state q
t
is simply part of the specication of the payment under the
contract. From his vantage point, it evolves as:
dq
t
= (
A)q
t
dt
e
(
A)
A
B
q
t
dW
e
t
= (
A)q
t
dt
e
(
A)
A
B
q
t
_
dy
t
(Ay
t
+Be
(
A) s(q
t
;
A) d
t
)dt
_
=
_
A e
(
A)
A(e
t
e
(
A))
_
q
t
dt
e
(
A)
A
B
q
t
dW
t
.
Here I use the fact that dW
e
t
is the driving Brownian motion under the optimal contract
for the principals information set. Moreover the agent begins with zero wealth m
0
= 0,
which then evolves as:
dm
t
=
_
[1
K
(
A)]Am
t
+s(q
t
;
A) c
t
_
dt
=
_
Am
t
+
e
(
A)
2
2
log(
A)
log(q
t
)
c
t
_
dt
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 29
Thus the agents value function V (q, m) solves the HJB equation:
V (q, m) = max
c,e
_
exp((c e
2
/2)) +V
m
(q, m)
_
Am +s(q;
A) c
_
+V
q
(q, m)q[
A e
(
A)
A(e
t
e
(
A))] +
1
2
V
qq
(q, m)q
2
2
e
(
A)
2
A
2
B
2
_
(34)
It is easy to verify that the agents value function is given by V (q, m) = q exp(
Am).
Substituting this into the HJB equation, and taking rst order conditions for (c, e) gives:
exp((c e
2
/2)) = V
m
=
Aq exp(
Am),
e exp((c e
2
/2)) = e
(
A)
AqV
q
= e
(
A)
Aq exp(
Am).
Taking ratios of the two equations gives e = e
(
A)
2
2
log
A
log(q)
+
Am.
Substituting this into the wealth equation gives dm
t
= 0. Thus, if the agent begins with
m
t
= 0, then he will remain at zero wealth, will consume the optimal amount under
the contract c = c
ha
(q;
A), and will attain the value V (q, 0) = q. Therefore the policies
associated with the optimal contracts can be implemented with this payment and savings
tax scheme.
7. CONCLUSION
In this paper I have established several key results for dynamic principal-agent prob-
lems. By working in a continuous time setting, I were able to take advantage of powerful
results in stochastic control, which led to some sharp conclusions. I characterized the
class of implementable contracts in dynamic settings with hidden actions and hidden
states via a rst-order approach. I also provided the rst general proof of the validity of
this rst-order approach in a dynamic environment. I showed that implementable con-
tracts must respect some additional state variables: the agents utility process and the
agents shadowvalue (in marginal utility terms) of the hidden states. I then developed
a constructive method for solving for an optimal contract. The optimal contract is in gen-
eral history dependent, but can be written recursively in terms of the state and the dual
adjoint variables associated with the agents utility and shadow value processes.
In our application, we show that as in Holmstrom and Milgrom (1987) the optimal
contract is linear in our setting. However now the payment is linear in an endogenous
object, the logarithm of the agents promised utility under the contract. Moreover, we
show that the main effect of hidden actions is in the distortion of effort, along with a
smaller effect on the agents implicit rate of return under the contract. Introducing hid-
30 NOAH WILLIAMS
den savings as well eliminates this second distortion, and thus increases the effort dis-
tortion.
Overall, the methods developed in the paper are tractable and reasonably general,
making a class of models amenable to analysis. As mentioned in the introduction, there
has been a renewed interest in dynamic models with hidden information in the past few
years. This provides much potential scope for applications of my results.
APPENDIX
A.1. DETAILS OF THE CHANGE OF MEASURE
I nowintroduce more technical detail associated with the change of measure used in Section 2.3. I start by working
with the induced distributions on the space of continuous functions, which I take to be the underlying probability
space. Thus I let the sample space be the space C
n
y
+n
z
, and let (W
0
t
, Z
t
) = (t) be the family of coordinate
functions, and F
0
t
= {(W
0
t
, Z
t
) s t} the ltration generated by (W
0
t
, Z
t
). I let P be the Wiener measure on (, F
0
T
),
and let F
t
be the completion of F
0
t
with the null sets of F
0
T
. This denes the basic (canonical) ltered probability space,
on which I have the Brownian motions (W
0
t
, Z
t
). Later I will construct another probability measure in which W
0
t
is
replaced with a different Brownian motion, but as contracts are conditioned on Z
t
I keep it xed. I now introduce
some regularity conditions on the diffusion coefcient in (1) following Elliott (1982).
Assumptions A.1. Denote by
ij
(t, y, s) the (i, j) element of the matrix (t, y, s). Then I that for all i, j, for some
xed K independent of t and i, j:
1.
ij
is continuous,
2. |
ij
(t, y, s)
ij
(t, y
, s
)| K (|y y
| + |s s
|),
3. (t, y, s) is non-singular for each (t, y, s) and |(
1
(t, y, s))
ij
| K.
Of these, the most restrictive is perhaps the nonsingularity condition, which requires that noise enter every part
of the state y. Under these conditions, there exists a unique strong solution to the stochastic differential equation:
dy
t
= (t, y,
Z)dW
0
t
, (A.1)
with y
0
given. I can interpret this as a benchmark evolution of output under an effort policy e
0
which makes
f(t, y,
Z, e
0
t
) =
0 at each date. Different effort choices alter the evolution of output by changing the distribution over outcomes in C
n
y
.
Now I state some required regularity conditions on the drift function f.
Assumptions A.2. I assume that for some xed K independent of t:
1. f is continuous,
2. |f(t, y, e, s)| K(1 + |y| + |s|).
Note that these imply the predictability, continuity, and linear growth conditions on the concentrated function
f
which are assumed by Elliott (1982). Then for e A I dene the family of F
t
-predictable processes:
t
( e) = exp
_
t
0
1
(v, y,
Z)
f(v, y,
Z, e
v
)dW
0
v
1
2
_
t
0
|
1
(v, y,
Z)
f(v, y,
Z, e
v
)|
2
dv
.
Under the conditions above,
t
is an F
t
-martingale (as the assumptions insure that Novikovs condition is satised)
with E[
T
( e)] = 1 for all e A. Thus by the Girsanov theorem, I can dene a new measure P
e
via:
dP
e
dP
=
T
( e),
and the process W
e
t
dened by:
W
e
t
= W
0
t
_
t
0
1
(v, y,
Z)
f(v, y,
Z, e
v
)dv
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 31
is a Brownian motion under P
e
. Thus from (A.1), its clear that the state follows the SDE:
dy
t
=
f(t, y,
Z, e
t
)dt + (t, y,
Z)dW
e
t
. (A.2)
Hence each effort choice e results in a different Brownian motion. Note that
t
dened above (suppressing e) satises
t
= E[
T
|F
t
], and thus is the relative density process for the change of measure.
A.2. ASSUMPTIONS FOR RESULTS
Here I list some additional regularity conditions for my results. The rst is a basic differentiability assumption
used for the agents problem.
Assumption A.3. The functions (u, v, b, ) are continuously differentiable in m.
The next set of assumptions guarantees that the maximization property is satised.
Assumptions A.4. Suppose that u is separable in e and c and concave in (e, c), f is concave in e and b is concave
in c, and that u
e
f
e
0 and u
c
b
c
0.
The following set of assumptions provide some regularity conditions which are needed for the principals problem
in the hidden action case. I impose the conditions on the composite functions
f and u (which clearly imply conditions
on f and e as well as on e), as well as the principals preference functions U and L. Recall that in this case U is
independent of c and m.
Assumptions A.5. For some xed K independent of t I assume:
1.
f, , u, U, and L are continuous, and they are continuously differentiable with respect to all variables except possibly t.
2. The derivatives of
f, , and u are bounded.
3. The derivatives of U are bounded by K(1 + |y| + |s|), and the derivative of L is bounded by K(1 + |y|).
Further extensions of Assumptions A.5 are necessary for the principals problem in the hidden state case.
Assumptions A.6.
1. The functions
p
and
q
are continuous, and are continuously differentiable with respect to all variables except possibly t.
The derivatives of
p
and
q
are bounded.
2. The derivatives of U are bounded by K(1 + |y| + |m| + |s|), and the derivative of L is bounded by K(1 + |y| + |m|).
A.3. PROOFS OF RESULTS
Proof (Proposition 3.1). This follows by applying the results in Bismut (1973)-(1978) to my problem. Under the
assumptions, the maximum principle applies to the system with (, x) as the state variables. I now illustrate the
calculations leading to the Hamiltonian (7) and the adjoint equations (8)-(9). I rst dene the stacked system:
X
t
=
_
t
x
t
_
,
t
=
_
q
t
p
t
_
,
t
=
_
t
t
Q
t
R
t
_
.
Then note from (5)-(6) that X
t
satises (suppressing arguments):
dX
t
=
t
_
0
b
_
dt +
t
_
1
f 0
m
t
1
f
__
dW
0
t
dZ
t
_
= M(t)dt + (t)[dW
0
t
, dZ
t
]
H = M + tr(
) + u = H,
32 NOAH WILLIAMS
where H is from (7). As 0, the maximum condition (10) is the same whether I take H or
H as the Hamiltonian.
The adjoint variables evolve as:
d
t
=
H
X
(t)dt +
t
[dW
0
t
, dZ
t
]
T
=
(
T
v(y
T
, m
T
))
X
T
.
By carrying out the differentiation and simplifying I arrive at (8)-(9).
Proof (Proposition 4.1). The result is an extension of Theorem 4.2 in Schattler and Sung (1993) to my environment.
The necessity of the conditions follow directly from my results above: if the contract is implementable then it clearly
must be acceptable, and by Proposition 3.1 it must respect the adjoint equation and have the minimization property.
To show the converse, I must verify that e is an optimal control when the agent faces the contract s
a
[ e]. Recall that
the expected utility from following e is given by V ( e) = q
0
. Then from the results above, for any e A the following
holds:
V ( e) V ( e) = E
e
_
T
0
[u(t, y
t
, e
t
, s
a
t
) u(t, y
t
, e
t
, s
a
t
)]dt +
_
T
0
t
(t, y
t
, s
a
t
)dW
e
t
= E
e
_
T
0
[u(t, y
t
, e
t
, s
a
t
) u(t, y
t
, e
t
, s
a
t
)]dt +
_
T
0
t
(t, y
t
, s
a
t
)dW
e
t
+E
e
_
T
0
t
[f(t, y
t
, e
t
, s
a
t
) f(t, y
t
, e
t
, s
a
t
)]dt
= E
e
_
T
0
[H
(t, e
t
) H
(t, e
t
)]dt +
_
T
0
t
(t, y
t
, s
a
t
)dW
e
t
E
e
_
T
0
t
(t, y
t
, s
a
t
)dW
e
t
= 0.
Here the rst equality uses (16), the second equality uses the denitions of the change of measure between W
e
and
W
e
, the third equality uses the denition of the H
e
(t, m
t
, e
t
, c
t
) = H
c
(t, m
t
, e
t
, c
t
) = 0.
Thus by the concavity of H
, for any (m
t
, e
t
, c
t
) we get:
H
(t, m
t
, e
t
, c
t
) H
(t, m
t
, e
t
, c
t
) H
m
(t, m
t
, e
t
, c
t
), m
t
m
t
. (A.3)
I extend the shorthand notation above to now write b(
t) = b(t, m
t
, c
t
, s
s
t
) and so on. Then I dene
t
= m
t
m
t
,
and note that I can write its evolution as:
d
t
= [b
m
(
t)
t
+
t
]dt + [
m
(
t)
t
+
t
]dZ
t
, (A.4)
with
0
= 0 where I dene:
t
= b
m
(
t)
t
+b(t) b(
t)
t
=
m
(
t)
t
+(t) (
t).
Then I have the following duality relationship between p
t
in (9) for the target policy and
t
in (A.4), which is easily
veried via Itos lemma (see Zhou (1996)):
E
e
_
T
0
[u
m
(
t) +
Q
t
(f(t) f(
t))]
t
dt +v
m
(
T)
T
= E
e
_
T
0
[ p
t
t
+
R
t
t
]dt
,
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 33
where the term in
Q
t
comes from the change in measure from the target control policy e to an arbitrary admissible
policy e. Hence by the concavity of v I have:
E
e
[v(T) v(
T)] E
e
_
T
0
[ p
t
t
+
R
t
t
[u
m
(
t) +
Q
t
(f(t) f(
t))]
t
]dt
(A.5)
Then I proceed as in Proposition 4.1 and note that for any ( e, c) A I have:
V ( e, c) V ( e, c) = E
e
_
T
0
[u(t) u(
t)]dt +
_
T
0
[ p
t
(
t) + m
t
R
t
+
t
]dZ
t
+E
e
_
T
0
[
t
+
Q
t
m
t
](t)dW
e
t
+v(T) v(
T)
= E
e
_
T
0
[u(t) u(
t) + (
t
+
Q
t
m
t
)(f(t) f(
t))]dt
+E
e
_
T
0
[
t
+
Q
t
m
t
](t)dW
e
t
+v(T) v(
T)
E
e
_
T
0
[u(t) u(
t) + [
t
+
Q
t
m
t
]f(
t)]dt
+E
e
_
T
0
[ p
t
t
+
R
t
t
[u
m
(
t) +
Q
t
(f(t) f(
t))]
t
]dt
= E
e
_
T
0
[u(t) u(
t) u
m
(
t)
t
+ (
t
+
Q
t
m
t
)(f(t) f(
t))]dt
+E
e
_
T
0
p
t
[b(t) b(
t) b
m
(
t)
t
] +
R
t
[(t) (
t)
m
(
t)
t
]
dt
= E
e
_
T
0
[H
(t) H
t) H
m
(
t)
t
]dt
0.
Here the rst equality uses (20), the second equality uses the denitions of the change of measure between P
e
and P
e
along with the martingale property of the stochastic integral with respect to Z (which coincides under the two mea-
sures). The rst inequality uses (A.5), while the next equality uses the denitions of
t
and
t
. The following equality
uses the denition of H
function in (19), and the nal result follows from its concavity as in (A.3). Thus since ( e, c)
was arbitrary, we get that ( e, c) is an optimal control, and the contract is thus implementable.
Proof (Corollary 4.1 and 4.2). In either case, the stated assumptions insure that H
is concave in e and H
is concave in (e, c). This is clear from (15) in the hidden action case. The parallel rst order conditions give the
result in the hidden state case, where the separability insures that I can simply look at the separate rst order con-
ditions. But by assumption, each function has a stationary point on the feasible set, which is then a maximum.
References
Abraham, A. and N. Pavoni (2003). Efcient Allocations with Moral Hazard and Hid-
den Borrowing and Lending. Working paper, University College London.
Abreu, D., D. Pearce, and E. Stacchetti (1986). Optimal Cartel Equilibria with Imperfect
Monitoring. Journal of Economic Theory 39, 251269.
Abreu, D., D. Pearce, and E. Stacchetti (1990). Toward a Theory of Discounted Repeated
Games with Imperfect Monitoring. Econometrica 58, 10411063.
Allen, F. (1985). Repeated Principal-Agent Relationships with Borrowing and Lending.
Economics Letters 17, 2731.
34 NOAH WILLIAMS
Bismut, J. M. (1973). Conjugate Convex Functions in Optimal Stochastic Control. Jour-
nal of Mathematical Analysis and Applications 44, 384404.
Bismut, J. M. (1975). Growth and the Optimal Intertemporal Allocation of Risks. Journal
of Economic Theory 10, 239257.
Bismut, J. M. (1978). Duality Methods in the Control of Densities. SIAM Journal on
Control and Optimization 16, 771777.
Brock, W. A. and M. J. Magill (1979). Dynamics Under Uncertainty. Econometrica 47,
843868.
Cadenillas, A., J. Cvitani c, and F. Zapatero (2005). Dynamic Principal-Agent Problems
with Perfect Information. Working paper, USC.
Chari, V. V., P. J. Kehoe, and E. McGrattan (2004). Business Cycle Accounting. NBER
Working Paper No. 10351.
Cole, H. L. and N. Kocherlakota (2001). Efcient Allocations with Hidden Income and
Hidden Storage. Review of Economic Studies 68, 523542.
Cvitani c, J., X. Wan, and J. Zhang (2006). Continuous-Time Principal-Agent Problems
with Hidden Action and Lump-Sum Payment. Working paper, USC.
Detemple, J., S. Govindaraj, and M. Loewenstein (2005). Wealth-Robust Intertemporal
Incentive Contracts. Working paper, Boston University.
Doepke, M. and R. M. Townsend (2006). Dynamic Mechanism Design with Hidden
Income and Hidden Storage. Journal of Economic Theory 126, 235285.
El Karoui, N., S. Peng, and M. C. Quenez (1997). Backward Stochastic Differential
Equations in Finance. Mathematical Finance 7, 171.
Elliott, R. J. (1982). Stochastic Calculus and Applications. New York: Springer-Verlag.
Fernandes, A. and C. Phelan (2000). A Recursive Formulation for Repeated Agency
with History Dependence. Journal of Economic Theory 91, 223247.
Fudenberg, D., B. Holmstrom, and P. Milgrom (1990). Short-Term Contracts and Long-
Term Agency Relationships. Journal of Economic Theory 51, 131.
Golosov, M., N. Kocherlakota, and A. Tsyvinski (2003). Optimal Indirect and Capital
Taxation. Review of Economic Studies 70, 569587.
Holmstrom, B. and P. Milgrom (1987). Aggregation and Linearity in the Provision of
Intertemporal Incentives. Econometrica 55, 303328.
Hopenhayn, H. A. and J. P. Nicolini (1997). Optimal Unemployment Insurance. Journal
of Political Economy 105, 41238.
Jewitt, I. (1988). Justifying the First-Order Approach to Principal-Agent Problems.
Econometrica 56, 11771190.
Kocherlakota, N. R. (2004a). Figuring Out The Impact of Hidden Savings on Optimal
Unemployment Insurance. Review of Economic Dynamics 7, 541554.
Kocherlakota, N. R. (2004b). Wedges and taxes. American Economic Review (Papers and
Proceedings) 94, 109113.
ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 35
Kydland, F. E. and E. C. Prescott (1980). Dynamic Optimal Taxation, Rational Expecta-
tions and Optimal Control. Journal of Economic Dynamics and Control 2, 7991.
Liptser, R. S. and A. N. Shiryaev (2000). Statistics of Random Processes (Second ed.),
Volume I. Berlin: Springer.
Ma, J., P. Protter, and J. Yong (1994). Solving Forward-Backward Stochastic Differential
Equations Explicitly A Four Step Scheme. Probability Theory and Related Fields 98,
339359.
Ma, J. and J. Yong (1999). Forward-Backward Stochastic Differential Equations and Their Ap-
plications, Volume 1702 of Lecture Notes in Mathematics. New York: Springer-Verlag.
Mirrlees, J. A. (1999). The Theory of Moral Hazard and Unobservable Behaviour: Part
I. Review of Economic Studies 66, 321.
Peng, S. (1993). Backward Stochastic Differential Equations and Applications to Opti-
mal Control. Applied Mathematics and Optimization 27, 125144.
Phelan, C. and E. Stacchetti (2001). Sequential Equilibria in a Ramsey Tax Model. Econo-
metrica 69, 14911518.
Rogerson, W. P. (1985a). Repeated Moral Hazard. Econometrica 53, 6976.
Rogerson, W. P. (1985b). The First-Order Approach to Principal-Agent Problems.
Econometrica 53, 13571367.
Sannikov, Y. (2006). AContinuous-Time Version of the Principal-Agent Problem. Work-
ing paper, UC Berkeley.
Schattler, H. and J. Sung (1993). The First-Order Approach to the Continuous-Time
Principal-Agent Problem with Exponential Utility. Journal of Economic Theory 61,
331371.
Spear, S. and S. Srivastrava (1987). On Repeated Moral Hazard with Discounting. Re-
view of Economic Studies 53, 599617.
Stole, L. A. (2001). Lectures on the Theory of Contracts and Organizations. Manuscript,
University of Chicago Graduate School of Business.
Sung, J. (1997). Corporate Insurance and Managerial Incentives. Journal of Economic
Theory 74, 297332.
Werning, I. (2001a). Optimal Unemployment Insurance with Unobservable Saving.
Working paper, MIT.
Werning, I. (2001b). Repeated Moral Hazard with Unmonitored Wealth: A Recursive
First-Order Approach. Working paper, MIT.
Westereld, M. (2006). Optimal Dynamic Contracts with Hidden Actions in Continu-
ous Time. Working paper, USC.
Williams, N. (2006). Optimal Contracts with Persistent Private Information. In prepa-
ration, Princeton University.
Yong, J. and X. Y. Zhou (1999). Stochastic Controls. New York: Springer-Verlag.
Zhou, X. Y. (1996). Sufcient Conditions of Optimality for Stochastic Systems with
Controllable Diffusions. IEEE Transactions on Automatic Control AC-41, 11761179.