Mean Field Games in Economics

Benjamin Moll

Princeton

IMA Mean Field Games and Applications Course

November 13, 2012

1 / 44
 The Theme of my Talk

... to help me out with my math problems

2 / 44
 Outline

(1) A class of Mean Field Games in economics: heterogeneous

agent models

(2) Mean Field Games with aggregate shocks

(3) Surpassing language barriers when talking to economists

(4) Boltzmann Mean Field Games

3 / 44
 Why Heterogeneous Agent Models?
Many interesting questions involve some kind of distribution:

(1) Does high income and wealth inequality lead to faster or

slower GDP growth?
(2) Does high income and wealth inequality increase or decrease
GDP volatility (severity of recessions)?
(3) Where does the firm size distribution come from? Why does
it follow a power law?

To theorize about these, need heterogeneous agent models.

Will argue: mathematics are somewhat underdeveloped,

potentially high payoff from well-trained mathematicians
working on these.
4 / 44
 A Prototypical Heterogeneous Agent Model

Based on Rao Aiyagari (1994), Uninsured Idiosyncratic Risk

and Aggregate Saving, Quarterly Journal of Economics

Taught in first year of every self-respecting economics PhD

program

Original paper: discrete time; here: continuous time

Two types of agents: households and firms

5 / 44
 Households
are heterogeneous in their wealth k and work ability z (joint
distribution g (k, z, t)), solve
Z
max E0 e t u(ct )dt s.t.
ct 0
dkt = [wt zt + rt kt ct ]dt
dzt = z (zt )dt + z (zt )dWt
kt k
ct : consumption
u: utility function, u > 0, u < 0.
: discount rate
wt : wage
rt : interest rate
Wt : standard Brownian motion, independent across households
k: borrowing limit, e.g. if k = 0, can only save
6 / 44
 Households: Simplified Version
Previous slide: taken one-for-one from Aiyagari.

Here: simplify further

Assume wt zt dt iid normal over time with mean wt dt and

variance 2 dt
Households solve
Z
max E0 e t u(ct )dt s.t.
ct 0

dkt = [wt + rt kt ct ]dt + dWt

kt k.
Advantage: only one state variable k

(bit weird: unbounded labor income shocks, even negative)

7 / 44
 Firms

All identical, solve

max AF (Kt , Lt ) (rt + )Kt wt Lt

Kt ,Lt

A: productivity, fixed for now

Kt : capital
Lt : labor
F : production function, increasing and concave
: depreciation rate, rt + : user cost of capital

In equilibrium
Z
Lt = 1, Kt = kg (k, t)dk

8 / 44
 Equilibrium
Prices are
Z
rt = AFK (Kt , 1) , wt = AFL (Kt , 1), Kt = kg (k, t)dk

Households HJB equation:

2
v (k, t) = max u(c) + vk (k, t)[wt + rt k c] + vkk (k, t) + vt (k, t)
c 2
Kolmogorov-Forward (Fokker-Planck) eq. for wealth dist.
g (k, t) 2 2
= [k (k, t)g (k, t)] + g (k, t)
t k 2 k 2
where c(k, t) is optimal consumption choice from HJB and
k (k, t) = wt + rt k c(k, t)
A mean field game! Interactions through prices.
Here: prices only depend on mean. But easy to think of other
setups where also other features of distribution matter.
9 / 44
 Solution in Practice
Assume some parametric functional forms for u and F

Most commonly used are CRRA utility and Cobb-Douglas

production function
c 1
u(c) = , > 0, F (K , L) = K L1 , (0, 1)
1
Therefore prices entering households HJB eq. are
Z
rt = AKt1 , wt = (1 )AKt , Kt = kg (k, t)dk

Note: different from linear-quadratic examples

mathematicians like to use

Borrowing constraint: k = 0.

Solve numerically. More on this momentarily.

10 / 44
 Open Questions

(1) Existence of equilibrium

(2) Uniqueness of equilibrium

(3) Also some scope for improving numerical solution.

11 / 44
 Open Questions
Most existing approaches focus on stationary equilibrium only.
Typical algorithm for solving stationary equilibrium uses
Monte Carlo method:
1. Guess interest rate, r
2. Compute households optimal decision rules from dynamic
programming problem
3. Simulate N (say 50,000) individuals and compute aggregate
capital demand and supply
4. If demand=supply, stop. Otherwise, update r using bisection
method and go back to 1.
5. Cross fingers, pray and hope that algorithm converges
Actually works surprisingly well in practice. Still
unsatisfactory. Non-stationary equilibria even trickier.
Some simple transparent MFG FD method much preferred. 12 / 44
Mean Field Games with
Aggregate Shocks

13 / 44
 Mean Field Games with Aggregate Shocks

This is where I think the real payoff could be

Consider Aiyagari (1994) with shocks to aggregate

productivity, A.

First studied by Per Krusell and Tony Smith (1998), Income

and Wealth Heterogeneity in the Macroeconomy, Journal of

Political Economy

14 / 44
 Mean Field Games with Aggregate Shocks
Households:
Z
max E0 e t u(ct )dt s.t.
ct 0

dkt = [wt + rt kt ct ]dt + dWt

kt k.
Firms:

max At F (Kt , Lt ) (rt + )Kt wt Lt

Kt ,Lt

dAt = A (At )dt + A (At )dZt

Equilibrium:
Z
Lt = 1, Kt = kg (k, t)dk
15 / 44
 Mean Field Games with Aggregate Shocks
Question: What is the appropriate state variable?

Answer: its really the entire wealth distribution (plus

aggregate productivity)

state = (g , A)

Problem: g is infinite-dimensional.

Why not enough to keep track of first moment

R
Kt = kg (k, t)dk? Answer: evolution dKt depends on entire
g , not just Kt (unless all individual dkt s are linear in kt ).
Nicely explained in Victor Rios-Rull (1997), Computation of
Equilibria in Heterogeneous Agent Models
16 / 44
 Mean Field Games with Aggregate Shocks
Krusell-Smith get around this with an approximation:
households care only about some finite set of moments of
wealth distribution (in practice, only first moment):

v (k; g , A) v (k; K , A)

convert infinite-dimensional problem into finite-dim. one.

Makes some sort of sense: mean field type interaction only
through prices (r , w ). Assumption is that households only
take into account future price movements that are due to
movements of the aggregate capital stock (and A). Bounded
rationality interpretation.
But obviously still unsatisfactory KS on accuracy of their algorithm

17 / 44
 What Im Hoping For

Someone comes up with a tractable, MFG version of

Krusell-Smith in continuous time

tractable = tractable enough to solve globally without

approximations/bounded rationality assumptions. Even better:

tractable enough to prove existence and uniqueness

Can publish that anywhere!

But even true for MFG version of Aiyagari (1994), i.e.

without aggregate shocks

18 / 44
Language Barriers

19 / 44
Surpassing Language Barriers when Talking to
Economists

Economists dont know , !

You like to write

u 2
+ u + H(x, m, u) u = 0 (HJB)
t 2 
2

m H
= m (x, m, u) + m (KFE)
t p 2

where H(x, m, p) = maxa h(x, m, a) + pa.

20 / 44
 Talking to Economists

If you talk to economists, may want to write

2 u(x, t)
uxx (x, t) +
u(x, t) = H(x, m, ux (x, t)) + (HJB)
2 t
m(x, t) 2
= [m(x, t)Hp (x, m, ux (x, t))] + mxx (x, t) (KFE)
t x 2

Or even better get rid of the Hamiltonian

2 u(x, t)
u(x, t) = max h(x, m, a) + ux (x, t)a + uxx (x, t) + (HJB)
a 2 t
m(x, t) 2
= [m(x, t)a (x, m)] + mxx (x, t) (KFE)
t x 2
where a (x, m) is maximand in (HJB)

21 / 44
 Another Observation
MFG literature seems to often focus on problems with a

separable Hamiltonian

H(x, p) + V (m)

This is of very limited use to economists because most

problems in economics do not have separable Hamiltonians

E.g. in Aiyagari and Krusell-Smith

H(x, m, p) = max u(c) + p[w (m) + r (m)x c]

c

which is not separable.

22 / 44
Boltzmann Mean Field Games

23 / 44
A Theory of Growth from Knowledge Diffusion
Based on Lucas and Moll (2012), Knowledge Growth and the

Allocation of Time
Consider economy with continuum of infinitely-lived agents

Productivity of each is a random variable, a density g , cdf

G . Entire distribution is state variable of economy.

Convenient to work with cost rather than productivity

Let a = z , (0, 1), where z density f , cdf F

Can always back out

G (a, t) = 1 F (a1/ , t)

Computations use z; figures use a.

24 / 44
 A Technology of Learning

s(z, t): fraction of time searching.

1 s(z, t): fraction of time working

Earnings:

y (z, t) = [1 s(z, t)]z

Per capita GDP:

Z
Y (t) = [1 s(z, t)]z f (z, t)dz
0

How do individual productivities evolve?

25 / 44
 Evolution of Productivity Distribution

Allocate s(z, t) to search.

Over (t, t + h), meet one other agent with probability

[s(z, t)] h.

Meeting = draw z from f (

z , t).

z(t + h) = min{
z , z(t)}.

Meetings completely asymmetric.

Number of agents at z:
 
f (z, t) f (z, t)  f (z, t) 
= +
t t out t in
26 / 44
 Evolution of Productivity Distribution
Outflow: Agent z adopts lower cost if he meets another with

cost y z.

Happens w/ prob (s(z, t))f (y , t). Hence

 Z z
f (z, t) 
= (s(z, t))f (z, t) f (y , t)dy
t out 0

Inflow: agent y z adopts z if he meets agent at z.

Happens w/ prob (s(y , t))f (z, t). Hence

 Z
f (z, t) 
= f (z, t) (s(y , t))f (y , t)dy
t in z

27 / 44
 Evolution of Productivity Distribution

Combining ins and outs we have

z
f (z, t)
Z Z
= (s(z, t))f (z, t) f (y , t)dy +f (z, t) (s(y , t))f (y , t)dy
t 0 z

A Boltzmann equation.

Many possible generalizations, present two later.

28 / 44
 Decentralized Time Allocation Decisions

Assume that agents maximize expected PV or earnings,

discounted at given > 0

No risk aversion, no theory of real interest rates: a kind of

partial equilibrium

Individual preferences:

Z  
( t)

V (z, t) = Et e [1 s(
z ( ), )] z( ) d z(t) = z

t

Poisson arrival rate (s) 0 with

(s) > 0, (s) < 0, all s.

29 / 44
 Equilibrium

Bellman equation:
 Z z 
V (z, t)
V (z, t) = max (1 s)z + (s) [V (y , t) V (z, t)]f (y , t)dy .
t s[0,1] 0

Boltzmann equation:
z
f (z, t)
Z Z
= (s(z, t))f (z, t) f (y , t)dy +f (z, t) (s(y , t))f (y , t)dy .
t 0 z

A Boltzmann mean field game!

30 / 44
 Balanced Growth Path
Definition
A balanced growth path (BGP) is a number and a triple of
functions (, , v ) on R+ such that

f (z, t) = e t (ze t ),

V (z, t) = e t v (ze t ),

s(z, t) = (ze t )

Intuitively: all z-quantiles shrink at

( all z -quantiles grow at ).
CDF F (z, t) = (ze t ) qth quantile defined by

(zq (t)e t ) = q zq (t) = e t 1 (q).

31 / 44
 Balanced Growth Path

Restate (BE), (LM) for BGP only. Use x = ze t

 Z x 
( ) v (x)v (x)x = max (1 )x + () [v (y ) v (x)] (y ) dy
[0,1] 0
Z Z x
(x) + (x)x = (x) ((y ))(y )dy ((x))(x) (y )dy .
x 0

Growth rate?
Z
(0) = (0) ((y ))(y )dy .
Z 0
= ((y ))(y )dy
0

32 / 44
 Balanced Growth Path

BGP with > 0 only if (0) = f (0, 0) > 0.

Lemma

(0) = f (0, 0) > 0 G (a, 0) has Pareto tail with parameter 1/.

Taken literally: all knowledge already exists at t = 0.

Alternative interpretation: G (a, 0) is bounded but new

knowledge arrives at arbitrarily low frequency (innovation).

Observationally equivalent, see section 6.3 in paper

33 / 44
 Balanced Growth Path

Next slide presents results of single simulation of BGP

Initial cost distribution is exponential: 0 (x) = e x

Function () given by () = k

Display time allocation function (x), density (x), and two

Lorenz curves

income Lorenz curve, based on (1 (x)) x

value Lorenz curve, based on calculated value function, v (x)

34 / 44
 Balanced Growth Path
TIME ALLOCATION PRODUCTIVITY DENSITY
1
0.06 Solid curve: calculated density
Dashed curve: Pareto Dist. with Parameter 1/
0.8 0.05

0.04
0.6
0.03
0.4
0.02

0.2
0.01

0 0
0.5 1 1.5 2 0 1 2 3 4 5 6

Productivity, x Productivity, x

LORENZ CURVES PARAMETER VALUES

1 1
Earnings
Value
0.9
Growth Rate = = .02
0.8 0.8
= .04
0.7

0.6 0.6 = 0.3

0.5
= .06
0.4 0.4

0.3 = 0.5
0.2 0.2
= 0.05
0.1

0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

35 / 44
 Two Sample Paths
(a) (b)
9 3.5

t
8
3

Productivity rel. to Trend, z e

7
2.5
Productivity, z

5 2

4 1.5

3
1
2
0.5
1

0 0
0 20 40 60 80 100 0 20 40 60 80 100
Years Years

(c) (d)
1 9

0.9 8

Earnings, (1s(z,t))z
0.8
Time Allocation, s(z,t)

7
0.7
6
0.6
5
0.5
4
0.4
3
0.3

0.2 2

0.1 1

0 0
0 20 40 60 80 100 0 20 40 60 80 100
Years Years

36 / 44
 Alternative Learning Technologies

Learning technology described above involves

probabilistic model of agents meetings

description of effects of meetings on agents knowledge

Easy to think of modifications, other Boltzman equations

Algorithm very flexible. Beginning to explore possible

variations

37 / 44
 Limits to Learning
Impose an order to learning, limits on intellectual range
If y meets z < y at t, he can adopt z with given probability
k(y , z, t)
w/prob. 1 k(y , z, t) he cannot do this; retains cost y

Natural assumption: k unchanged if z and y at same

quantiles. Along BGP this requires

k(y , z, t) = k(ye t , ze t , 0).

Convenient to work with

k(y , z, 0) = e |y z|

Alternative interpretation: meeting probabilities depend on

|y z|; socioeconomic stratification or segregation.

38 / 44
 Balanced Growth Path

BGP equations become

 Z x 
( ) v (x) v (x)x = max (1 )x + () [v (y ) v (x)](y )e (xy) dy
[0,1] 0

Z
(x) + (x)x = (x) ((y ))(y )e (yx) dy
x
Z x
((x))(x) (y )e (xy) dy
0

Z
= ((y ))(y )e y dy
0

39 / 44
Optimal Time Allocation for Various Values

0.9

0.8 =0
0.7
Time Allocation, (x)

0.6

0.5

0.4 = .01
0.3

0.2
= .015
0.1

0
0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Productivity, x, rel. to Median 40 / 44
Limits to Learning Reduce Equality and Growth

0.9
0 0.01 0.015
0.8
0.02 0.014 0.01
0.7
Y(0) .37 .36 0.34
0.6

0.5

0.4

0.3

0.2
=0
0.1 = 0.01
=0.015
0
0 0.2 0.4 0.6 0.8 1
41 / 44
Limits to Learning Reduce Equality and Growth

0.9
0 0.01 0.015
0.8
0.02 0.014 0.01
0.7
Y(0) .37 .36 0.34
0.6

0.5

0.4

0.3

0.2
=0
0.1 = 0.01
=0.015
0
0 0.2 0.4 0.6 0.8 1
42 / 44
 Conclusion

Mean field games potentially extremely useful in economics...

... lots of exciting questions involve mean field type

interactions...

... but mathematics often pretty challenging, at least for the

average economist.

Potentially high payoff from mathematicians working on this!

43 / 44
Krusell-Smith on Accuracy of their Algorithm Back

p.897: Like most numerical procedures, the present one does

not provide bounds on how far the approximate equilibrium
deviates from an exact equilibrium. In particular, one might
imagine that there are self-fulfilling approximate equilibria:
because agents perceive a simple law of motion, they behave
accordingly.

44 / 44