Professional Documents
Culture Documents
8.1 Overview
Consider the problem of computing the mixed strategy equilibrium of a finite two
player game. These games are sometimes called bi-matrix games because the game can
be characterized by two matrices. Let player 1s utility be given by Ai , j and player twos
utility given by Bi , j .
A Nash equilibrium is a set of probability distributions, ( p*, q*) , such that,
p* = arg max pT Aq * such that pi 0 ,
p
p
i =1
=1
=1
q
j =1
We can obtain first-order conditions for each of these problems by forming the
Lagrangian,
L = pT Aq (eT p 1) T p
L = pT Bq (eT q 1) T q
The first-order conditions will yield,
Aq e = 0 ,
pB e = 0
eT p = 1 ,
eT q = 1
i pi = 0 ,
i qi = 0
p 0, 0, q 0, 0
Hence, finding all mixed strategy equilibria of a game ( A, B) involving all solutions
( p, q, , , , ) to the system of nonlinear equations and inequalities above. Notice that
the problem above is linear except for the quadratic terms i pi = 0 and i qi = 0 .
A Linear Complimentarity Problem is the problem of finding vectors ( w, z ) that
satisfy the following set of equality and inequalities,
w Mz = q ,
w 0,
z 0,
wi zi = 0 for all i
strategies of player 1 and let 1,..., J denote the strategies of player 2. Suppose that Ai , j is
the utility player one gets when player 1 plays strategy i and player 2 plays strategy j .
Let Ai , j denote player twos utility.
A mixed strategy for players 1 and 2 can be characterized by the vectors p and
q . The expected utility of player 1 is given by pT Aq and the utility of player 2 can be
characterized by pT Aq . We can characterize the equilibrium by,
(1)
(2)
p
i =1
=1
=1
q
j =1
We can show that ( p*, q*) is a mixed-strategy equilibrium to the above game if
and only if the following conditions hold,
(3)
(4)
p
i =1
=1
=1
q
j =1
One can show that this the solution to (3) and (4) can be formulated as the
following linear programming problems,
I
(5)
i =1
(6)
j =1
i =1
q
j =1
=1
=1
Using the duality theorem, we can show that the optimal value from these two problems
in the same. This, in turn implies that,
min Ai , j pi * = max Ai , j q j *
j
i =1
j =1
This, finally, implies that any solution to (3) and (4) must also be a solution to (1) and
(2).
Now, we know that a linear program is (i) infeasible, (ii) unbounded, or (iii) has
an optimal solution. We can immediately see that it is not infeasible or unbounded.
Hence, an optimal solution exists. Furthermore, we know that aside from degeneracy,
linear programs admit a unique solution, and in case of degeneracy, the set of solution set
will be convex.
Example 1
1 0
1 1
A=
,
1 0
Notice that this is not a zero-sum game. Let us start with the textbook approach to
solving this game. We start by showing that each player will not play a pure strategy.
Suppose that player 1 players row 1. In this case, player 2s best response is to colum 2,
but in this case player 1s best response is to play row 2. Hence, there is equilibrium
where player 1 plays row 1 with probability 1. Next, suppose player 1 plays row 2. The
player 2s best response is to player column 1. But player 1s best response to column 1 is
to play row 1. Hence, player 1 will not play a pure strategy in equilibrium. A similar
argument shows that player 2 will not play a pure strategy. Hence, any equilibrium
requires both players to mix over both their strategies.
Now, for a player to mix over both strategies, each strategy must be providing
him with the same utility. Player 1 gets q1 q2 in expected utility from playing row 1 and
q1 in expected utility from playing row 2. Combined with the fact that q1 + q2 = 1 , we
have that q = ( 13 , 32 ) . Player 2 gets p1 + p2 from playing column 1 and 2 p1 from playing
column 2, so his equilibrium strategy must by p = ( 14 , 34 ) .
Now, let us consider the algorithms for finding equilibrium. We can determine
that the equilibrium must satisfy the following equations,
q1 q2 1 = 0 ,
q1 2 = 0
p1 + p2 1 = 0 ,
2 p1 2 = 0
p1 + p2 = 1 ,
q1 + q2 = 1
1 p1 = 0 ,
2 p2 = 0 ,
1q1 = 0 ,
2 q2 = 0
p1 0 ,
p2 0 ,
1 0 ,
2 0
q1 0 ,
q2 0 ,
1 0 ,
2 0
Solving the last three equations amount to selecting a subset of the coefficients of ( p, q )
to be zero. If we set k coefficients, this will require 4 k of the lagrange multiplier to be
zero. We them have 4+6=10 equations and 10 unknowns, which we can solve. We can do
this for every subset of ( p, q ) (of which there are 24 . In each case, we check whether the
solutions satisfy the remaining inequalities.
Let us start by looking for a solution where non of the probabilities are positive.
We have,
q1 q2 1 = 0 ,
q1 2 = 0
p1 + p2 1 = 0 ,
2 p1 2 = 0
p1 + p2 = 1 ,
q1 + q2 = 1
1 = 0 ,
2 = 0,
1 = 0 ,
2 = 0
p1 0 ,
p2 0 ,
q1 0 ,
q2 0
1 = 0 ,
2 = 0,
1 = 0 ,
q2 = 0
The solution to the equalities will imply q1 = 0 and q2 = 1 , which violates the
assumption that q2 = 0 . Hence, not equilibrium has this form. If we repeated this
procedure for all possibilities, we would find that no other equilibrium exists.
Example 2
1 0
1 1
A=
,
1 0
Notice that A = B . We can solve this game by solving the following linear programming
problem,
(5)
(6)
z, p
z ,q
q2 0 , q1 + q2 = 1
Notice that these are the same problem, so we need only solve one of them. We can
reduce that first problem to,
max z such that z 2 p1 + 1 0 , z + p1 0 , p1 0 , p1 1
z, p
1
p1+
p1 1
p1 0 z
Consider a finite N -person game. Player n has the pure-strategy set {1, 2,..., J n } .
Let n denote the J n -dimensional unit simplex. We let pn n denote a mixed strategy
j th position
g n , j ( p) = max{zn , j ( p ), 0} .
As McLennan and McKelvey (1996) show, the Nash equilibrium can be characterized in
the following ways.
(ii)
pn, j zn, j ( p) = 0 , pn n .
N
(iii)
Jn
There must exist a j such that c j [ pn pn *] j > 0 . Suppose that pn , j > pn , j * , then we
must have c j > 0 , which means that c j [ sn , j pn *] j > 0 . Alternatively, if pn, j < pn, j * , the
c j < 0 , in which case c j [ sn ,k pn *] j > 0 for k j , proving that a profitable deviation
We have,
c ' sn, j = c ' pn for j Jn , hence, we can write,
10
This implies that c j < ck for j Jn , k Jn . Now, if pn, j > 0 for some j Jn , then we can
improve player n utility by setting pn, j = 0 and moving this probability to some strategy
k such that k Jn , by the fact that ck > c j , proving the result.
To show that (i) (iii) , notice that v( p*) = 0 if any only if g n , j ( p*) = 0 for all
z ( p ), zn , j ( p) 0
n and 1 j J n . Since g n , j ( p) = n , j
, we have that this is equivalent
otherwise
0,
Characterizations (ii) and (iii) each lead to important solution algorithms for this
problem. A nonlinear complementarity problem can be solved using direct methods. One
such method is the SLCP (Sequential Linear Complementarity Problem) algorithm. This
is an algorithm that will find a single solution to the NCP. The algorithm is quite similar
to the SQP algorithm for nonlinear programming.
Alternatively, we can solve (ii) by enumerating the possible solutions, solving the
resulting system of polynomial equations, and checking if these polynomial equations
satisfy the remaining inequality constraints. Methods exist to find all solutions of a
system of polynomial equations. Hence, we can use this method to find all the mixedstrategy equilibria of a multiplayer game.
Alternatively, we can find a single mixed strategy equilibrium by solving the
nonlinear programming problem described in (iii). We can find multiple mixed strategy
equilibria by using different starting points.
11
In this section, we will consider continuous games. Let j = 1, 2,..., J denote the
players in the game. Let X j \ D denote the strategy space of player j where D j
j
denotes the dimension of player j s strategy space. Let x j X j denote a strategy played
by player 1. Let X = X 1 ... X J denote the strategy space for all the players and let
x = ( x1 ,..., xP ) X
players except j . Let x j = ( x1 ,..., x j 1 , x j +1 ,..., xJ ) X j denote a vector of strategies for all
other player.
We let u j ( x1 ,..., xJ ) denote the utility received by player j if the players player
strategies ( x1 ,..., xJ ) X . We have also denote U j ( x j ; x j ) = u j ( x1 ,..., xJ ) . We define the
best response functions as, BR j ( x j ) = arg max U j ( x j ; x j ) where BR j ( x j ; x j ) is understood
x j X j
12
There are few general conditions that ensure the existence of a Nash Equilibrium.
The following theorem gives one such result:
Theorem 8.2 (Debreu-Fan-Glicksberg): Suppose that the utility functions of each player
are continuous and quasi-concave in his own strategy. Then a pure strategy Nash
Equilibrium exists.
Response Iteration
dampened
iterations.
For
example,
we
could
consider
13
Finally, we could consider steps that increase utility, but do not necessarily maximize
utility at each step.
In the event that D j = 1 for each j , we recommend the following strategy.
Employ Guass Jacobi iterations. At each iterations, solve for the optimal value using
Brents method. In the multi-dimensional case, we cannot employ Brents method.
Instead, we suggest applying the Nelder Mead Simplex Method. In the event that the
game is smooth enough such that Newtons method will be effective, we would
recommend apply the approach in the next section.
We define a response function for player j as a function R j ( x j ; x j ) that satisfies
U j ( R j ( x j , x j ); x j ) > U j ( x j ; x j ) if there exists such a point and R j ( x j , x j ) = x j otherwise.
Notice that a response function always exists. Notice furthermore that the Best Response
function can be used to yield a response function,
x j BR j ( x j )
xj ,
R j ( x j , x j ) =
otherwise
BR j ( x j ),
Algorithm 8.1:
1.
2.
14
4.
5.
6.
Go back to step 1
x j
U j ( x j *; x j *) = 0 for j = 1,..., J
This forms a system of D non-linear equations. We solve these equations using any
approach to solving nonlinear equations (e.g. Newtons method or Nelder Mead).
Notice that this method does not guarantee that the point we have converged to is
an equilibrium (each player current strategy is a global maximum of their response
problem). In fact, the current point may not even be a local equilibrium. We have found
that it is very useful to have tool for checking whether any proposed equilibrium is in fact
one. In order to verify an equilibrium, we suggests a grid search approach. Checking for a
local equilibrium is easier- we simply need to check each players Hessian for positive
definiteness.
15
Here, we consider a model of two candidate spatial competition one candidate has
a valence advantage, candidates are uncertain about the position of the median voter, and
candidates are policy-motivated. This model follows the work of Groseclose (2001). Let
Fv (v) = Gv (v ) be the distribution of the voters ideal point, where Gv is know to the
u R ( y; v ) = ( y v ) 2
=
sL ( yL , yR ; ) = Fv
G
v
2( yR yL )
2( yR yL )
+ yR 2 yL 2
+ yR 2 yL 2
sR ( yL , yR ; ) = 1 Fv
G
1
v
2( yR yL )
2( yR yL )
16
+ yR 2 yL 2
sL ( yL , yR ; ) = 1 Gv
2( yR yL )
+ yR 2 yL 2
sR ( yL , yR ; ) = Gv
2( yR yL )
When yL = yR , sL ( yL , yR ; ) = 1 and sR ( yL , yR ; ) = 0 .
Candidate L will when the election when sL ( yL , yR ; ) 12 . The probability of this
event is given by,
+ yR 2 yL 2
Pr( sL ( yL , yR ; ) 12 ) = Pr Gv
2( yR yL )
1
2
+ yR 2 yL 2
Gv 1 ( 12 )
= Pr
2( yR yL )
+ yR 2 yL 2
+ yR 2 yL 2
= Pr
Gv 1 ( 12 ) = F
2( yR yL )
2( yR yL )
We assume that candidates care about both policy and holding office. Let x
denote the policy outcome and let wk = 1 if candidate k wins office. We have,
U L ( x, wk ) = wk ( x qL ) 2 ,
U R ( x, wk ) = wk ( x qR ) 2
+ yR 2 yL 2
+ yR 2 yL 2
2
2
(
)
1
VL ( yL , yR ) = F
y
q
F
L
( yR qL )
L
2( yR yL )
2( yR yL )
+ yR 2 yL 2
+ yR 2 yL 2
2
2
(
)
1
VR ( yL , yR ) = F
y
q
F
L
( yR qR )
R
2(
)
2(
)
y
y
y
y
R
L
R
L
where yL < yR . A similar expression can be derived for the other case.
We can compute the equilibrium of this game using the various strategies we have
described. Using Response Iteration, we simply need to supply the algorithm with the
utility function. We will then set,
17
G (v) = ( v / )
Consider the following situation. There are J candidates running for office, who
propose policies y j \ D . Voters are characterized by their ideal points v . Candidates
are characterized by their mean valence j . The utility of a voter with ideal point v
receives from voting for candidate j is given by,
u j ( y j ; v) = j ( y j v) 2 + j
where j has the logistic distribution. We can compute the probability that a voter with
ideal point v votes for candidate j ,
q j (v, y j , j ) =
j ( y j v )2
( yk v ) 2
k =1
e j ( y j v )2
sj (yj ,j ) = J
dFv (v)
v
2
ek ( yk v )
k =1
18
y j
2
j ( y j v )2
(
y
v
)
e
ek ( yk v )
k j
s j ( y j , j ) = 2
dFv (v) = 0
2
v
J k ( yk v )2
k =1
yj =
Notice that this is a fixed point problems, which we can potentially solve using fixed
point iterations.
Now, consider a convergent equilibrium, with y1 = ... = yJ = y * . In this case, we
have q j (v, y*, j ) =
k =1
The first-order conditions will hold when each candidate locates at the mean-voter
position. Consider the second-order condition as well,
2
y j 2
= 4 2 ( y j v ) e
j ( y j v )2
( yk v ) 2
k j
sj (yj ,j )
J ( y v )2
( y j v )2
y j e k k + 2( y j v)e j
k =1
dFv (v) = 0
3
J
k ( yk v )2
e
k =1
At y * , we have,
4 2 e
y j 2
2
sj (yj ,j ) =
k j
J k
e
k =1
J k
(
*
)
y
v
y * e
v
k =1
19
j
+ 2( y * v)e dFv (v)
= 8 2 q j 2 (1 q j )Var (v)
D -dimensional integral. We will compute this integral using simulation methods, i.e.,
e j ( y j vn )2
s j ( y j , j ) = N1 J
k ( yk vn )2
n =1
e
k =1
The integrals in the fixed point equations can be computed in a similar way,
N
yj =
1
N
v q (v , y , )(1 q (v , y , ))
1
N
n =1
N
q (v , y , )(1 q (v , y , ))
n =1
We will assume throughout that v ~ N (0, I ) and we will vary the mean valence
parameters (1 , 2 , 3 , 4 ) . Merrill and Adams (2001) show that argue that when the fixed
point mapping above is a contraction mapping, then there exists a unique solution to the
fixed-order conditions which will satisfy the conditions of being an equilibrium.
Let us consider an example of two-dimensional spatial competition with 4 parties.
Let us assume that v ~ N (0, I ) . We start with the case where 1 = 2 = 3 = 4 = 0 .
20
voters for each candidate with probability one-half is he is indifferent. The probability
that each candidate wins the election is given by,
{n : y1 vn
P1 ( y1 , y2 ) = {n : y1 vn
{n : y1 vn
< y2 vn
< y2 vn
< y2 vn
} > {n :
} = {n :
} < {n :
y1 vn
> y2 vn
y1 vn
> y2 vn
y1 vn
> y2 vn
},
},
},
1
1
2
P2 ( y1 , y2 ) = 1 P1 ( y1 , y2 )
We assume that candidates would like to maximize the probability that they win
the election. Notice that this game is a zero-sum game. Hence, we can represent the game
by the function, A( y1 , y2 ) = P1 ( y1 , y2 ) . Plott (1967) considered such a game and found
that a pure strategy equilibrium existed only under unlikely conditions, when D > 1 .
However, mixed strategy equilibria are more likely to exist (Duggan, various references).
Let X be a (finite or infinite) set of alternatives and let P represent the MajorityRule social preference relation. We say that x X covers y X if the following
conditions are met.
(i)
xPy
(ii)
P( x) P( y )
(iii)
R( x) R( y )
21
Where we define,
P ( x) = { y X : yPx}
R ( x) = { y X : yRx}
We will denote the covering relation by C . The Uncovered Set is the set of maximal
elements of C . That is,
UC = {x : C ( x) = }
where,
C ( x) = { y X : yCx}
The following proposition summarizes some properties of the Uncovered Set.
Proposition 1: (i) The Uncovered Set in non-empty. (ii) The Uncovered Set is Unique.
(iii) The support of mixed strategy equilibria to the multidimensional Downsian model is
contained in the Uncovered Set.
For a proof of this result, see Banks, Duggan, and Le Breton (2000).
Developing an algorithm for computing the Uncovered Set when the set of
alternatives is finite is relatively straight-forward. We assume that we can somehow
compute the Majority Rule social preference relation over the alternatives. For example,
where the Majority Rule Social Preference is determined by majority rule and there are
N voters, this step will have complexity O( NJ 2 ) . We can compute the covering relation
by brute force, by looping over all triples x, y, z X . The algorithm above clearly has
complexity O( J 3 ) where J is the number of alternative. Then, we can compute the
maximal element of the covering relational as follows.
22
1:
2:
Change = 1
3:
while ( Change = 1 )
4:
Change = 0
5:
for i = 1 to J, j = i+1 to J
6:
7:
8:
UC(j)=0
9:
Change = 1
10:
break for
11:
12:
UC(i)=0
13:
Change = 1
14:
break for
23
computing integrals, solving for the equilibrium of a model. In the case where ideal
points of a finite number of voters are drawn independently from some distribution, the
theoretical complexity will be O( NJ 2 ) . The theoretical complexity of computing the
covering relation will be O( J 3 ) .
The easiest way to solve such a game is to approximate that game using a finite
one. This problem then reduces to a finite two-player zero-sum game, which we can solve
using linear programming. To simplify the computational complexity of the problem, we
can first compute the Uncovered Set. This allows us to reduce the size of the linear
programming problem we must ultimately solve.
We will approximate the solution to the above equations using the approximation,
J
p
j =1
= 1 , p j 0 , j = 1, 2,..., J
24
z = 1
and p = (1, 0,..., 0) is a feasible solution. Thus, we know there must exist a solution
[0,1] [0,1] . We will consider J = 21, 41,81 . We present the results below,
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
25
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
0.6
Uncovered Set
0.4
Mixed Strategies
0.2
0
0
0.2
0.4
0.6
0.8
26
( ) which represents the offer that the plaintiff makes conditions on their information.
We will solve this game by applying Perfect Bayesian Equilibrium.
We let a *( ; , ) denote the equilibrium strategy of the plaintiff and let
27
( ) = f ( )
( | ) =
f ( ) *( , )
if
f ( ) *( , )d
f ( ) *( , )d > 0
( | ) =
f ( ) *( )
f ( ) *( )d
if
f ( ) *( , )d > 0
Let us look for an equilibrium where *( ) is one to one. In this case, we have that
places probability 1 on the truth. Hence, we have,
a *( ; ) = 1
= arg min F ( ) +
0
dF ( )
to obtain *( ) , which suffices to solve he game. The first-order conditions are given
by,
28
F ( ) + f ( )[1 1 1 ]
Notice that when = 1 , we have = 0 as the unique solution. If both players are risk
neutral, there is no need to bargain to avoid trial.
Suppose that two firms produce a product at cost c1 and c2 . Each firm knows its
own cost, but the other firms cost is only know to be drawn from the distribution F (c) .
Each firm must decide how much to produce x1 [0, 12 ] and x2 , where x = x1 + x2 .
Demand for the product is given by 1 p where p is the price of the product. The price
is chose to equalize demand and supply, so that p = 1 x1 x2 . A firms profit will by,
2 ( x1 ) = (1 x1 *(c1 ) x2 c2 ) x2 dF (c1 )
c1
An equilibrium satisfies,
x1 *(c1 ) = arg max (1 x1 x2 *(c2 ) c1 ) x1dF (c2 ) for 0 c1 1
c2
x1[0, 12 ]
c1
x1[0, 12 ]
x2 [0, 12 ]
29
x2 *(c2 ) = 13 12 c2 + 16 E[c]
Hence, the optimal strategies only defend on the distribution of costs through its mean.
Suppose, however, that we did not know this problem admitted an analytical
solution. Consider solving this problem by approximating it on a finite grid,
x1 *(c1 ) = arg max (1 x1 x2 *(c2 ) c1 ) x1dF (c2 ) for 0 c1 1
x1[0, 12 ]
c2
c1
k =1
k =1
x2 *(c2 ) = 125 12 c2
30
We computed the solution using a grid size of n = 21,51,101, 201 , and produced the
results below. Results were compute using a starting value of x1 = x2 = 0.5 and a
dampening factor of 0.1 (although a dampening factor of 1.0 seems to work as well).
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
n=21
n=51
n=101
n=201
0.2
0.4
0.6
0.8
1.2
Consider a model where two countries 1 and 2 must decide whether to fight a war.
Each country makes a proposal xk [0,1] on how much of a pie to allocate to country 1.
In the event that that x1 x2 , the offers can be reconciled and a mediator country decides
to allocate of the surplus to country 1, so that country 1 receives (1 )x1 + x2 and
country 2 receives 1 (1 )x1 x2 . In the event that their offers cannot be reconciled
(i.e. x1 < x2 ), the countries must go to war to settle their differences. Let 0 < p < 1 denote
the probability that country 1 wins the war and let 1 denote the fraction of the surplus
that is left after the war. The winning country obtains the whole leftover pie in the event
of a war, of size . Each country must pay a cost to fight the war, which is drawn from a
31
distribution Fk which has support on [0,1] . Hence, the utilities of each player in the event
of a war are p c1 and *(1 p ) c2 .
Let x1 *(c1 ) and x2 *(c2 ) denote the optimal strategies of both players. We
conclude that the equilibrium satisfies the functional equations,
x1[0,1]
x2 [0,1]
Playing around with the conditions will indicate that the model does not have an
analytical solution, even where both cost are assumed to be uniformly distributed.
In order to solve this problem numerically, we must somehow translate a
functional problem into a finite one. The first approach we can use is to approximate this
functional problem with a finite grid. Consider approximating this problem on a finite
grid of n points, {0, n11 , n21 ,...,1} . We let ai denote the strategies of country 1 and bi
denote the strategies of country 2. We have the following two equations,
ai =
bi =
n 1
1
n 1
j:0 a j n 1 k = 0
n 1
1
n 1
arg max 1{ n j 1 bk }(
j
n 1
} 1 ak (1 ) n j 1 + 1{ak <
j
n 1
}( (1 p ) n i 1 ) Fk1
j
n 1
Notice further that we now have a finite fixed point problem, which suggests a solution
method- (dampened) fixed point iterations.
32
Let us try to solve this problem in the special case where p = 0.75 , = 0.5 ,
= 1 , and F1 = F2 = Uniform(0, 0.2) . When we start the fixed point iterations from 0.5
and used = 1 , the algorithm converged to a point where both players offer to hold onto
the entire pie by themselves. When we used = 0.1 , we obtained the following more
interesting equilibrium,
1.20
1.00
0.80
x1
x2
0.60
0.40
0.20
0.00
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
8.9- References
[1]
[2]
[3]
[4]
[5]
[6]
Banks, Jeffery, John Duggan, and Michel Le Breton (2002). Bounds for Mixed
Strategy Equilibrium and the Spatial Model of Elections. Journal of Economic
Theory 103:88-105.
[7]
Duggan, John, and Matthew O. Jackson (2006). Mixed Strategy Equilibrium and
Deep Covering in Multidimensional Electoral Competition. Working Paper.
34