Overview of Theorems and Properties

Matrices, Graphs & Convexity

Introduction to matrices

Property 1.1. Let A Rnn be given. Then:

A has n linear independent eigenvectors s1 , . . . , sn corresponding to real eigenvalues 1 , . . . , n respectively
S 1 AS = diag(1 , . . . , n ) where the vector sk is the kth column of S (1 k n).
Property 1.2. If A Rnn and diag(1 , . . . , n ) Rnn are congruent, then A is symmetric.
Property 1.3. Let A Rnn be given. Then:
A is symmetric A has n real orthonormal eigenvectors.
Property 1.4. Let A be a real symmetric n n matrix. Then:
1. A is positive (semi-)definite all eigenvalues of A are positive (non-negative).
2. A is negative (semi-)definite all eigenvalues of A are negative (non-positive).
Property 1.5. Let A Rnn be given. Then:
A2 = A Ax = x, for all x Im(A) x Ax Ker(A), for all x Rn
Property 1.6. Let A Rnn be given. Then: A2 = A there exists a regular matrix S Rnn
and a (0, 1)-diagonal matrix D Rnn such that A = SDS 1 .
Property 1.7. Let the sizes of the matrices A, B and C be appropriate such that the operations below
are well defined. Then:
1. (A) B = A (B) = (A B), for all R;
2. (A + B) C = (A C) + (B C);
3. A (B + C) = (A B) + (A C);
4. A (B C) = (A B) C;
5. (A B)T = AT B T .
Property 1.8.
1. If A, C Rmm and B, D Rnn , then: (A B)(C D) = AC BD.
2. If A Rmp , C Rpq and B Rnr , D Rrs , then: (A B)(C D) = AC BD.
Property 1.9. If A Rmm and B Rnn , then:
1. A B = (A In )(Im B) = (Im B)(A In ).
2. (A B)1 = A1 B 1 , provided A and B are regular.
Property 1.10. Let A Rmm and B Rnn be given. Then:
There exists a permutation matrix P Rmnmn such that P T (A B)P = B A.
1

Singular value decomposition and generalized inverse

Theorem 2.1 (Singular Value Decomposition Theorem). Let m, n N be given and let p = min{m, n}.
Let A Rmn be given. Then:
There exist orthogonal matrices U Rmm and V Rnn , and a diagonal matrix
DA = diag(1 , 2 , . . . , p ) Rmn
such that 1 2 p 0 and A = U DA V T .
Property 2.1. Let A Rmn . The following statements are equivalent:
1. The matrix A is left invertible.
2. m n and rank(A) = n.
3. The columns of A are linearly independent.
4. Ker(A) = {0}.
Property 2.2. Let A Rmn . The following statements are equivalent:
1. The matrix A is right invertible.
2. m n and rank(A) = m.
3. The rows of A are linearly independent.
4. Im(A) = Rm .
Theorem 2.2.
1. Let A Rmn be a left invertible matrix.
Then the collection of all left inverses of A are given by


.
1
1 .
A1
=
A

BA
A
.
B
P
2
1
1
L
where P is an m m permutationmatrix,
B Rn(mn) is an arbitrary matrix and A1 Rnn is

A1
a regular matrix for which P A =
.
A2
A similar result holds for the right inverse.
2. Let A Rmn be a right invertible matrix.
Then the collection of all right inverses of A are given by
" 1
#
A3 A1
3 A4 C
1
AR = Q
C
where Q is an n n permutation matrix,
C R(nm)m is an arbitrary matrix and A3 Rmm is

a regular matrix for which AQ = A3 A4 .
Property 2.3.
1. Let A Rmn be left invertible. Then:
1
Every left inverse A1
L of A satisfies: the matrix AAL is idempotent.

2. Let A Rmn be right invertible. Then:

1
Every right inverse A1
R of A satisfies: the matrix AR A is idempotent.

Theorem 2.3. Let A Rmn be left invertible and let b Rm . Then:

1
1. Ax = b has a solution (Im AA1
L )b = 0 for some AL .
1
2. If (Im AA1
L )b = 0 then Ax = b has a unique solution given by x = AL b.

Theorem 2.4. Let A Rmn be right invertible. Then:

1. Ax = b has a solution for every b Rm .
2. For every b Rm \ {0}, every solution x of the equation Ax = b is of the form x = A1
R b for some
right inverse A1
of
A.
R
Theorem 2.5. Every matrix A Rmn has a generalized inverse AI .
Property 2.4. Let A be an m n matrix. Then:
AI is unique either A = 0 or {m = n and det(A) 6= 0}.
Property 2.5. Let A Rmn be given and let AI be a generalized inverse of A. Then:
1. AI A and AAI are idempotent matrices.
2. rank(AI ) = rank(A).
3. The matrix (AI )T is a generalized inverse of AT .
Theorem 2.6. Let A Rmn be a given matrix and let AI be a generalized inverse of A.
Then: Im(A) Ker(AI ) = Rm and Ker(A) Im(AI ) = Rn .
Property 2.6. Let A Rmn and let AI be a generalized inverse of A. Then:
AI Ax = x, for all x Im(AI ) and AAI y = y, for all y Im(A).
Theorem 2.7. Let A Rmn . Let Sm Rm and Sn Rn be given subspaces such that: Im(A) Sm =
Rm and Ker(A) Sn = Rn . Then: there exists a generalized inverse AI of A such that Ker(AI ) = Sm
and Im(AI ) = Sn .
Theorem 2.8. Let A Rmn be an m n matrix. Then:
There exists a unique n m matrix X Rnm for which
AXA = A, XAX = X, (AX)T = AX, (XA)T = XA.
Property 2.7. For every A Rmn with MoorePenrose inverse A+ :
1. Ker(A+ ) = Ker(AT ) and Im(A+ ) = Im(AT ).
2. (AT )+ = (A+ )T and (A)+ =

1 +
A , for

all R \ {0}.

3. If 1 2 r > 0 = r+1 = = n are the singular values of A, then r1 r1

11 are the positive singular values of A+ .

Graphs

Property 3.1. Let G = (V, E) be an undirected graph and let A, B V with A 6= B. If there exists a
simple path from A to B (in G), then there exists a simple path from A to B in which no vertex occurs
repeatedly.

Property 3.2. Let V = {Pk : 1 k n} and let G = (V, E) be an undirected graph or multi-graph.
Then:
n
X
deg(Pk ) = 2|E|, where |E| is the number of edges in G.
k=1

Property 3.3. For every undirected graph or multi-graph, the number of vertices of odd degree is even.
Theorem 3.1. Let G = (V, E) be an undirected graph or multi-graph. Then:
G has an Euler circuit G is connected and the degree of every vertex of G is even.
Property 3.4. Let G = (V, E) be an undirected graph or multi-graph. Then:
G has an Euler path but no Euler circuit G is connected and the degree of two vertices is odd.

Non-negative matrices

Property 4.1. For matrix A Rmn , A 0 = A(Rn+ ) Rm

+.
Property 4.2. Let A be an irreducible non-negative matrix and let x Rn+ .
Then: Ax = 0 = x = 0.
Theorem 4.1. Let A be an irreducible non-negative n n matrix and let x Rn+ be a non-negative
vector. If the number of positive coordinates of x is equal to k, then the vector (In + A)x has more than
k positive coordinates, where 1 k n 1.
Property 4.3. For an irreducible non-negative n n matrix A and a non-negative vector x Rn+ \ {0},
(In + A)n1 x  0.
Property 4.4. Let A be a non-negative n n matrix. Then: A is irreducible (In + A)n1  0.
Theorem 4.2. Let A be a non-negative n n matrix. Then:
(k)

A is irreducible for every (i, j) there exists some k N such that aij > 0 (1 i, j n).
Property 4.5. Every non-negative eigenvector of an irreducible non-negative n n matrix is positive.
Property 4.6. An irreducible non-negative n n matrix A has a positive eigenvalue A such that
A || for every eigenvalue of A and A has a positive eigenvector corresponding to A .
Property 4.7. Let A be a non-negative n n matrix A has a non-negative eigenvalue A such that
A || for every eigenvalue of A and A has a non-negative eigenvector corresponding to A .
Theorem 4.3. Let A 0 be an irreducible n n matrix and let A be the dominant eigenvalue of A.
Then:
1. A is a simple zero of the characteristic polynomial of A.
2. If Ax = A x then (the dominant eigenvector) x of A is a scalar multiple of a positive vector.
3. A has exactly one eigenvector in Rn+ (ignoring scalar multiples).
Property 4.8. Let A 0 be an n n matrix and assume that A is a simple dominant eigenvalue of A.
Then:
A is irreducible A and AT both have a positive eigenvector corresponding to A .
4

Property 4.9. Let A = [aij ]1i,jn Rnn be non-negative. Let A be the dominant eigenvalue of A.
n
X
Write Ri =
aij , 1 i n. Define Rmin = min {Ri : 1 i n} and Rmax = max {Ri : 1 i n}.
j=1

Then: Rmin A Rmax .

Theorem 4.4. Let A = [aij ]1i,jn Rnn be an irreducible non-negative matrix and assume that
n
X
Tr(A) =
aii > 0. Then A is primitive.
i=1

Property 4.10. Let A = [aij ]1i,jn Rnn be non-negative. Then:

A is primitive there exists a natural number m such that Am is positive.
Theorem 4.5. Let A Rnn be non-negative and irreducible and let the index h of A be at least 2.
Then there exists a permutation matrix P for which

0
A12
0
0
0
0
0
A23 0
0

..
..
..
.
.
.
.

.
.
.
.
.

P T AP =
.
.
..
..
..
..
..

.
.
.

0
0
0 Ah1,h
Ah1
0
0
0
where Ai,i+1 (1 i h 1) and Ah1 are irreducible.
Theorem 4.6. Let A be a non-negative n n matrix and let GA be the directed graph corresponding
to the matrix A. Then:
A is irreducible GA is strongly connected.
Theorem 4.7. Let A Rnn be a matrix of which the entries outside the diagonal are non-positive.
Then:
A is a regular M -matrix the inverse A1 exists and A1 0.
Property 4.11. Let A, B Rnn be non-negative matrices and let Un be the n n matrix of which
each entry is equal to 1.
1. A is stochastic AUn = Un .
2. A is stochastic = for every eigenvalue of A, || 1.
3. A is stochastic Au = u , where = 1 is the dominant eigenvalue of A, and

T
u = 1 1 1 is the dominant eigenvector corresponding to .
Property 4.12. Let A Rnn be a non-negative matrix with positive dominant eigenvalue and a
positive dominant eigenvector. Then:
The matrix

A is similar to a stochastic matrix.

Introduction to convexity

Theorem 5.1. Let V Rn be a non-empty set. Then: V is convex for all m N and all convex
m
m
X
X
combinations
k xk of points xk V (1 k m),
k xk V .
k=1

k=1

Property 5.1. For two convex sets V1 , V2 Rn , the following holds:

1. The intersection V1 V2 is convex.
2. For all 1 , 2 R, 1 V1 + 2 V2 is convex.
Property 5.2. For collections of open sets in Rn the following holds:
1. The sets Rn and the empty set are open.
2. The union of every collection of open sets is open.
3. The intersection of every finite collection of open sets is open.
For collections of closed sets in Rn the following holds:
1. The sets Rn and the empty set are closed.
2. The union of every finite collection of closed sets is closed.
3. The intersection of every collection of closed sets is closed.
Property 5.3. Let V Rn be a convex set. Then:
1. Int(V ) is an open convex set.
2. Cl(V ) is a closed convex set.
3. If V is bounded, then Cl(V ) is a compact convex set.
Property 5.4. For every a Rn \ {0} and b R the following holds:
1. The hyper-plane {x Rn : (a, x) = b} is a closed convex set.
2. The half-space {x Rn : (a, x) b} is a closed convex set.
3. The half-space {x Rn : (a, x) < b} is an open convex set.
Theorem 5.2. For every matrix A Rmn and vector b Rm ,
the set {x Rn : Ax b} is a closed convex set.
Theorem 5.3. For every set V Rn , the convex hull of V is the collection of all convex combinations
of elements in V .
Theorem 5.4. Let D Rn be a convex set and let f : D R be a given map.
Then: the function f is convex the set Epi(f ) Rn R is convex.
Property 5.5. Let D Rn be a convex set and let f : D R be a given map.
Then: the function f is concave the set Hypo(f ) Rn R is convex.
Theorem 5.5. Let V Rn R be a convex set and let D Rn be the projection of V such that inf{y
R : (x, y) V } > for all x D. Define the map f : D R by f (x) = inf{y R : (x, y) V }.
Then: f is convex.
Property 5.6. Let a Rn and b R. Let the function f : Rn R be defined by f (x) = (a, x) + b.
Then: f is convex and concave.
Property 5.7. Let D Rn be a convex set. Let f, g : D R be convex maps. Then:
1. For every 0, the function f : D R is convex.
2. The function f + g : D R is convex.
3. f is strictly convex = f + g : D R is strictly convex.
6

4. The maximum function Mf,g of f and g is convex, where

Mf,g : D R is defined by Mf,g (x) = max{f (x), g(x)}.
Let D Rn be a convex set. Let f, g : D R be concave maps. Then:
1. For every 0, the function f : D R is concave.
2. The function f + g : D R is concave.
3. f is strictly concave = f + g : D R is strictly concave.
4. The maximum function Mf,g of f and g is concave, where
Mf,g : D R is defined by Mf,g (x) = max{f (x), g(x)}.
Property 5.8. Let D Rn be a convex domain and let fk : D R be given functions (1 k N ) for
some N N.
Define Mf : D R by Mf (x) = max{fk (x) : 1 k N }, and
Define mf : D R by mf (x) = min{fk (x) : 1 k N }. Then:
1. fk is convex for all k (1 k N ) = Mf is convex.
2. fk is concave for all k (1 k N ) = mf is concave.
Theorem 5.6. Let D Rn be a convex set and let f : D R be a given map.
Then: f is a convex function = f is continuous on Int(D).
Property 5.9.
(a) Let D Rn be a convex domain and let f : D R be a convex map. Then: For every R, the
level set {x D : f (x) } is convex.
(b) Let D Rn be a convex domain and let f : D R be a concave map. Then: For every R, the
level set {x D : f (x) } is convex.
Property 5.10. Let D Rn be a convex domain.
Let fk : D R (1 k N ) be given functions.
Define Mf : D R by Mf (x) = max{fk (x) : 1 k N },
and define mf : D R by mf (x) = min{fk (x) : 1 k N }. Then:
1. fk is quasi-convex for all k (1 k N ) = Mf is quasi-convex.
2. fk is quasi-concave for all k (1 k N ) = Mf is quasi-concave.
Property 5.11. Let D Rn be an open convex domain and let f : D R be a map such that f has a
locally strict maximum and a locally strict minimum. Then: the function f is neither quasi-convex nor
quasi-concave.

Hyper-planes and cones

Property 6.1. Let V Rn be a closed convex set and let p Rn \ V . Then:

There exists a separating hyper-plane H of V and {p} such that p H and H V = .
Property 6.2. Let V Rn be a closed convex set and let p V . Then:
There exists a separating hyper-plane H of V and {p} such that p H.
7

Theorem 6.1.
(a) Let V1 , V2 Rn be convex sets such that V1 V2 = . Then:
There exists a separating hyper-plane H of V1 and V2 .
(b) Let V1 , V2 Rn be convex sets such that V1 V2 = and V1 V2 is closed. Then:
There exists a separating hyper-plane H of V1 and V2 which has a non-empty intersection with at
most one of the sets V1 and V2 .
Theorem 6.2. Let V1 , V2 Rn be convex sets. If Int(V1 ) 6= and V2 Int(V1 ) = , there exists a
separating hyper-plane H of V1 and V2 .
Property 6.3. Let V Rn be a closed set. Let a Rn \ {0} and b R.
Let f : Rn R be defined by f (x) = (a, x). Then:
H = {x Rn : (a, x) = b} is a supporting hyper-plane of V such that (a, x) b for all x V
f assumes a global maximum on V and max{f (x) : x V } = b.
Property 6.4. Let V Rn be a non-empty closed convex set and let p Rn \ V . Then:
There exists a unique point q V for which kx pk kq pk for all x V and
H = {x Rn : (p q, x) = (p q, q)} is a separating hyper-plane of V and {p} and it is a supporting
hyper-plane of V for which (p q, x) (p q, q) for all x V and (p q, x) > (p q, q) for x = p.
Property 6.5. Let K Rn be a non-empty set. Then:
K is a convex cone For all x, y K and > 0, x + y K and x K.
Theorem 6.3. Let K be a non-empty set in Rn . Then:
K is a convex pointed cone all non-negative linear combinations of points of K belong to K.
Property 6.6. For every i N, let Ki Rn be a convex pointed cone. Then:
The intersection

Ki is a convex pointed cone.

i=1

Property 6.7. Let m N. Let V = {vk Rn : 1 k m} be a non-empty set in Rn consisting of m

points. Then: the set of all non-negative linear combinations of the points vk of V (1 k m) is a
convex pointed cone.
Property 6.8. Let A Rmn be an m n matrix. Then:
The polyhedral convex set {x Rn : Ax 0} is a convex pointed cone.
Property 6.9. Let V Rn be a non-empty convex set and p Rn \ Int(V ).
For a Rn \ {0}, define the function fa : Rn R by fa (x) = (a, x) and define the set Ha Rn by
Ha = {x Rn : fa (x) = fa (p)}. Then:
1. There exists some a Rn \ {0} such that Ha is a separating hyper-plane of V and {p} through p.
2. The set {a Rn \ {0} : fa (x) fa (p) for all x V } {0} consisting of the normal-vectors of V
and the zero-vector is a closed convex pointed cone.

Theorem 6.4. Let D Rn be an open convex set and let hk : D R (1 k N ) be given quasi-convex
differentiable functions such that:
There exists x V such that hk (x ) < 0 for all 1 k N . (Slater condition)
Define for p V the set Cp consisting of non-negative linear combinations of the gradients of the functions
hk at p by
(N
)
X
Cp =
k hk (p) : k 0, k = 0 if hk (p) < 0, where 1 k N
k=1

Then:
1. For every p V , the set Cp is a closed convex pointed cone and Cp is generated by
{hk (p) : hk (p) = 0 (1 k N )}.
2. If hk (p) < 0 for all k (1 k N ), then Cp = {0}.
3. If Cp 6= {0} then for a Cp \ {0}, the set Ha = {x Rn : (a, x) = (a, p)} is a supporting
hyper-plane of V through p and (a, x) (a, p) for all x V .

Extrema

Theorem 7.1. Let V Rn be a non-empty compact convex set. Then:

1. V has extreme points.
2. V is the convex hull of the profile of V .
Property 7.1. Let V Rn be a non-empty convex set and let H be a supporting hyper-plane of V .
Assume that H V 6= . Then:
1. H V is convex.
2. The profile of H V is the set {x H : x is an extreme point of V }.
Property 7.2. Let V Rn be a non-empty closed convex set. If V has an extreme point, then for every
supporting hyper-plane H of V , the set H V has an extreme point.
Property 7.3. Let m, n N with m n. Let A Rmn and b Rm be given.
Let the polyhedral set V be the set of solutions Ax b. Then:
p V is an extreme point of V there exists a system

n
X

ari j xj = br of n equations, r {1, . . . , m}

j=1

(1 i n) having p V as the unique solution.

Property 7.4. Let V Rn be a polyhedral convex set.
Then: The profile of V is either empty of it consists of finitely many points.
Property 7.5. Let D Rn be a convex domain and let f : D R be a convex function. Let V Int(D)
be a non-empty compact set.
Then: f has a global minimum and a global maximum on V .
Property 7.6. Let D Rn be a convex domain and let f : D R be a continuous function for which
f has a local minimum at a Int(D). Then:
1. f is convex = f has a global minimum on D at a.
9

2. f is strict convex = f has a globally strict minimum on D at a.

Property 7.7. Let D Rn be a convex domain. Let f : D R be a quasi-convex function such that f
has a global minimum on D, and this minimum value is m.
Then: The set Vmin = {x D : f (x) = m} is convex.
Property 7.8. Let a Rn \ {0} and b R.
Let D Rn be a convex domain and let f : D R be given by f (x) = (a, x) + b.
1. Assume that D is closed and D has at least one extreme point. Then:
If f has a global minimum value m on D, then the set {x D : f (x) = m} contains at least one
extreme point of D; and
If f has a global maximum value M on D, then the set {x D : f (x) = M } contains at least one
extreme point of D.
2. Assume that D is compact. Then:
(a) D has at least one extreme point.
(b) f has a global minimum, m, and a global maximum, M , on D.
(c) The convex subsets of D consisting of points at which f assumes the minimum value m and
the maximum value M , respectively, are the convex hulls of the extreme points of D at which
f assumes the minimum value m and the maximum value M , respectively. That is,
{x D : f (x) = m} = Conv{x D : x is an extreme point of D and f (x) = m} and
{x D : f (x) = M } = Conv{x D : x is an extreme point of D and f (x) = M }.
Property 7.9. Let f : Rn R be a continuous quasi-convex function. Let V Rn be a convex set such
that Rn \ V 6= . Let a V be given. Then:
f has a global minimum on V at a either {x Rn : f (x) < f (a)} = or there exists a separating
hyper-plane of the sets V and {x Rn : f (x) < f (a).
Property 7.10. Let D Rn be open and let f : D R be a C 2 function.
Assume that a D is a stationary point of f . Then:
1. Every eigenvalue of Hf (a) is positive = f has a local minimum at a.
2. Every eigenvalue of Hf (a) is negative = f has a local maximum at a.
3. Hf (a) has a positive and negative eigenvalue = f has no extremum at a.
Property 7.11. Let D Rn be an open, convex domain and let f : D R be a C 2 function. Then:
1. f is convex Hf (x) is positively semi-definite for all x D.
2. Hf (x) is positively definite for all x D = f is strictly convex.
3. f is concave Hf (x) is negatively semi-definite for all x D.
4. Hf (x) is negatively definite for all x D = f is strictly concave.
Theorem 7.2. Let D Rn be an open convex domain.
Let N convex C 1 functions fk : D R (1 k N ) be given.
Let F : D R be defined by F (x) = max{fk (x) : 1 k N }.
Furthermore, let m quasi-convex C 1 functions gk : D R be given.
Define V = {x D : gk (x) 0 for all k (1 k m)}.
10

Let x V such that gk (x ) < 0 for all k (1 k m) (Slater condition).

Let p V such that: if gk (p) = 0, then gk (p) 6= 0 (1 k m).
Then: F has a global minimum on V at the point p there exists a convex combination of the
N
X
fk (p)s, denoted by w, for which w =
k fk (p) where k 0, k = 0 if fk (p) < F (p) and
k=1
N
X

k = 1, such that w is a non-negative combination of the gk (p)s for which w =

k=1

m
X

k gk (p)

k=1

where k 0 and k = 0 if gk (p) < 0.

Theorem 7.3. Let D Rn be an open convex domain.
Let F : D R be a convex C 1 function.
Furthermore, let m quasi-convex C 1 functions gk : D R be given.
Let the set V be given by V = {x D : gk (x) 0 for all k (1 k m)} and assume that the Slater
condition is satisfied (that is, there exists an x V such that gk (x ) < 0 for all k (1 k m).)
Let p V such that: if gk (p) = 0 then gk (p) 6= 0 (1 k m).
Then: F has a global minimum on V at the point p
F (p) =

m
X

k gk (p) where k 0 and k = 0 if gk (p) < 0.

k=1

11