Professional Documents
Culture Documents
Summary on
INTRODUCTORY
LINEAR
ALGEBRA
A Comprehensive Summary on Introductory Linear Algebra
A Comprehensive Summary on
Introductory Linear Algebra
mathvault.ca
Table of Contents 2
A Comprehensive Summary on Introductory Linear Algebra
14 Matrix Transformation 16
14.1 Preliminaries . . . . . . . . . . . . 16 2.3 Multiplication
14.2 Standard Matrix Transformations in 2D 16
• Requires two matrices with matching “inner
dimensions".
$ 1 Matrix Terminologies • Produces a matrix with the corresponding
“outer dimensions” (i.e., m × n times n × p →
• Diagonal matrix: A matrix whose non-zero en- m × p).
tries are found in the main diagonal only. • The ij entry of AB results from dot-multiplying
• Identity matrix: A diagonal, n × n matrix with the ith row of A with the jth column of B.
1 across the main diagonal. Usually denoted • (AB)C = A(BC), but AB 6= BA in general.
by I.
• (kA)B = k(AB) = A(kB)
• Upper-triangular matrix: A matrix whose non-
zero entries are found at or above the main • A(B +C) = AB +AC, (A+B)C = AC +BC
diagonal only.
• Lower-triangular matrix: A matrix whose non- 2.4 Transposition
zero entries are found at or below the main
diagonal only. • Reverses the dimension of the matrix.
• (AT )ij = Aji
2 Operations on Matrices 3
A Comprehensive Summary on Introductory Linear Algebra
• Tr(AT ) = Tr(A), Tr(AB) = Tr(BA) • Create a leading number in the ith entry
of the column.
• Reduce all entries beneath it to zero.
$ 3 Elementary Row Opera- and proceed to search for the (i + 1)th leading
tions number in the next column.
Once all the columns are handled, the matrix would
The three elementary row operations are: be in a ladder form, where
• Row Multiplication (Ri → kRi , k 6= 0) • Dividing each non-zero row with its leading
number would put the matrix into a row-
• Row Swapping (Ri ↔ Rj ) echelon form.
• Row Absorption (Ri → Ri + kRj ) • From here, further reducing all non-leading-
one entries in each column to zero would put
Note that each elementary row operation can be
the matrix into the reduced row-echelon form.
reversed by another elementary row operation of
the same type.
4.3 Comparison of Different
Forms
$ 4 Row Reduction on Ma-
trices • A ladder form is similar to a row-echelon form,
except that a non-zero row needs not start with
1.
4.1 Preliminaries
• In a row-echelon form, all entries beneath a
At the most fundamental level, to perform row reduc- leading one is 0. In a reduced row-echelon
tion on a matrix is to alternate between the following form, all entries above and beneath a leading
two processes: one is 0.
1. Finding a leading number (i.e., a small, non- • While a matrix can have several ladder forms
zero number) in a column — whenever appli- and row-echelon forms, it can only have one
cable. reduced row-echelon form.
5.1 Procedure for Solving a Sys- • A homogeneous system — a system whose con-
tem stant terms are all zero — has at least the trivial
solution.
To solve a system of linear equations with m equa-
tions and n variables using matrices, proceed as
follows: $ 6 Determinant
1. Convert the equations into an augmented ma-
trix — with the m × n coefficient matrix on the 6.1 Definition
left and the m × 1 constant matrix on the right.
Given an n × n matrix A, the determinant of A —
2. Reduce the augmented matrix into a ladder or det(A) for short — is a scalar quantity which can
form, or — if needed — a row-echelon form be defined recursively:
or a reduced row-echelon form. Once there,
three scenarios ensue: • For a 2 × 2 matrix
a b
:
c d
• If an inconsistent row — a row with zero
everywhere except the last entry — pops
a b df
det = ad − bc
up during the row reduction process, then c d
the original system is unsolvable.
• For a n×n matrix A, Caij — the cofactor of the
• If the reduced matrix has n leading num- ij entry of A — is defined to be the signed de-
bers and no inconsistent row exists, then terminant of the matrix resulted from removing
the system has a unique solution. the ith row and the j th column of A.
• If the reduced matrix has less than n lead- • For a general n × n matrix A (with n ≥ 3),
ing numbers and no inconsistent row ex- the determinant can be defined as the cofactor
ists, then the system has infinitely many expansion along the first row of A:
solutions, in which case: df
det(A) = a11 Ca11 + · · · + a1n Ca1n
• When converted back into equation
form, the system will have less than
n leading variables. 6.2 Facts
• By turning the non-leading variables
Guveb an n × n matrix A, det(A) can be obtained
into parameters and applying back
by cofactor-expanding along any row or any column
substitution, we can then find the gen-
of A. As a result:
eral solution to the system — along
with the basic vectors that generate • If A has a row of zeros or a column of zeros,
them. then det(A) = 0.
• If A is upper or lower-triangular, then det(A)
5.2 Facts is the product of its main-diagonal entries.
6 Determinant 5
A Comprehensive Summary on Introductory Linear Algebra
Row/Column Multiplication
det · · · Ci · · · Cj + kCi · · · =
. . det · · · Ci · · · Cj · · ·
.. ..
det kRi = k det Ri
.. .. 6.4 Properties on Matrices
. .
Given a n × n matrix A:
det · · · kCi · · · = k det · · · Ci · · ·
7 Inverse 6
A Comprehensive Summary on Introductory Linear Algebra
In which case, since B is the only matrix with such In particular, in the case of a 2 × 2 matrix:
properties, it is referred to as the inverse of A — or
A−1 for short. d −b
−1
−c a
a b
The following claims are all equivalent: c d
=
ad − bc
• A is invertible.
• det(A) 6= 0 • Row reduction method: By writing A and I
alongside each other and carry out row reduc-
• The equation Ax = 0 (where x and 0 are n- tion until A is reduced to the identity matrix,
entry column vectors denoting the variable vec- the original I would be reduced to A−1 as well:
tor and the zero vector, respectively) has only
the trivial solution. Row Reduction −1
A I −−−−−−−→ I A
• The reduced row-echelon form of A is I.
• The equation Ax = b has a unique solution
for each n-entry column vector b. 7.3 Properties on Matrices
• The rows of A are linearly independent. Given an invertible n × n matrix A:
• The columns of A are linearly independent. −1
• (A−1 ) =A
A−1
7.2 Procedures for Finding In- • (kA) −1
= (where k 6= 0)
k
verses −1 T
• AT = (A−1 )
Ï 7.2.1 Terminologies
In addition, if B is also an invertible n × n matrix,
Given a n × n matrix A: then:
(AB)−1 = B −1 A−1
• The cofactor matrix of A is the n × n matrix
whose ij entry is the cofactor of aij . In particular:
m
• Adj(A), the adjoint of A, is the transpose of (Am )−1 = (A−1 ) (where m ∈ N)
the cofactor matrix of A.
$ 8 Elementary Matrices
Ï 7.2.2 Procedures
Given an invertible n × n matrix A, two methods 8.1 Definition
for finding A−1 exist:
An n × n elementary matrix is a matrix obtainable
• Adjoint method: Since A Adj(A) = det(A)I by performing one elementary row operation on
and det(A) 6= 0, A−1 can be determined using the n × n identity matrix I.
the following formula:
As a result, three types of elementary matrices exist:
−1 Adj(A)
A = • Those resulted from row multiplication
det(A)
8 Elementary Matrices 7
A Comprehensive Summary on Introductory Linear Algebra
• Performing an elementary row operation on • The equation (λI − A)x = 0 has a non-zero
A is equivalent to left-multiplying A with the solution.
n × n elementary matrix associated with the • det(λI − A) = 0
said operation.
• Since each elementary row operation is re- In other words:
versible, each elementary matrix is invertible
• If we define det(xI − A) as the characteristic
In particular, if A is an n × n invertible matrix, then: polynomial of A, then its roots are precisely
the eigenvalues of A.
• A−1 can be conceived as the series of ele- • Once an eigenvalue λ is determined, its
mentary row operations leading A to I (i.e., eigenspace and basic eigenvectors can also be
A−1 = En . . . E1 ). found by solving the equation (λI − A)x = 0.
• Similarly, A can be conceived as the series of
elementary row operations leading I to A (i.e.,
A = E1−1 . . . En−1 ). 9.3 Properties of Eigenvectors
More schematically:
• Since eigenspaces of distinct eigenvalues are
E1 ··· En — apart from the zero vector — disjoint from
A
9 Diagonalization 8
A Comprehensive Summary on Introductory Linear Algebra
(Note: This formula can be used to compute • v1 , . . . , vn are linearly dependent ⇐⇒ one
any power of A quickly.) of the vector vi can be expressed as a linear
combination of the other vectors.
• Tr(A) = λ1 + · · · + λn
• v1 , . . . , vn is a basis of V =⇒ every vector in
• det(A) = λ1 × · · · × λn
V can be expressed as a linear combination of
V in a unique way.
• v is a linear combination of v1 , . . . , vn ⇐⇒
Ï 11.3.1 Definitions
the augmented matrix v1 . . . vn v is
Given a m × n matrix A:
solvable.
11 Subspaces 10
A Comprehensive Summary on Introductory Linear Algebra
12 Operations on Vectors 11
A Comprehensive Summary on Introductory Linear Algebra
• u · u = |u|2 • v×v =0
• u·0=0·u=0 • u × v = −(v × u)
• u · (v + w) = u · v + u · w • (u + v) × w = u × w + v × w
• (u + v) · w = u · w + v · w • (u × v) · u = 0, (u × v) · v = 0
u
• u · (v × w) = det v
12.4 Cross Product
w
u×v =
df Given a vector v and a non-zero directional vector
d in R3 (or in R2 ):
u u3 u1 u3 u1 u2
det 2 , − det , det
v2 v3 v1 v3 v1 v2 df
• projd v = the projection of v onto d
In the case where u and v are non-zero vectors, we df
• oprojd v = the orthogonal projection of v
have that: onto d
df
|u × v| = |u| |v| sin θ • refld v = the reflection of v about d
df
(θ = the angle between u and v) oprojd v
projd v
From which it follows that: refld v v
d
u × v = 0 ⇐⇒ u and v are parallel.
for the missing value of t — and hence • |v| would give the distance between
determine the coordinates of X as well. Q and P.
−−→
3. Once there, |QX| would give the distance • Q + v would give the point on P clos-
between Q and `. est to Q.
13.4 Point vs. Plane 2. By plugging X (in the above form) into
the system:
Given a point Q = (x1 , y1 , z1 ) and a plane P in R3 : (−−→
QX · d1 = 0
−−→
• If P is in the point-direction form P + d1 s + QX · d2 = 0
d2 t:
we can solve for the missing values of s
df
Q is on P ⇐⇒ Q = P + d1 s + d2 t and t — and hence determine the coordi-
for some numbers s and t nates of X as well.
−−→
• If P is in the standard form ax + by + cz = k: 3. Once there, |QX| would give the distance
df between Q and P.
Q is on P ⇐⇒ ax1 + by1 + cz1 = k
In the case where Q is not on P, the distance be- • Intersection Point Approach (for P in standard
tween Q and P can be determined in three ways: form)
2. Once there:
• `1 and `2 are parallel ⇐⇒ d1 and d2 are In what follows, we assume that a plane is always
parallel. converted into standard form for easier analysis.
More specifically, if ` is in the form (x(t), y(t), z(t))
• `1 and `2 are intersecting ⇐⇒ the equation with directional vector d and P is in the form ax +
P1 + d1 s = P2 + d2 t is solvable for some num- by + cz = k with n = (a, b, c), then:
bers s and t.
(in the case where such a (s, t) pair exists • ` and P are parallel ⇐⇒ d · n = 0
and is unique, the coordinates of the intersec- • ` and P are intersecting ⇐⇒ the equation
tion point can be determined by, say, back- ax(t) + by(t) + cz(t) = k is solvable for some
substituting the value of s into P1 + d1 s.) t. Moreover:
In the case where `1 and `2 don’t intersect, the dis- • If the equation holds for all t, then ` and
tance between them can be determined as follows: P are overlapping.
• If the equation holds for a single t, then
• If `1 and `2 are parallel, then the distance be-
` and P intersect at a unique point —
tween them is simply the distance between `1
whose coordinates can be determined
and any point on `2 .
by back-substituting the value of t into
• If `1 and `2 are non-parallel, then: ` : (x(t), y(t), z(t)).
1. The shortest distance must occur between In the case where ` and P are non-intersecting
a point X1 = P1 + d1 s on `1 , and a point (hence parallel), the distance between ` and P is
X2 = P2 + d2 t on `2 . simply the distance between P and any point on `.
13.7 Plane vs. Plane • f (v1 + v2 ) = f (v1 ) + f (v2 ) for all n-entry
vectors v1 and v2 .
In R3 , any pair of planes must fall into exactly one
• f (kv) = kf (v) for all numbers k and n-entry
of the following categories:
vectors v.
• Parallel intersecting (i.e., overlapping)
In fact, any function from Rn to Rm with these two
• Parallel non-intersecting properties must be a matrix transformation as well.
• Non-parallel intersecting
14 Matrix Transformation 16