DIGRESSION A.

LINEAR INDEPENDENCE
Graphically,
A.1 Two vectors with the same direction are linearly
dependent; otherwise, they are linearly independent.

v2

⎡1 ⎤ x-axis 4
v=⎢ ⎥
⎣2⎦ y-axis
x
⎡3⎤
w=⎢ ⎥ 2
⎣1⎦
v
⎡2⎤ 1
x=⎢ ⎥
⎣4⎦ w
v1
1 2 3
A.2 A combination of linearly independent vectors is also
linearly independent of the vectors that form it.
(−1 / 5)v + (2 / 5)w = u

Multiply the number w/ each ⎡1 ⎤ ⎡3⎤ ⎡1⎤

element in the vector ( −1 / 5) ⎢ ⎥ + ( 2 / 5) ⎢ ⎥ = ⎢ ⎥
⎣ 2⎦ ⎣1⎦ ⎣0⎦
v2
v

v1
u u is independent of both v & w!
A.3 Two vectors, v and w, are linearly independent
if the scalars λ1 and λ2 satisfying
λ1v + λ2w = 0
are zero. Here 0 is a null vector.

If vectors are linearly dependent, then

λ2
v=− w λ2/λ1 = scalar multiple = constant
λ1
This means that v is just a scalar multiple of w.
Mathematically, two vectors are linearly dependent if one is the multiple of the other

Example:

⎡3⎤ ⎡1 ⎤
v=⎢ ⎥ w=⎢ ⎥ Linearly dependent
⎣6⎦ ⎣ 2⎦
 ⎡3⎤ ⎡1⎤ ⎡0⎤
λ1 ⎢ ⎥ + λ2 ⎢ ⎥ = ⎢ ⎥ ⇒ λ1 = 1 λ2 = −3
⎣6⎦ ⎣2⎦ ⎣0⎦

⎡3⎤ ⎡1⎤
x=⎢ ⎥ y=⎢ ⎥ Linearly independent
⎣6⎦ ⎣0⎦

⎡3⎤ ⎡1⎤ ⎡0⎤

λ1 ⎢ ⎥ + λ2 ⎢ ⎥ = ⎢ ⎥ ⇒ λ1 = 0 λ2 = 0
⎣6⎦ ⎣0⎦ ⎣0⎦
Ex: λ1 [ 1 2 ] + λ2 [ 2 4 ] = [ 0 0 ]
λ1 = -2 & λ2 = 0
 A.4 A set of vectors {a1, ....., am } in ℜn is a
linearly independent set if λ1a1 + ....+ λm am = 0
implies λ1 = ..... = λm = 0.

Examples:
1st & 2nd
Vectors are
⎡4 2 − 8 ⎤ ⎡4 2 5⎤
independent ⎢3 6 − 6⎥ ⎢ 3 6 1⎥
1st & 3rd
Vectors are
⎢ ⎥ ⎢ ⎥
dependent ⎢⎣1 8 − 2⎥⎦ ⎢⎣1 8 3⎥⎦

A.5 The rank of matrix A, ρ (A), is the number of

linearly independent columns in A.

⎡ 2 3⎤ ⎡3 6⎤
ρ⎢ =2 ρ⎢ ⎥ =1
⎥ ⎣ 2 4⎦
⎣8 6 ⎦
A.6 Some Properties:
a) Column rank = row rank Ex: With 2 rows
& 3 columns,
rows x columns
b) If A is m x n, then ρ(A) = integer ≤ min (m, n). rank cannot be >2
but can be <2
c) ρ(I) = n * Identity Matrix = 0 elements except the diagonal elements
* P = Permutation, similar to identity matrix where there are 1’s &
d) ρ(0) = 0, ρ(P) = n. 0’s, s.t. there’s only 1 non-zero element
e)  ρ(Aʹ) = ρ(A) = ρ(Aʹ A) Transpose = Rows of Matrix A = Columns of Matrix B
f) The rank of a matrix is not changed if one row
(column) is multiplied by a nonzero constant or if such
a multiple of one row (column) is added to another row
(column). Ex:
^J = [ 2 6 ] & J=[1 3]
[2 1] [2 1]
Notes: To multiply, multiply each element in the row of - Multiply the first row by
the 1st matrix to teach element in the 2nd matrix and sum -1 & add it to the 2nd row
- The column of the first matrix = row of the 2nd matrix so that you produce ^^J = [ 1 3 ] * -1 & 2
- m x n of the new matrix = row x column of the matrices [1 2]
Digression B. System of Linear Equations

B.1 The system of m linear equations in n unknowns:

a11 x1 + a12 x2 + ! + a1n xn = b1
"
an1 x1 + an 2 x2 + ! + ann xn = bn
is summarized by the matrix equation:
Ax=b
where A = [aij ] is the m x n coefficient matrix, x = [xj ]
is the column vector of variables, and b = [bj] is the
column vector of constants, i = 1, ...., m, j = 1, ...., n.

B.2 A solution to A x = b is a vector of x satisfying the

equation. If Ax = b has a solution, it is said to be
consistent. Otherwise, it is inconsistent.
B.3 The solution is trivial if x = 0. Any nonzero solution x
is nontrivial.

B.4 If the vector b = 0, then Ax = 0 is called a homogenous

equation.

B.5 The system of linear equations can have a unique

solution, a non-unique solution, or no solution.
where n = variables (i.e. x1, x2)
Augmented matrix

•  A solution exists iff ρ (A) = ρ (A | b) = r.

Ex: If A & A|B have common ranks
•  The solution is unique if r = n. n=5&r=3
There is a solution, but it’s not unique bec I
can assign n-r any value

•  If a solution exists, but r < n, then n - r of the

variables can be assigned arbitrary values.
B.6 Examples:
a) 3x1 + 8x2 = 53
6x1 + 2x2 = 50

⎡3 8 ⎤
ρ[ A] = ⎢ ⎥ =2
⎣6 2⎦

⎡3 8 53⎤
ρ [A b] = ⎢ ⎥ =2
⎣6 2 50⎦
•  unique solution

b) ⎡ 2⎤ ⎡6⎤
⎢8 ⎥ x1 = ⎢24⎥ since variable = 1, rank = 1
⎣ ⎦ ⎣ ⎦
•  unique solution
c) No solution
Rank of A = 1 & A|B = 2
No solution because:
⎡ 2⎤ ⎡6⎤
⎢3⎥ x1 = ⎢− 1⎥
2x1 = 6 -> x1 = 3
3x1 = -1 -> x1 = -1/3
⎣ ⎦ ⎣ ⎦ only 1 variable!

⎡2 4⎤ ⎡ x1 ⎤ ⎡ 6 ⎤
⎢3 6⎥ ⎢ x ⎥ = ⎢10⎥
p(A|B) = 2 but 3 variables
common rank is less than the unknowns
⎣ ⎦⎣ 2 ⎦ ⎣ ⎦
x1 + 2x2 - 3x3 = 4
2x1 + 4x2 - 6x3 = 6

Let us say that x3 = 0 :

⎡ x1 ⎤ 2x1 - 3x2 = 2 -> 2x1 = 3x2 + 2 -> x1 = 3/2 x2 + 1

⎡1 2 − 3⎤ ⎢ ⎥ ⎡ 4 ⎤
8x1 + 2x2 = 4 -> 8(3/2 x2 + 1) + 2x2 = 4
12x2 + 8 + 2x2 = 4
⎢2 4 − 6⎥ ⎢ x2 ⎥ = ⎢6⎥ x2 = -2/7, x1 = 4/7

⎣ ⎦ ⎣ ⎦
⎢⎣ x3 ⎥⎦ However, if x1 = 0
-3x2 + x3 = 2
This shows that the numbers depend on 2x2 = 4 -> x2 = 2
the arbitrary variable of what I assign x3 = 8
d) Non-unique solution

⎡3 6 ⎤ ⎡ x1 ⎤ ⎡ 9 ⎤ p(A) = 1 = p(A|B) but 2 variables :. r < n

solution exists but is not unique
⎢8 16⎥ ⎢ x ⎥ = ⎢24⎥ 3x1 + 6x2 = 9
⎣ ⎦⎣ 2 ⎦ ⎣ ⎦ 8x1 + 16x2 = 24
If x2 = 1 and I set a diff number for x1, I can get a diff value for x2

⎡2 4⎤ ⎡ x1 ⎤ ⎡0⎤
⎢ 3 6 ⎥ ⎢ x ⎥ = ⎢0 ⎥
⎣ ⎦⎣ 2 ⎦ ⎣ ⎦

⎡ x1 ⎤
⎡ 2 − 3 1 ⎤ ⎢ ⎥ ⎡ 2⎤ r = 1, n = 3

⎢8 2 0⎥ ⎢ x2 ⎥ = ⎢4⎥
p(A) = 2 :. p(A|B) = 2

⎣ ⎦ ⎣ ⎦
⎢⎣ x3 ⎥⎦