Basic Linear Algebra in MATLAB

Russ Tedrake
9.29 Optional Lecture 2: February 13, 2002

In the last optional lecture we learned the the basic type in MATLAB is a matrix
of double precision floating point numbers. You learned a number of different tools
for initializing matrices and some basic functions that used them. This time, we’ll
make sure that we understand the basic algebraic operations that can be performed on
matrices, and how we can use them to solve a set of linear equations.

A Note on Notation
The convention used in this lecture and in most linear algebra books is that an italics
lower case letter (k) denotes a scalar, a bold lower case letter (x) denotes a vector, and
a capital letter (A) denotes a matrix. Typically we name our MATLAB variables with
a capital letter if they will be used as matrices, and lower case for scalars and vectors.

1 Vector Algebra
Remember that in MATLAB, a vector is simply a matrix with the size of one dimension
equal to 1. We should distinguish between a row vector (a 1xn matrix) and a column
vector (an nx1 matrix). Recall that we change a row vector x into a column vector
using the transpose operator (x0 in MATLAB). The same trick works for changing a
column vector into a row vector.
We can add two vectors, x and y, together if they have the same dimensions. The
resulting vector z = x + y is simply an element by element addition of the components
of x and y: zi = xi + yi . From this is follows that vector addition is both commutative
and associative, just like regular addition. MATLAB also allows you to add a scalar k
(a 1x1 matrix) to a vector. The result of z = x + k is the element by element addition
zi = k + x i .
Vector multiplication can take a few different forms. First of all, if we multiply a
scalar k times a vector x, the result is a vector with the same dimension as x: z = kx
implies zi = kxi . There are two standard ways to multiply two vectors together: the
inner product and the outer product.
The inner product, sometimes called the dot product, is the resultP of multiplying a
row vector times a column vector. The result is a scalar z = xy = i xi yi . To take
the inner product of two column vectors, use z = x 0 y. As we’ll see, the orientation of
the vectors matters because MATLAB treats vectors as matrices.

1
 Unlike the inner product, the result of the outer product of two vectors is a matrix.
In MATLAB, you get the outer product my multiplying a column vector times a row
vector: Z = xy. The components of Z are Zij = xi yj . To take the outer product of
two column vectors, use Z = xy 0 .
Occassionally, what we really want to do is to multiply two vectors together ele-
ment by element: zi = xi yi . MATLAB provides the .* command for this operation:
z = x. ∗ y.
To test our understanding, let’s try some basic matlab commands:
x = 1:5
y = 6:10
x+y
x+5
5*x
x*y’
x’*y
x.*y
How would you initialize the following matrix in MATLAB using outer products?
 
1 2 3 4 5
1 2 3 4 5 
 
1 2 3 4 5 
 
1 2 3 4 5 
1 2 3 4 5

2 Matrix Algebra
The matrix operations are simply generalizations of the vector operations when the
matrix has multiple rows and columns. You can think of a matrix as a set of row
vectors or as a set of column vectors.
Matrix addition works element by element, just like vector addition. It is defined
for any two matrices of the same size. C = A + B implies that C ij = Aij + Bij . Once
again, it is both commutative and associative. Scalar multiplication of matrices is also
defined as it was with vectors. The result is a matrix: C = kA implies C ij = kAij .
If you multiply a matrix times a column vector, then the result P is another column
vector - the column of inner products: b = Ax implies b i = j Aij xj . Similarly,
P a row vector times a matrix to get a row of inner products: b = xA
you can multiply
implies bi = j Aji xj . Notice that in both cases, the definitions require that the first
variable must have the same number of columns as the the second variable has rows.
This idea generalizes to multiplying two matrices together. For the multiplication
P
C = AB, the matrix C is simply a collection of inner products: C ik = j Aij Bjk .
In this case, A must have the same number of columns as B has rows. Like ordinary
multiplication, matrix multiplication is associative and distributive, but unlike ordinary
multiplication, it is not commutative. In general, AB 6= BA.
Now we are in a position to better understand the matrix transpose. If B = A 0 , then
Bij = Aji . Think of this as flipping the matrix along the diagonal. This explains why

2
the transpose operator changes a row vector into a column vector and vice versa. The
following identity holds for the definitions of multiplication and transpose: (AB) 0 =
B 0 A0 . This help us to understand the difference between x 0 A and Ax. Notice that for
column vector x, (Ax)0 = x0 A0 .
There are a few more matrix terms we should know. A square matrix is an nxn ma-
trix (it has the same number of rows and columns). A diagonal matrix A has non-zero
elements only along the diagonal (Aii ), and zeros everywhere else. You can initialize
a diagonal matrix in MATLAB by passing a vector to the diag command. The iden-
tity matrix is a special diagonal matrix with all diagonal elements set to 1. You can
initialize an idenitity matrix using the eye command.
Try the following matlab commands:

diag(1:5)

3 Solving Linear Equations

Let’s take a step back for a moment, and try to solve the following set of linear equa-
tions:

x1 + 3x2 = 4
2x1 + 2x2 = 9

With a little manipulation, we find that x1 = 4.75 and x2 = −0.25. We could solve
this set of equations because we had 2 equations and 2 unknowns. How should we
solve a set of equations with 50 equations and 50 unknowns?
Let’s rewrite the previous expression in matrix form:
    
1 3 x1 4
=
2 2 x2 9

Notice that we could use the same form, Ax = b, for our set of 50 equations with 50
unknowns. As expected, MATLAB provides all of the tools that we need to solve this
matrix formula, and it uses the idea of a matrix inverse.
The inverse of a square matrix A, which is A−1 in the textbooks and inv(A) in
MATLAB, has the property that A−1 A = AA−1 = I. Using this, let’s manipulate our
previous equation:

Ax = b
−1
A Ax = A−1 b
x = A−1 b

Now solve the original equations in MATLAB using inv(A) ∗ b. You should get the
vector containing 4.75 and -0.25.
There are a few things to remember about matrix inverses. First of all, they are only
defined for square matrices. It works with the transpose and multiplication operations
with the following identities: (A0 )−1 = (A−1 )0 and (AB)−1 = B −1 A−1 . You should

3
be able to verify these properties on your own using the ideas we’ve developed. But the
most important thing to know about matrix inverses is that they don’t always exist, even
for square matrices. Using MATLAB, try taking the inverse of the following matrix:
 
1 2
A=
2 4

Now try inserting this A into the system of equations at the beginning of this section,
and solving it using good old fashioned algebra. Why doesn’t the inverse exist?

4 Quadratic Optimization
We would like to solve the equation Ax = b even if A is not square. Let’s seperate the
problem into a few cases where the matrix A is an mxn matrix:
If m < n, then we have more unknowns than equations. In general, this system
will have infinitely many solutions.
If m > n, then we have more equations than unknowns. In general, this system
doesn’t have any solution. What if we don’t want to have matlab always return ”no
solution”, but we actually want the closest solution in the least squares sense? This is
equivalent to minimizing the following quantity:
1X X
E= ( Aij xj − bi )2
2 i j

MATLAB provides the backslash operator to accomplish the least squares fit for a
matrix equation: x = A \ b. Type help slash to appreciate the power of this command.
You’ll see that we could have used this command to solve the square matrix equations,
too.

5 Eigenmannia
How does this relate to the fish data that we used in problem set 1? Recall that we were
given a set of points (xi , yi ), and we were asked to find the coefficients a and b to fit
the following linear model:
yi ≈ a + bxi
You can think of each point as an equation, and write the entire data set in matrix form:
   
1 x1 y1
 1 x 2     y2 
  a  
 .. ..  b =  .. 
. .   . 
1 xm ym

If you call the left matrix A and the right side b, then calling A \ b will ask MATLAB
to solve for the values of a and b that minimize the least-squared error of the model. It
will return exactly the same values for a and b that the polyfit command returns.

4
 For this simple example, the polyfit command and the backslash command accom-
plished the same task. But what if you were given a set of points (x i , yi , zi ) and you
were asked to fit the following linear model:

zi ≈ a + bxi + cyi

The matrix notation easily scales to this problem, but the polyfit function does not.

6 Where to find more information

A good book on this material is Introduction to Linear Algebra by Gilbert Strang. All
of the material we’ve covered here can be found in the first two chapters of that book.
Sebastian also has a number matrix handouts from different courses which we can post
to the web if there is a demand for them. Make sure that you post to the class news
group athena.classes.9.29 if you have any questions.