LINEAR ALGEBRA & ODE

MATH
MATHEMATICS


INTODUCTION
ALGEBRA IS THE STUDY OF OPERATIONS AND THEIR
APPLICATIONS TO SOLVE EQUATIONS.
LINEAR ALGEBRA HAS WIDE APPLICATIONS IN VARIOUS
MATHEMATICAL AS WELL AS PHYSICAL DISCIPLINES
SUCH AS DIFFERENTIAL GEOMETRY, FUNCTIONAL
ANALYSIS, THEORY OF MATRICES, PHYSICS ETC.
APPLICATIONS OF LINEAR ALGEBRA TO OTHER
BRANCHES OF SCIENCE, ENGINEERING AND ECONOMICS
GENERALLY OCCUR VIA THE NEED TO SOLVE SYSTEMS
OF LINEAR EQUATIONS. ONE OF THE MAIN OBJECTIVE OF
LINEAR ALGEBRA & ODE
STUDYING LINEAR ALGEBRA IS TO FIND THE MOST
ECONOMIC WAY OF MANIPULATING AND SOLVING
SYSTEMS OF LINEAR EQUATIONS.
IT IS A TWO OPERATIONAL ALGEBRAIC STRUCTURE
WHICH CONNECTS BEAUTIFULLY THE TWO STRUCTURES
NAMELY GROUP AND FIELD.
APPLICATIONS
COMPUTER SCIENCE APPLICATIONS: WAVELETS,
TRANSFORMATIONS IN COMPUTER GRAPHICS,
COMPUTER VISION, LINEAR PROGRAMMING, LINEAR
REGRESSION, MARKOV CHAINS ETC.
OTHER APPLICATIONS: LINEAR AND NONLINEAR
OPTIMIZATION, CONTROL THEORY, COMBINATORIAL
OPTIMIZATION, NUMERICAL SOLUTIONS TO ODES,
ANALYSIS OF ELECTRICAL NETWORKS, QUNATUM
MECHANICS.





LINEAR ALGEBRA & ODE

COURSE OUT LINE
1. Matrices
Introduction to matrices, Algebra of matrices, Special
matrices.
Elementaryrowoperations
Echelonandreducedechelonform
Inverseofamatrixbyelementaryrowoperations
2. Determinants
Introductiontodeterminants
Propertiesofdeterminantofordern
Applicationsofdeterminants.
3. Linear Systems of Equations
Introductiontolinearsystems
Homogeneousandnonhomogeneouslinearequations
GuasseliminationandGaussJordanmethods
Consistencycriterionforsolutionoflinearsystems
Applicationsoflinearsystems
4. Vector Spaces and Subspaces
Vectorspacesandsubspaces
Linearcombinationofvectors
Linearindependenceanddependence
LinearTransformation
5. Eigenvalues and Eigenvectors
Introductiontoeigenvaluesandeigenvectors
Diagonalization
Applicationsofeigenvaluesandeigenvectors
6. Ordinary Differential Equations
Differentialequationsandtheirclassification
Formationofdifferentialequations

LINEAR ALGEBRA & ODE

7. First order Differential Equations
Solutionofdifferentialequations
Separabledifferentialequations
Homogeneousdifferentialequations
Exactdifferentialequations
SolutionsofdifferentialequationsbyIF
Lineardifferentialequations
Bernoullidifferentialequations
Applicationsoffirstorderdifferentialequations
8. Second Order Differential Equations
Solutions of homogeneous linear 2
nd
and higher order
differentialequations
Solutionsofnonhomogeneouslinear2
nd
andhigherorder
differentialequations
TheCauchyEulerequations
Applicationsof2
nd
andhigherorderdifferentialequations


BOOKS
a. INTRODUCTION TO LINEAR ALGEBRA WITH
APPLICATIONS BY BERNARD KOLMAN.
b. A COURSE IN LINEAR ALGEBRA BY HALL MOORE.
c. INTRODUCTION TO LINEAR ALGEBRA BY LEE W
JOHNSON.
d. ADVANCED ENGINEERING MATHEMATICS BY E.
KREYSZIG, 9
TH
ED.
e. ADVANCED ENGINEERING MATHEMATICS BY D. G.
ZILL M. R. CULLEN.
LINEAR ALGEBRA & ODE
f. DIFFERENTIAL EQUATIONS WITH BOUNDARY
VALUE PROBLEMS BY D. G. ZILL
g. TOPICS IN ALGEBRAS BY HERSTIEN.
h. LINEAR ALGEBRA BY A. M. TROOPER.
i. ELEMENTARY LINEAR ALGEBRA BY H. ANTON.
j. LINEAR ALGEBRA BY STEPHEN H.

GRADING CRITERIA
ASSIGNMENTS 10%
QUIZZES 10%
MID TERM TEST 30%
FINAL 50%













LINEAR ALGEBRA & ODE

INTRODUCTION TO MATRICES
MATRICES OCCUR IN VARIOUS PROBLEMS SUCH AS
ARRAYS OF COEFFICIENTS OF EQUATIONS. FOR
EXAMPLE, MATRIX OF THE SYSTEM OF EQUATIONS:
0 2 4
0 4 3
= +
= +
z x
z y x

IS THE ARRAY OF COEFFICIENTS
.
2 0 4
1 4 3

SUPPOSE A COMPANY DEALS IN THREE DIFFERENT
PRODUCTS I, II AND III. THE WEEKLY SALES (IN
THOUSANDS) CAN BE REPRESENTED IN A MATRIX FORM
AS FOLLOWS:

M T W T F S

68 33 17 0 0 9
86 30 40 68 10 0
37 21 0 71 23 30


IF THE COMPANY HAS FIVE STORES, WE CAN SET UP FIVE
SUCH MATRICES, ONE FOR EACH STORE, THEN BY
ADDING CORRESPONDING ENTRIES OF THOSE MATRICES.
LINEAR ALGEBRA & ODE


WE CAN GET A MATRIX SHOWING THE TOTAL SALES OF
EACH PRODUCT ON EACH DAY.

MATRICES ARE USEFUL BECAUSE THEY ENABLES US TO
CONSIDER AN ARRAY OF MANY NUMBERS AS A SINGLE
OBJECT, DENOTE IT BY A SINGLE SYMBOL, AND PERFORM
CALCULATIONS WITH THOSE SYSMBOLS IN A VERY
COMPACT FORM.

MATRICES HAVE VARIOUS ENGINEERING APPLICATIONS,
SUCH AS, THEY CAN BE USED TO CHARACTERIZE
CONNECTIONS IN ELECTRICAL NETWORKS, IN NETS OF
ROADS, IN PRODUCTION PROCESSES.

WITH EACH MATRIX WE CAN ASSOCIATE A UNIQUE REAL
NUMBER CALLED ITS DETERMINANT. DETERMINANTS
ARE USED FOR SOLVING CERTAIN SYSTEMS OF LINEAR
EQUATIONS AND FOR FINDING EIGENVALUES OF
MATRICES.


LINEAR ALGEBRA & ODE

REVIEW OF MATRICES AND DETERMINANTS
1. MATRIX (MATRICES)
SQUARE MATRIX
RECTANGULAR MATRIX
2. ALGEBRA OF MATRICES
EQUALITY
ADDITION OF MATRICES
SUBTRACTION OF MATRICES
MULTIPLICATION OF MATRICES
ADDITIVE INVERSE OF A MATRIX
SCALAR MULTIPLICATION
3. TYPES OF MATRICES
NULL MATRIX OR ZERO MATRIX
DIAGONAL MATRIX
SCALAR MATRIX
IDENTITY MATRIX

LINEAR ALGEBRA & ODE
UPPER TRIANGULAR MATRIX
LOWER TRIANGULAR MATRIX
PERIODIC MATRIX
IDEMPOTENT MATRIX
NILPOTENT MATRIX
INVOLUTORY MATRIX

4. TRANSPOSE OF A MATRIX

5. SYMMETRIC & SKEW SYMMETRIC MATRIX

6. ORTHOGONAL MATRIX

7. CONJUGATE OF A MATRIX

8. HERMITIAN & SKEW HERMITIAN MATRIX

9. UNITARY MATRIX

10. STOCHASTIC MATRIX

11. LINEAR COMBINATION

LINEAR ALGEBRA & ODE

12. DETERMINANTS AND THEIR PROPERTIES

13. MINOR OF AN ELEMENT OF A MATRIX

14. COFACTORS OF ELEMENTS OF A MATRIX

15. SINGULAR AND NON-SINGULAR MATRICES

16. ADJOINT OF A MATRIX

17. INVERSE OF A SQUARE MATRIX

18. ELEMENTARY MATRICES

19. ROW AND COLUMNS OPERATIONS

20. ECHELON & REDUCED ECHELON FORM

21. HOMOGENEOUS AND NON-HOMOGENEOUS LINEAR
EQUATIONS

22. CRAMERS RULE

LINEAR ALGEBRA & ODE

MATRICES AND DETERMINANTS


INTRODUCTION
IT IS APPARENT, OR SHOULD BE AT ANY RATE, THAT IN
THE PROCESS OF SOLVING SIMULTANEOUS LINEAR
EQUATIONS, THE USAGE OF MATRICES IS BENEFICIAL.
FOR THIS REASON, WE INTRODUCE IN THIS SECTION THE
MATRICES ALGEBRA AND DETERMINANTS OF MATRICES.

A MATRIX, AS WE HAVE ALREADY EXPLAINED, IS A
RECTANGULAR ARRAY OF ELEMENTS OF A FIELD F

(REAL OR COMPLEX).
THAT IS, IF
ij
a
IS IN
, F
WHERE
m i ,..., 2 , 1 =
AND
n j ,..., 2 , 1 =
THEN, BY
[ ]
ij
a
WE MEAN A
MATRIX
.
. . .
. . .
. . .
. . .
. . .
2 1
2 22 21
1 12 11

mn m m
n
n
a a a
a a a
a a a

NOTICE THAT
[ ]
ij
a
IS A MATRIX WITH
m
ROWS AND
n

COLUMNS AND HENCE IS OF ORDER m n.

LINEAR ALGEBRA & ODE
IF m = n, IT IS CALLED A RECTANGULAR MATRIX
AND IF m= n, THEN IT IS KNOWN AS SQUARE MATRIX.
A MATRIX IN WHICH
0 =
ij
a
FOR ALL
i
AND
j
IS
CALLED A ZERO MATRIX, WHEREAS A MATRIX IN WHICH
1 =
ij
a
IF
i
=
j
AND
0 =
ij
a
IF
j i
IS CALLED AN
IDENTITY MATRIX.

TWO MATRICES
[ ]
ij
a
AND
[ ],
ij
b
WHERE
m i ,..., 2 , 1 =
AND
n j ,..., 2 , 1 =
ARE SAID TO BE EQUAL IF
AND ONLY IF ij ij
b a =
FOR ALL
i
AND
. j


TWO MATRICES
[ ]
ij
a
AND
[ ],
ij
b
WHERE
m i ,..., 2 , 1 =

AND
n j ,..., 2 , 1 =
CAN BE ADDED AS
[ ] [ ] [ ]
ij ij ij
c b a = +

WHERE
.
ij ij ij
b a c + =
NOTE THAT THE ADDITION FOR
TWO MATRICES IS POSSIBLE ONLY IF THEY HAVE THE
SAME ORDERS.

IF
F k
IS ANY SCALAR AND
[ ]
ij
a
IS A MATRIX AS
DEFINED ABOVE, THEN BY k|a
|j
j WE MEAN A MATRIX
LINEAR ALGEBRA & ODE
|ka
|j
j IN WHICH EACH ENTRY IS MULTIPLIED BY THE
SCALAR k
, KNOWN AS SCALAR MULTIPLICATION.


THE PRODUCT OF TWO MATRICES IS LITTLE
COMPLICATED. IN ORDER TO MULTIPLY TWO MATRICES,
NAMELY,
[ ]
ij
a
AND
[ ]
ij
b
THEY FIRST NEED TO BE
COMPATIBLE. THAT IS, THE TWO MATRICES HAVE TO BE
OF ORDERS SUCH THAT THE NUMBER OF COLUMNS IN

THE FIRST MATRIX SHOULD BE EQUAL TO THE NUMBER
OF ROWS IN THE SECOND MATRIX.

LET
A
BE AN
p m
MATRIX WITH ENTRIES
. F a
ij


THEN THE PRODUCT
AB
IS THE MATRIX
C
WHOSE
th j i ) , (
ENTRY IS
kj
p
k
ik ij
b a c

=
=
1

LET
A
BE AN
n m

MATRIX OVER A FIELD
. F
OUR
EARLIER SHORTHAND NOTATION,
, Y AX =
FOR
SYSTEMS OF LINEAR EQUATIONS IS CONSISTENT WITH
LINEAR ALGEBRA & ODE
OUR DEFINITION OF MATRIX PRODUCTS. FOR IF

=
n
x
x
x
X
.
.
.
2
1

WITH , F x
i
THENAX IS THE 1 m MATRIX

=
n
y
y
y
Y
.
.
.
2
1
SUCH THAT
. ...
2 2 1 1 n in i i i
x a x a x a y + + + = A PRODUCT OF MATRICES DEPENDS

UPON THE ORDER IN WHICH THE FACTORS ARE
WRITTEN, BUT IT IS INDEPENDENT OF THE WAY IN
WHICH THEY ARE ASSOCIATED. THIS MEANS THAT IF
MATRICES C B A , , ARE COMPATIBLE FOR
MULTIPLICATION THEN A(BC) = (AB)C.

A MATRIX OF ORDER n n IS CALLED A SQUARE
MATRIX. FOR INSTANCE IDENTITY MATRICES [ ],
ij
a WHERE
1 =
ij
a IF i = j AND 0 =
ij
a IF j i , ARE ALL SQUARE MATRICES.
IF A IS A SQUARE MATRIX THEN AA IS DENOTED BY A
2
.


IF A AND B ARE MATRICES OF COMPATIBLE
ORDERS, THEN IT IS NOT ALWAYS NECESSARY THAT
LINEAR ALGEBRA & ODE
. BA AB =

FOR INSTANCE
,
1 0
1 1
,
1 1
0 1

= B A
THEN

=
0 1
1 1
AB

AND

=
1 1
1 0
BA
SHOWING THAT
. BA AB
THUS IN
GENERAL COMMUTATIVE LAW FOR MULTIPLICATION
DOES NOT HOLD IN THE CASE OF MATRICES.

WE CAN PUT ALL THESE PROPERTIES OF MATRICES
IN THE FORM OF A THEOREM.


LINEAR ALGEBRA & ODE

THEOREM - 1
LET
) (F M
n

DENOTE A SET OF ALL MATRICES OF
ORDER
n n
WITH ENTRIES FROM THE FIELD F .
(i) IF
) (F M A
n

AND
, F k
THEN
), (F M kA
n

(ii) IF
), ( , F M B A
n

THEN
), (F M B A
n
+

(iii) IF
), (F M A
n

THEN
, 0 0 A A + = +
WHERE
), ( 0 F M
n

(iv) IF
), (F M A
n

THEN THERE EXISTS

) (F M A
n

SUCH
THAT
, 0 ) ( ) ( = + = + A A A A

(v) IF
), ( , , F M C B A
n

THEN
, ) ( ) ( C B A C B A + + = + +

(vi) IF
), ( , F M B A
n

THEN
, A B B A + = +

(vii) IF
F k
AND
), ( , F M B A
n

THEN
, ) ( kB kA B A k + = +

(viii) IF
F k k
2 1
,
AND
), (F M A
n

THEN
, ) (
2 1 2 1
A k A k A k k + = +

(ix) IF
F k k
2 1
,
AND
), (F M A
n

THEN ), ( ) (
2 1 2 1
A k k A k k =
(x) IF I

DENOTE THE MULTIPLICATIVE IDENTITY OF
, F

THEN
A IA =
FOR ALL
). (F M A
n

LINEAR ALGEBRA & ODE

SPECIAL TYPES OF MATRICES
A SQUARE MATRIX IS CALLED A DIAGONAL MATRIX IF IT
IS OF THE TYPE

n
k
k
k
. . . 0 0
. . .
. . .
. . .
0 . . . 0
0 . . . 0
2
1

WHERE
. ,..., ,
2 1
F k k k
n

LET
A
BE A DIAGONAL MATRIX
WITH DIAGONAL ENTRIES
. ,..., ,
2 1 n
k k k
THEN
A
IS CALLED
A SCALAR MATRIX IF
. ...
2 1 n
k k k = = =


IF
[ ]
ij
a A =
IS AN
n m
MATRIX, THE TRANSPOSE OF
A
IS THE
m n
MATRIX DENOTED BY
t
A
OBTAINED BY
INTERCHANGING THE ROWS AND COLUMNS OF
, A
THAT
IS
[ ]
ij
t
a A =
, WHERE
n i a b
ij ij
,..., 2 , 1 , = =
AND
. ,..., 2 , 1 m j =
FOR
INSTANCE, IF
LINEAR ALGEBRA & ODE

=
1 8 3
0 1 2
A
AND
.
1 0
8 1
3 2

=
t
A


THEOREM 2
IF A AND
), (F M B
n

THEN
(i)
t t t
B A B A + = + ) (
PROVIDED
B A,
ARE OF THE SAME
ORDERS,
(ii)
t t t
A B AB = ) (
PROVIDED
B A,
ARE COMPATIBLE BY
MULTIPLICATION.
IF
[ ]
ij
a A =
IS A SQUARE MATRIX OF ORDER
n
THEN
A SQUARE MATRIX
B
OF ORDER
n
IS CALLED INVERSE
OF
A
IF AND ONLY IF
.
n
I BA AB = =


REMARKS
(i) THE INVERSE OF A MATRIX IS UNIQUE,
(ii) IF
B A,
ARE MATRICES WITH INVERSES, THEN
AB

IS ALSO A MATRIX WITH INVERSE. ALSO,
. ) (
1 1 1
= A B AB

(iii) A MATRIX WHICH HAS AN INVERSE IS CALLED A
NON-SINGULAR OR INVERTIBLE MATRIX. IF IT DOES
LINEAR ALGEBRA & ODE
NOT HAVE AN INVERSE THEN IT IS CALLED A
SINGULAR OR NON-INVERTIBLE MATRIX.
(iv) IF
A
IS A SQUARE MATRIX THEN
t
A
IS INVERTIBLE
IF AND ONLY IF
A
IS INVERTIBLE. FURTHERMORE,
. ) ( ) (
1 1 t t
A A

=


A SQUARE MATRIX
A
OF ORDER
n
IS CALLED
SYMMETRIC MATRIX IF
, A A
t
=
SKEW SYMMETRIC IF
, A A
t
=
ORTHOGONAL IF
.
n
t t
I AA A A = =


EXAMPLES
THE MATRICES:
,
1 2 1
2 1 1
1 1 1

,
1 2 1
2 1 1
1 1 1

1 2 1
2 1 1
1 1 1

ARE SYMMETRIC, SKEW SYMMETRIC AND ORTHOGONAL.

ELEMENTARY MATRICES
IF
B
IS A GIVEN MATRIX AND
C
IS OBTAINED FROM
B
BY MEANS OF ELEMENTARY ROW OPERATIONS, THEN
EACH ROW OF
C
IS A LINEAR COMBINATION OF THE
ROWS OF
B
, AND HENCE THERE IS A MATRIX
A
SUCH
THAT
C AB =
. IN GENERAL, THERE ARE MANY SUCH
LINEAR ALGEBRA & ODE
MATRICES
, A
AND AMONGST ALL SUCH IT IS
CONVENIENT AND POSSIBLE TO CHOOSE ONE HAVING A
NUMBER OF SPECIAL PROPERTIES. BEFORE GOING INTO
THIS WE INTRODUCE A CLASS OF MATRICES.

AN
n m
MATRIX IS CALLED AN ELEMENTARY
MATRIX IF IT CAN BE OBTAINED FROM THE
n m
IDENTITY MATRIX BY MEANS OF SINGLE ELEMENTARY
ROW OPERATION.

FOR INSTANCE A
2 2
ELEMENTARY MATRIX IS
NECESSARILY ONE OF THE FOLLOWING
,
0 1
1 0

,
1 0
0 1

,
1 0
0 1

,
1 0
0 0

0 0
0 1

ELEMENTARY MATRICES ARE ONE OF THE FOLLOWING
THREE TYPES CORRESPONDING TO THE ELEMENTARY
ROW OPERATIONS
i
R
,
i
aR
,
i
R

j
R
,
i
R

j i
aR R +
.
,
1 0 0 0
0 1 0 0
0 0 0
0 0 0 1

a

,
0 1 0 0
1 0 0 0
0 0 1 0
0 0 0 1

.
1 0 0 0
0 1 0 0
0 1 0
0 0 0 1

a

LINEAR ALGEBRA & ODE

INVERSE OF A MATRIX
Let us find the inverse of
.
2 1 0
4 2 3
2 1 1

Solution

1 0 0 2 1 0
0 1 0 4 2 3
0 0 1 2 1 1

1 2 2
3R R R

1 0 0 2 1 0
0 1 3 2 5 0
0 0 1 2 1 1

2 3
R R

0 1 3 2 5 0
1 0 0 2 1 0
0 0 1 2 1 1

2 1 1
R R R +

0 1 3 2 5 0
1 0 0 2 1 0
0 0 1 0 0 1

2 3 3
5R R R

LINEAR ALGEBRA & ODE

0 1 3 2 5 0
1 0 0 2 1 0
0 0 1 0 0 1

3 3
8
1
R R

8
5
8
1
8
3
1 0 0
1 0 0 2 1 0
1 0 1 0 0 1

3 2 2
2R R R +

.
8
5
8
1
8
3
1 0 0
8
2
8
2
8
6
0 1 0
1 0 1 0 0 1

Therefore
.
8
5
8
1
8
3
4
2
4
1
4
3
1 0 1
1

A


COLUMN OPERATIONS

SO FAR, ELEMENTARY OPERATIONS ON MATRICES HAS
BEEN ROW OPEREATIONS. IN A SIMILAR FASHION WE
CAN DEFINE COLUMN OPERATIONS AND ELEMENTARY
COLUMN OPERATIONS. AS A CONSEQUENCE OF
ELEMENTARY COLUMN OPERATIONS ON A MATRIX, THE
MATRIX WILL BE COLUMN EQUIVALENT TO THE
ORIGINAL MATRIX.

LINEAR ALGEBRA & ODE
NOTE: WE SAY THAT TWO
n m
MATRICES A AND B
ARE EQUIVALENT IF THERE EXIST INVERTIBLE
MATRICES P AND Q SUCH THAT
. B PAQ =

EXAMPLE
IF ,
5 2 1
1 1 2
1 0 1

= M

THEN , 4 ) det( = M AND SO

+ +

+
=
1 2
0 1
2 1
0 1
2 1
1 2
1 2
1 1
5 1
1 1
5 1
1 2
1 1
1 0
5 2
1 0
5 2
1 1
) (M adj .
THEREFORE, .
1 2 3
1 6 11
1 2 7
4
1
1

M

CRAMERS SOLUTION OF LINEAR EQUATIONS

ANOTHER METHOD OF CALCULATING THE INVERSE
OF A MATRIX IS DUE TO CRAMER. WE EXPLAIN IT AS
FOLLOWS.
CONSIDER
n
LINEAR EQUATIONS IN
n
VARIABLES.
THAT IS,
.
1
i
n
j
j ij
b x a =

=
WHERE
. , , 2 , 1 n i K =
IN MATRIX
NOTATION THIS MEANS b AX = WHERE
). , , , ( ), , , , ( ), (
2 1 2 1 n
t
n
t
ij
b b b b x x x X a A K K = = = IF A

IS INVERTIBLE,
LINEAR ALGEBRA & ODE
THEN
b
A
A adj
b A X
) det(
) (
1
= =

OR FOR
; , , 2 , 1 n i K =

) det(
) det(
) det( ) 1 (
) det(
1
1
A
A
b A
A
X
i
j ji
j i
n
j
i
= =
+
=

WHERE
i
A
IS THE MATRIX
OBTAINED FROM A BY REPLACING ITS A det th i COLUMN
WITH THE COLUMN , b THAT IS

=
nn n n n
n
i
a b a a
a b a a
a b a a
A
n
. . . . . .
. . .
. . .
. . .
. . . . . .
. . . . . .
2 1
2 22 21
1 1 12 11
2
.
PERHAPS THE FOLLOWING EXAMPLE CAN EXPLAIN
THE APPLICATION OF CRAMERS RULE TO CALCULATE
SOLUTIONS OF A SET OF LINEAR EQUATIONS.
EXAMPLE
SOLVE THE LINEAR EQUATIONS:

-4x +2y -9z = 2
3x +4y +z = 5
x -3y +2z = 8
|A| = 3
x =
_
2 2 -9
5 4 1
8 -3 2
_
|3|
= 7

Similarly, y = -3, z = -4