MATH3701 Probability and Stochastic Modelling

AIM:
Randomness is highly inuential in real-world systems (e.g. nancial markets, weather forecast-
ing, ecology), and random phenomena that change over time are known as stochastic processes.
So how should we model them? We start by describing the modern framework for probability
theory, before moving on to classify and investigate various useful stochastic processes, showing
how they may be applied to real-world problems.
SYLLABUS:
1. Axiomatic probability theory: Probability spaces (the elementary approach to prob-
ability and where it fails, axioms of probability, the Borel -algebra, properties of prob-
abilities); random variables (measurability, indicator variables, properties); Properties of
expectation; Conditional expectation (motivation, inner product spaces of random vari-
ables, denition and proof of optimality, properties).
2. Introduction to stochastic processes: Denition; State space / time set charac-
terisation; nite-order distributions; sample paths; strong and wide-sense stationarity;
Increments; The Markov property and Markov processes.
3. Markov Chains: Transition probabilities and the Chapman-Kolmogorov equations;
Markov Chains (transition matrices, transition graphs); Applications (insurance mod-
els, random walks, random walks with boundary conditions and the gamblers ruin, the
martingale betting strategy); Long-term behaviour (stationary distributions and period-
icity).
4. Markov Jump Processes: Transition matrices; Transition rates and generators; Kol-
mogorov forward and backward equations; Poisson process (generator and transition ma-
trices, interarrival times); Inhomogeneous processes and applications.
5. Queues: The Bernoulli single server queuing process: limited and unlimited capacity
queues, arrival process, service process; M/M/1 queuing process, limiting distributions;
M/M/k queuing process.
6. Brownian motion: Gaussian random variables; Standard Brownian motion; Reection
principle and distribution of the maximum; Brownian motion with drift and the gamblers
ruin.
1
ASSESSMENT:
1. Final exam: two-hour written paper 60%
2. In-course test: 20%
3. Group project: 20%
TEXT BOOKS:
1. Mark A. Pinsky, Samuel Karlin. An Introduction to Stochastic Modeling (Fourth Edi-
tion), Academic Press, 2011.
2. Sheldon M. Ross. Introduction to Probability Models (Ninth Edition), Academic Press,
2006.
2