Queuing Theory

Queuing Theory

• Queuing theory is the study of waiting in lines

or queues.

Server
Rear of Front of
queue queue
Server

Line (or queue) of customers

Server
Pool of potential
customers List of servers able to
service the customers
 Queuing Theory -- cont.

What do we want to know about a queuing

system?
• The average or expected wait time
• The percentage of customers who experience
long wait times
• The probability that a customer must wait
(The probability that all servers are busy)
• The average number of customers in the
queue.
• The probability that servers are idle
 Simulations

• Simulation is a method of evaluation without

using methmatical models such as queuing
theory.
• In general, a simulation is a computer-
programmed model of something.
• The best test of an OS: the real marketplace.
• The next best test: create situation similar to
the real world.
• Simulations play a key role in the
development of complex system such as
computer networks, databases, and OS
 Queueing Theory
• Customer arrive at a queueing system randomly at
time (arrival times)
t0  t1  t2    tn
 k  tk  tk 1
= interarrival times
= the time between successive arrivals
• Poisson arrival process: the interarrival times are
distributed exponentially:

P (  t )  1 e  lt

l : constant average arrival rate = customer/unit time

The # of arrivals / unit time is poisson distributed
with mean l
 P(   t )  1  e lt

y
y=1

P(  t) = P (   t )  1  e  lt
Probability of for small l
an arbitrary P (   t )  1  e  lt
interarrival time
 being less than t for large l

t=0
t (time)
 Example:
arrival rate: one every two minutes
The rate of one customer every two
minutes l = 0.5 customers per minute

t P(t)
P ( t )  1  e  lt  1  e  . 5t -------------------------------------
0 0
1 .393
2 .632
The probability function does not tell 3 .777
us when customers will arrive. 4 .865
It does, however, provide information 5 .918
about the random arrival process. .. ..
10 .993
.. ..
20 .99995
 Total # of customers in
the queueing system

N
Server 1

# of customers Ns
Average in the queue # of customers
arrival rate in service
Nq
l

 q
Service time
interarrival Time spent
time in the queue S
Server C

W
Total time a customer spends
in the queueing system
• The probability that exactly n customers will
arrive in any time interval of length t is

e lt ( lt ) n
( n  0, 1, 2)
n!

• Let Sk denotes the service time that the Kth

arriving customer requires from the system. An
arbitrary service time is referred to as S, and
the distribution of service time is
Q (t )  P ( s  t )
For random service with average service rate m
 mt
Q ( t )  P ( s  t )  1 e
 Queue Disciplines
• A queue discipline is the rule used for
choosing the next customer from the queue to
be serviced.
• Kendall notation
A/B/c
A is the interarrival time distribution
B is the service time distribution
c is the number of servers
A and B may be
– GI for general independent interarrival time
– G for general service time
– M for exponential interarrival or service time distribution
– D for deterministic interarrival or service time distribution
• M/M/1
• M/M/c
 Traffic Intensity
• A measure of that system's ability to service its
customer effectively
E( s) l
arrival rate
u  lE ( s ) 
E( ) m
service rate
It is useful for determining the minimum number
of identical servers a system will need in order
to service its customer without its queue
becoming indefinitly large or having to turn
customer away.
• Ex: E(s) = 17 sec E() = 5 sec

u = 17/5 = 3.4
need at least 4 servers
 Case Studies
A shared Laser printer
• An average of 64 requests occurring at
random times during eight-hour day.
• Each request require an average of about 5
minutes to print.
• Receiving compaints from employees that
they must wait nearly half an hour for their
printout.
8 requests/hour
l = 2/15 requests/minute
12 requests/hour
m = 1/ 5 request/minute
 Server Utilization
• Server utilization r is defined as the traffic
intensity per server
u l
r  r  2/3
c mc
• is the probability that a particular server is
busy
• this is approximately the fraction of time that
each server is in use
• For single-server system, u = r
Probability of All Server Idle

c 1
li lc
p0  [   ]1

i 0 i !* m i
c !* m c
*( 1  r )

When c= 1

p0  1  r (1 - 2/3 = 1/3)
 Probability of All Server Busy

• Erlang’s C Formula:

p0  lc
C ( c , l /m ) 
c ! m c  ( 1  r )

When c = 1

C ( c , l /m )  r (2/3)
Expected Number of Customers

• The expected number of customers in the

queue:

L q  rC ( c , l / m )/( 1  r )

When c = 1
2
r (2/3)2/(12/3)4/3
Lq 
1 r
 Expected Wait Time

• Little’s Formula

L q  lWq (2/15)* 10 = 4/3

• Expected wait time

Wq  C ( c , l / m )/[ m c  ( 1  r )]
When c = 1
r /m [(2/3)/0.2]/(1-2/3)
Wq 
1 r
= 10 minutes
 90th Percentile Wait Time

• 90% of the customers wait less than  q ( 90 )

 q ( 90 )  ln[ 10C ( c , l / m )]/[ m c  ( 1  r )]

When c = 1

ln( 10 r )
 q ( 90 )  ln(10*2/3)/(1/5 - 2/15)
ml = 28.4 minutes
 Case Study 2
Master Scheduler
• Computer running simulation programs
requires a lot of CPU time and scheduled on a
FIFO basis.
• Generally submit about 100 programs per day
• The programs require an average of about
one hour of CPU time.
100 requests/day
l = 100/24 = 4.2 requests/hour
1 requests/hour
m = 1 request/hour
 Example 1
• The election of the President of
Student Association（學生會）is P0  1  r
just finished in Tamkang C (c, l / m )  r
University. There are 27,000
students and the average rate of r2
voting is 15%. The official voting Lq 
1 r
period was 5 days, from 8:00 to
17:00. The association prepared r
only one counter for everyone to m
Wq 
vote and each voting requires half 1 r
minute to complete. Use the
equation provided to analyze the ln( 10 * r )
 q (90) 
condition of student getting in-line m l
and wait for the voting. You must
explain the meaning of each
equation. (ln(2.5)=0.9, ln(7.5)=2,
ln(15)=2.7)
Example 2
• There are 36 programmers P0  1  r
per day, in average, come in to C (c, l / m )  r
use the mainframe computer
r2
in the computing center. Each Lq 
1 r
programmer uses the
computer for about 15 r
m
minutes. Use the equation Wq 
1 r
provided to analyze the usage
of the computer. You must ln( 10 * r )
 q (90) 
explain the meaning of each m l
equation.