Process Scheduling

Fred Kuhns

WASHINGTON UNIVERSITY IN ST LOUIS

Washington

Policy versus Mechanism

Policies set rules for determining when to switch and which process/thread to run Implementation consists of the data structures and algorithms used to implement policy Policy and Implementation influenced by platform - for example, context switch cost Policy must balance needs of different application types:
Interactive, Batch and Real-Time

Fred Kuhns (12/2/2012)

CS523 Operating Systems

Consider Three Basic Policies

First-In First-Out (FIFO)
runs to completion

Round Robin (RR)

runs for a specified time quantum

Time-Sharing (TS)
Multilevel feedback queues

Fred Kuhns (12/2/2012)

CS523 Operating Systems

Platform Issues
Interrupt latency Clock resolution Cost of a context switch
saving processor state and loading new instruction and data cache misses/flushes memory map cache (TLB) FPU state

Fred Kuhns (12/2/2012)

CS523 Operating Systems

Typical Scheduler Goals

Interactive
shells, GUI. Spend most of their time waiting for I/O. Minimize perceived delay (50-150msec) compiles, computations. Optimize throughput. Require predictable behavior. May require guarantees on throughput, latency or delay.

Batch

Real-time

Fred Kuhns (12/2/2012)

CS523 Operating Systems

Some RT Characteristics
Determinism - one measure: delay between interrupt assertion and executing the ISR Responsiveness - time to service interrupt User Control Reliability Fail-soft operation

Fred Kuhns (12/2/2012)

CS523 Operating Systems

Clock Interrupts
How system keeps track of time implemented using a hardware clock interrupt interrupt at periodic intervals (clock tick) typically 10msec, see tick frequency defined by HZ in param.h High priority, second to power-failure interrupt (NMI).

Fred Kuhns (12/2/2012)

CS523 Operating Systems

Interrupt Latency
time to recognize interrupt and execute first instruction of ISR - Determinism
hardware, for example will CPU finish current instruction. kernel time to dispatch interrupt to correct ISR

plus time to run ISR

affected by interrupt nesting

plus worst-case interrupt disable time

Fred Kuhns (12/2/2012)

CS523 Operating Systems

Clock Interrupt Handler

Update CPU usage for current process scheduler functions - priority computation, time-slice expiration (major/minor ticks) quota policing update time-of-day and other clocks callout queue processing process alarms run system processes

Fred Kuhns (12/2/2012)

CS523 Operating Systems

Callout queue
Queue of functions to be processed at a particular tick system context, base interrupt priority (sw int) Example: for real-time systems insertion time is important Typical to optimize lookup time not insertion time
time-to-fire, absolute time, timing wheel (fixed size circular array)
packet retransmission, system management functions, device monitoring and polling

Fred Kuhns (12/2/2012)

CS523 Operating Systems

10

Alarms
BSD: alarm(), setitimer() SVR4: alarm(), hrtsys() Real-time - actual elapsed time
sends SIGALRM

profiling - execution time

sends SIGPROF

virtual time - user mode time

sends SIGVTALRM

Fred Kuhns (12/2/2012)

CS523 Operating Systems

11

BSD Scheduler
Policy - Multilevel Feedback queues Policy Goal - good response time for interactive tasks while guaranteeing batch job progress Priority based, dynamically adjusted Preemptive time-slicing (time quantum)

Fred Kuhns (12/2/2012)

CS523 Operating Systems

12

4.4 BSD Process Scheduling

Scheduling priority range, hi to lo: 0 - 127 32 run queues (prio/4 = run queue) priority adjusted based on resource usage. time quantum set to 0.1 second for over 15 years! Sleep priorities
0-49 kernel mode 50-127 user mode

Fred Kuhns (12/2/2012)

CS523 Operating Systems

13

Scheduling Related Attributes

PROC structure
p_priority (kernel pri) p_usrpri (user pri) p_estcpu p_slptime

Predefined Process Priorities

PSWP PVM PINOD PRIBIO PVFS PZERO PSOCK PWAIT PLOCK PPAUSE PUSER 0 4 8 16 20 22 24 32 36 40 50 while swapping process wait for memory wait for file control info wait on disk I/O wait for kernel-level FS lock baseline priority wait on socket wait for child to exit wait for user-level FS lock wait for signal to arrive base user priority

...

Fred Kuhns (12/2/2012)

CS523 Operating Systems

14

Calculation of Priority
Proc structure:
p_estcpu - estimate of cpu utilization p_nice - user-settable (-20 to 19, default = 0)

priority (p_usrpri) recalculated every 4 clock ticks p_estcpu incremented each clock tick process is running every second CPU usage is decayed
sleeping processes are ignored

Fred Kuhns (12/2/2012)

CS523 Operating Systems

15

BSD Formulas
Priority calculation (PUSER = 50)

Decay calculation - schedcpu () each second

p_estcpu = (2load)/(2load+1)p_estcpu+p_nice ex, load=1: p_estcpu = 0.66T4 + + 0.13T0

p_usrpri = PUSER + (p_estcpu/4) + 2 p_nice

Processes sleeping for > 1 second

p_slptime set to 0 then incremented by 1 ea. second p_estcpu = p_estcpu((2load)/(2load+1))p_slptime

Fred Kuhns (12/2/2012)

CS523 Operating Systems

16

Context switch on 4.4 BSD

Synchronous vs. asynchronous Interprocess: voluntary vs. involuntary
voluntary - process blocks because it requires an unavailable resource. sleep () involuntary - time slice expires or higher priority process runnable. mi_switch ()

Fred Kuhns (12/2/2012)

CS523 Operating Systems

17

BSD - Voluntary Context Switch

Invoke sleep() with a wait channel (resource address) and (kernel) priority sleeping process organized as an array of queues. Wait channel is hashed to find bin. sleep() raises priority level splhigh (block interrupts). Set kernel priority (p_priority) wakeup(): remove process(es) from queue splhigh and recalculate user priority (p_usrpri).

Fred Kuhns (12/2/2012)

CS523 Operating Systems

18

BSD - Involuntary context switch

Results from an asynchronous event kernel schedules an AST, need_resched (), and sets global flag want_resched current proc checks this flag before returning to user mode. If set, then call

mi_switch () Note: BSD does not preempt process in kernel mode.

Fred Kuhns (12/2/2012)

CS523 Operating Systems

19

BSD Interprocess Context Switch

Change user and kernel context All user mode state located in
kernel-mode HW state - stored in PCB (u area) user-mode HW state - kernel stack (u area) proc structure (not swapped)

kernel changes the current process pointers and loads new state.

Fred Kuhns (12/2/2012)

CS523 Operating Systems

20

Selecting a process to run

cpu_switch (), called by mi_switch ()
block interrupts and check whichqs for nonempty run queue. If non, unblock interrupts and loop. Remove first process in queue. If queue is not empty then reset bit in whichqs clear curproc and want_resched set new process (context switch) and unblock interrupts

Fred Kuhns (12/2/2012)

CS523 Operating Systems

21

Limitations of the BSD Model

Limited scalability No resource guarantees non-deterministic, unbounded delays (priority inversion) limited application control over priorities

Fred Kuhns (12/2/2012)

CS523 Operating Systems

22

Scheduler Implementations
SVR4 Solaris Mach Digital UNIX Other RT

Fred Kuhns (12/2/2012)

CS523 Operating Systems

23

SVR4 Scheduler
Redesigned from traditional approach Separate policy from implementation
Define scheduling classes

define framework with well defined interfaces Attempt to bound dispatch latencies Enhance application control

Fred Kuhns (12/2/2012)

CS523 Operating Systems

24

SVR4 Scheduling Classes

Scheduler represents an abstract base class
defines interface performs class independent processing

Derived classes are specific implementations of a scheduling policy

priority computation, range of priorities, whether quantums are used or if priorities can vary dynamically.

Fred Kuhns (12/2/2012)

CS523 Operating Systems

25

SVR4 - Issues Addressed

Dispatch Latencies - time between a process becoming runnable and when it executes.
Does not include interrupt processing. Includes

nonpreemptive kernel processing and context switch time.

Kernel preemption points - PREMPT checks kprunrun at well defined points in the kernel runrun also used as in traditional implementations

Response time - total time from event to application response.

Fred Kuhns (12/2/2012)

CS523 Operating Systems

26

SVR4 - Class-Independent
Responsible for
context switching, run queue management and preemption.

Highest (global) priority always runs priority range: 0 - 160, each with own queue
Default allocations:
real-time 100-159, system 60-99, timesharing 0-59

Fred Kuhns (12/2/2012)

CS523 Operating Systems

27

SVR4 - Scheduler Interface

CL_TICK CL_FORK, CL_FORKRET CL_ENTERCLASS, CL_EXITCLASS CL_SLEEP, CL_WAKEUP CL_PREEMPT CL_YIELD CL_SETRUN

...
Fred Kuhns (12/2/2012) CS523 Operating Systems
28

SVR4 Class Implementations

Time-Sharing: event driven scheduling.
priority changed dynamically, round-robin within priority, time-slice and priority static parameter table.

System - fixed priority (FIFO). System tasks. Real-time: fixed priority and time quantum.

Fred Kuhns (12/2/2012)

CS523 Operating Systems

29

Solaris Overview
Multithreaded, Symmetric Multi-Processing Preemptive kernel - protected data structures MP Support - per cpu dispatch queues, one global kernel preempt queue. System Threads Priority Inheritance Turnstiles rather than wait queues
Interrupts handled using threads

Fred Kuhns (12/2/2012)

CS523 Operating Systems

30

Hidden Scheduling
Hidden Scheduling - kernel performs work asynchronously on behalf of threads, without considering priority of requester Solaris addresses by using system threads
Examples: STREAMS process and callout queue run at kernel priority which is lower than the RT class.

Callout processing is also a problem. Solaris uses two different callout queues: real-time and nonreal-time (callout thread).

Fred Kuhns (12/2/2012)

CS523 Operating Systems

31

Priority Inversion
Low priority thread holds a resource required by a higher priority thread. Partial solution - Priority Inheritance.
High priority thread lends its priority to the lower priority thread. Must be transitive kernel keeps a synchronization chain

Fred Kuhns (12/2/2012)

CS523 Operating Systems

32

Priority Inheritance in Solaris

Each thread has a global and inherited priority. Object must keep a reference to the owner If requester priority > owner, then owner priority raised to that of the requesters This works with Mutexes, owner always known Not used for semaphores or condition variables For reader/write,
owner of record inherits priority, only partial solution

Fred Kuhns (12/2/2012)

CS523 Operating Systems

33

MACH
Inherited base scheduling priority which is combined with a CPU usage factor CPU usage factor decayed 5/8 each second inactive threads set own priority after waking up. Clock handler charges current thread. Every 2 seconds, system thread scans run queue and recomputes priorities (addresses starvation) Fixed quantums, preemptive scheduling handoff scheduling - used by IPC

Fred Kuhns (12/2/2012)

CS523 Operating Systems

34

MACH MP Support
No cross processor interrupts processor sets thread runs on any of the processors from the assigned set. Processor allocation can be handled by a userlevel server Gang scheduling
dedicate one processor per thread. Minimize barrier synchronization delay

Fred Kuhns (12/2/2012)

CS523 Operating Systems

35

Digital UNIX
Time-sharing and real-time classes:
SCHED_OTHER, SCHED_FIFO, SCHED_RR,

Highest priority process is always run Priority range: 0-60 (0-29 TS, 20-31 SYS, 3263 RT) nonpreemptive kernel no control for priority inversion

Fred Kuhns (12/2/2012)

CS523 Operating Systems

36

Digital UNIX - MP
Per processor dispatch queues Scheduler will move threads between queues to balance load soft affinity

Fred Kuhns (12/2/2012)

CS523 Operating Systems

37