William Stallings

Computer Organization
and Architecture
8th Edition
Chapter 4: Cache Memory
Lecture Outline
Characteristics of Memory
Memory hierarchy
Locality of Reference
Cache structure and function

Characteristics (1)
Location
CPU (Registers)
Internal (Main)
External (Secondary)
Capacity
Word size: The natural unit of organization
Internal Memory: byte or word (8, 16, 32 bits)
External: byte
Unit of transfer
Internal: Usually governed by data bus width
External: Usually a block which is much larger than a word
Addressable unit
Smallest location which can be uniquely addressed
Word, some allow at byte level

Characteristics (2): Access Method

Sequential
Start at the beginning and read through in order
Access time depends on location of data or previous access
e.g. tape units
Direct
Individual blocks have unique address
Access is by jumping to vicinity plus sequential search
e.g. disk
Random
Individual addresses identify locations exactly
Access time is independent of location
e.g. RAM
Associative
Data is located by a comparison with contents rather than address
Access time is independent of location or previous access
e.g. cache

Characteristics (3)
Performance
Access time (latency): For random-access memory; its time to
perform read or write operation
Memory cycle time: Access time plus any additional time
required before second access.
Transfer rate: Rate at which data can be transferred into or out
of memory
Physical type
Semiconductor (RAM)
Magnetic (Disk and tape)
Optical (CD & DVD)
Magneto-Optical (Bubble, Hologram)
Physical characteristics
Volatile / nonvolatile
Erasable / nonerasable
Organization
Physical arrangement of bits to form words

Memory Hierarchy
Registers
In CPU
Internal or Main memory
May include one or more
levels of cache
RAM
External memory
Backing store

The Bottom Line-Three Key Characteristics

How much?
Capacity
How fast?
Access time
Time is money
How expensive?
Cost
Trade-off among three characteristics
Faster access time, greater cost per bit
Greater capacity, smaller cost per bit
Greater capacity, slower access time

The Dilemma
The way out of this dilemma is not to rely on a single memory
component or technology, but to employ a memory hierarchy.
Down the hierarchy, the following occur:
Decreasing cost per bit
Increasing capacity
Increasing access time
Decreasing frequency of access of the memory by the
processor

Locality of Reference
During the course of the execution of a program, memory
references by the processor (for both instruction and data)
tend to cluster
This principle states that memory references tend to cluster.
e.g. loops or subroutines (for instructions)
e.g. operations on tables and arrays (data)

Cache Memory Principles

Small amount of fast memory
Sits between normal main memory and CPU
May be located on CPU chip or module

Cache operation overview

Fastest memory speed, large memory size at price of less
expensive memory
The cache contains copy of portion of main memory
Main memory has 2n addressable words having unique n-bit
address.
For mapping purposes, main memory is considered to consist
of fixed-length blocks of K-words i.e. M = 2n / K blocks
The cache consists of C lines
Each line contains K words plus tag of few bits
Line size: Number of words in the line
C << M

Cache/Main Memory Structure

Cache Read Operation - Flowchart

Typical Cache Organization

Elements of Cache Design (1)

Few basic design elements to classify and differentiate cache
architectures.
Cache Size
medium size
large cache, slower
not possible to arrive at a single optimum cache size
Mapping Function
C << M
algorithm is needed for determining which memory block
currently occupies a cache line
mapping function dictates how cache is organised
three techniques can be used
direct, associative and set associative

Elements of Cache Design (2)

Direct
maps each block of main memory into one possible cache
lines
simple and inexpensive
disadvantage: fixed cache location for any given block
if a program to reference words repeatedly from two different
blocks that map into same line then blocks will be continually
swapped in the cache
hit ratio will be low (thrashing)
Associative
each memory block can be loaded into any line of the cache
disadvantage: complex circuitry required to examine the tags
of all ache lines in parallel
Set associative
exhibits the strengths of both approaches while reducing their
disadvantages

Elements of Cache Design (3)

Replacement Algorithms
one of the existing blocks must be replaced when cache
is filled
for direct mapping, no choice
for others, replacement algorithm is needed
number of algorithms has been tried, four mentioned here
1) Least Recently Used (LRU): replace that block that has
been in the cache longest with no reference to it
2) First In First Out (FIFO): replace that block that has
been in the cache longest
3) Least Frequently Used (LFU): replace that block that
has experienced the fewest references
4) A technique not based on usage is to pick a line at
random from among the candidate lines

Elements of Cache Design (4)

Write Policy
Must not overwrite a cache block unless main memory is up to
date
Multiple CPUs may have individual caches
I/O may address main memory directly
Write Through: Write operation made to main memory as well
as cache
Disadvantage: generates substantial memory traffic and may
create bottleneck
Write Back: updates are made only in the cache
Disadvantage: portions of memory are invalid
I/O modules can be allowed only through cache
Complex circuitry and a potential bottleneck
Line Size
Larger blocks reduce the number of blocks that fit into a cache
Small number of blocks, data overwritten shortly
Larger block size, each additional word farther from the
requested word, less likely to be needed in future