Memory Hierarchy Levels

Block (aka line): unit of copying

May be multiple words

If accessed data is present in

upper level

Hit: access satisfied by upper level

Hit ratio: hits/accesses

If accessed data is absent

Miss: block copied from lower level

Time taken: miss penalty

Miss ratio: misses/accesses
= 1 hit ratio

Then accessed data supplied from

upper level

Chapter 5 Large and Fast: Exploiting Memory Hierarchy 1

Memory Technology

Static RAM (SRAM)

Dynamic RAM (DRAM)

50ns 70ns, $20 $75 per GB

Magnetic disk

0.5ns 2.5ns, $2000 $5000 per GB

5ms 20ms, $0.20 $2 per GB

Ideal memory

Access time of SRAM

Capacity and cost/GB of disk

Properties of Memory Devices

Semiconductor Memory

RAM

Actually, all semiconductor memory is random access but RAM

typically used to refer to read/write memory
Readable/Writeable
Volatile
Temporary storage
Static RAM (cache memory): SRAM
Dynamic RAM (main memory): DRAM
SRAM relies on transistors for storage
DRAM relies on capacitors for storage
Both designs rely on transistors for address switching

Memory Design conceptual cell organization

SRAM Cell- using crosscoupled inverters and

MOSFETs
Read Operation
- Word line activated to close
switches T1 and T2
- Sense/Write circuits (on
previous slide) at the end of the
bit lines monitor b and b and
set the output accordingly

- used where speed is of critical

concern
- can be accessed very quickly (a
nanosecond or so)

SRAM Cell cont

Write Operation

-The state of the cell is set by

placing the appropriate value on bit
line b and its complement b
- Signals on bit lines generated by
Sense/Write circuit
-Then, activate the word line,
forcing the cell into the
corresponding state
- Can be optimized to 6 transistors
(so-called 6T cell)

Dynamic RAM Design

Dynamic RAM (DRAM) stores data in the
form of a charge on a capacitor. Charge
leaks away with time (about ten
milliseconds) so must be refreshed.
Hence, called dynamic RAM. In return for
this refresh overhead, the density is much
higher.
- Only (1 transistor + 1 capacitor) vs 6T per
bit for SRAM
- Higher density (more capacity / unit area)

- Reduced cost/bit

Dynamic RAM Design

Address line active when bit read or written

Transistor switch closed (current flows)

Write

Voltage to bit line

Then signal address line

Transfers charge to capacitor

Read

Address line selected

transistor turns on

Charge from capacitor fed via bit line to sense amplifier

High for 1 low for 0

Compares with reference value to determine 0 or 1

Capacitor charge must be restored

Dynamic RAM Summary

Bits stored as charge in capacitors

Charges leak in standby and discharge when read
Need refreshing even when powered
Simpler construction
Smaller per bit
Less expensive
Need refresh circuits
Slower: time constant of capacitor to charge
Main memory usage

Static vs Dynamic RAM

Static RAM: bits stored as on/off switches: more space

using (6 transistors ) vs (1 transistor plus 1 capacitor) for
DRAM
SRAM has more complex construction: reliability
SRAM has larger area per bit = less dense
SRAM more expensive per bit
SRAM does not need refresh circuits
SRAM has faster access time
DRAM overall advantage is high density at low cost

DRAM Technology

Data stored as a charge in a capacitor

Single transistor used to access the charge

Must periodically be refreshed

Read contents and write back

Performed on a DRAM row Act: activate

Chapter 5 Large and Fast: Exploiting Memory Hierarchy 12

Advanced DRAM Organization

Bits in a DRAM are organized as a rectangular

array

Double data rate (DDR) DRAM

Transfer on rising and falling clock edges

Quad data rate (QDR) DRAM

DRAM accesses an entire row

Burst mode: supply successive words from a row with
reduced latency

Separate DDR inputs and outputs

DIMM (Dual inline memory modules)

Small boards for server 4-16 DRAM , DDR4

Chapter 5 Large and Fast: Exploiting Memory Hierarchy 13

DRAM Generations
300

Year

Capacity

$/GB

1980

64Kbit

$1500000

1983

256Kbit

$500000

1985

1Mbit

$200000

1989

4Mbit

$50000

1992

16Mbit

$15000

1996

64Mbit

$10000

1998

128Mbit

$4000

2000

256Mbit

$1000

2004

512Mbit

$250

2007

1Gbit

$50

250
200
Trac
Tcac

150
100
50
0
'80 '83 '85 '89 '92 '96 '98 '00 '04 '07

Chapter 5 Large and Fast: Exploiting Memory Hierarchy 14

DRAM Performance Factors

Row buffer

Synchronous DRAM

Allows several words to be read and refreshed in

parallel

Allows for consecutive accesses in bursts without

needing to send each address
Improves bandwidth

DRAM banking

Allows simultaneous access to multiple DRAMs

Improves bandwidth

Chapter 5 Large and Fast: Exploiting Memory Hierarchy 15

Increasing Memory Bandwidth

1 cycle to transmit
the address
15 cycle to access
the memory
1 cycle for data
transfer

4-word wide memory

Miss penalty = 1 + 15 + 1 = 17 bus cycles

Bandwidth = 16 bytes / 17 cycles = 0.94 B/cycle

4-bank interleaved memory

Miss penalty = 1 + 15 + 41 = 20 bus cycles

Bandwidth = 16 bytes / 20 cycles = 0.8 B/cycle