Professional Documents
Culture Documents
Multilevel caches Critical word first: Dont wait for the full block to be loaded before sending the requested word and restarting the CPU
Read Miss Before write miss: This optimization serves reads before writes have been completed.
SW R2, 512(R0) ; M[512] R2 (cache index 0) LW R1,1024(R0) ; R1 M[1024] (cache index 0)
- If the write buffer hasnt completed writing to location 512 in memory, the read of location 512 will put the old, wrong value into the cache block, and then into R2.
Victim Caches
2
Victim Caches
One approach to lower miss penalty is to remember what was discarded in case it is needed again. This victim cache contains only blocks that are discarded from a cache because of a miss victimsand are checked on a miss to see if they have the desired data before going to the next lower-level memory. The AMD Athlon has a victim cache with eight entries.
3
Jouppi [1990] found that victim caches of one to ve entries are effective at reducing misses, especially for small, direct-mapped data caches. Depending on the program, a four-entry victim cache might remove one quarter of the misses in a 4-KB direct-mapped data cache.
Higher associativity,
Way prediction Pseudo-associativity,
- In way-prediction, extra bits are kept in the cache to predict the set of the next cache access.
Compiler optimizations
small and simple caches,
Cache Optimization
Complier-based cache optimization reduces the miss rate without any hardware change For Instructions
Reorder procedures in memory to reduce conflict Profiling to determine likely conflicts among groups of instructions
For Data
Merging Arrays: improve spatial locality by single array of compound elements vs. two arrays Loop Interchange: change nesting of loops to access data in order stored in memory Loop Fusion: Combine two independent loops that have same looping and some variables overlap Blocking: Improve temporal locality by accessing blocks of data repeatedly vs. going down whole columns or rows
5
Examples
Reduces misses by improving spatial locality through combined arrays that are accessed simultaneously
Sequential accesses instead of striding through memory every 100 words; improved spatial locality
Examples
Some programs have separate sections of code that access the same arrays (performing different computation on common data)
Fusing multiple loops into a single loop allows the data in cache to be used repeatedly before being swapped out
Loop fusion reduces missed through improved temporal locality (rather than spatial locality in array merging and loop interchange)
Accessing array a and c would have caused twice the number of misses without loop fusion
Blocking Example
Example
B called Blocking Factor Conflict misses can go down too Blocking is also useful for register allocation
10
VIRTUAL MEMORY
Rewrite your program so that it implements overlays Execute the first portion of code (fit it in the available
memory) When you need more memory... Find some memory that isnt needed right now Save it to disk Use the memory for the latter portion of code So on... The memory is to disk as registers are to memory Disk as an extension of memory Main memory can act as a cache for the secondary stage (magnetic disk)
11
A Memory Hierarchy
CPU
Registers
Store Load or I-Fetch
Cache
512KB typical
256MB typical
manages movement The operating system is responsible for managing the movement of memory between disk and main memory, and for keeping the address translation table accurate.
40GB typical
Disk
12
Virtual Memory
Idea: Keep only the portions of a program (code, data) that are currently needed in Main Memory
Currently unused data is saved on disk, ready to be brought in when needed Appears as a very large virtual memory (limited only by the disk size)
Advantages:
Programs that require large amounts of memory can be run (As
long as they dont need it all at once) Multiple programs can be in virtual memory at once, only active programs will be loaded into memory A program can be written (linked) to use whatever addresses it wants to! It doesnt matter where it is physically loaded! When a program is loaded, it doesnt need to be placed in continuous memory locations
Disadvantages:
The memory a program needs may all be on disk The operating system has to manage virtual memory
13
Virtual Memory
We will focus on using the disk as a storage area for chunks of main memory that are not being used.
The basic concepts are similar to providing a cache for main memory, although we now view part of the hard disk as being the memory.
Only few programs are active An active might not need all the memory that has been reserved by the program (store rest in the Hard disk)
14
Main Memory
15
Virtual Memory Space Error Disk Swap Space Disk Address: 58984 Not in main memory
16
The Process
The CPU deals with Virtual Addresses 1. Convert the virtual address to a physical address Need a special table (Virtual Addr-> Physical Addr.) The table may indicate that the desired address is on disk, but not in physical memory
Read the location from the disk into memory (this may require
moving something else out of memory to make room)
2. Do the memory access using the physical address Check the cache first (note: cache uses only physical addresses) Update the cache if needed
17
Address Translator
Physical Address
To Memory
18
Since the hardware access memory, we need to convert from a logical address to a physical address in hardware The Memory Management Unit (MMU) provides this functionality
2n-1
CPU
Virtual Address (Logical)
MMU
Physical Memory
19
Address Translation
In Virtual Memory, blocks of memory (called pages) are mapped from one set of address (called virtual addresses) to another set (called physical addresses)
20
Page Faults
If the valid bit for a virtual page is off, a page fault occurs. The operating system must be given control. Once the operating system gets control, it must find the page in the next level of the hierarchy (usually magnetic disk) and decide where to place the requested page in main memory.
21
Terminology
page: The unit of memory transferred between disk and the main memory. page fault: when a program accesses a virtual memory location that is not currently in the main memory. address translation: the process of finding the physical address that corresponds to a virtual address.
22
VM system must have an entry for all possible locations When there is a hit, the VM system provides the physical address in memory (not the actual data, in the cache we have data itself )
- Saves room one address rather than 8 KB data
Since miss penalty is very huge, VM systems typically have a miss (page fault) rate of 0.00001- 0.0001%
23
Organizations that reduce the page fault rate are attractive. The primary technique used here is to allow flexible placement of pages. (e.g. fully associative)
Page
Write-back
we
All programs use the same virtual addressing space Each program must have its own memory mapping Each program has its own page table to map virtual addresses to physical addresses
virtual Address
Page Table
Physical Address
The page table resides in memory, and is pointed to by the page table register The page table has an entry for every possible page (in principle, not in practice...), no tags are necessary. A valid bit indicates whether the page is in memory or on disk.
25
(index)
Page Offset
Note: may involve reading from disk Page tables are stored in main MEM 13 12 0
Example 4GB (32-bit) Virtual Address Space 16MB (24-bit) Physical Address Space 8 KB (13-bit) page size (block size)
The virtual page number (index) is derived from this by removing the page (block) offset
The Virtual Page Number is looked up in a page table When found, entry is either: The physical page number, if in memory The disk address, if not in memory (a page fault) If not found, the address is invalid
V->1 V->0
Virtual Address
4GB / 8KB = 512K entries
Index
19
219=512K
Page offset
13
Disk Address
Physical Address
27
Bits for page address Bits for virtual page number Number of virtual pages Entries in the page table Bits for physical page number Number of physical pages Bits per page table line Total page table size
28
Write issues
+ Easy to implement - Requires a write buffer - Requires a separate disk write for every write to memory - A write miss requires reading in the page first, then writing back the single word
29
Exact Least Recently Used (LRU) but it is expensive. So, use Approximate LRU: a use bit (or reference bit) is added to every page table line
If there is a hit, PPN is used to form the address and reference bit is turned on so the bit is set at every access
the OS periodically clears all use bits the page to replace is chosen among the ones with their use bit at zero
If the OS chooses to replace the page, the dirty bit indicates whether the page to be written out before its location in memory can be given to another (give a Figure)
30
Pick a page to kick out of memory (use LRU). Read data from sector 1239 Assume LRU is VPN 000101 for this example. into PPN 1010
31