Cache Allocation and Management

Anil Olivera Roll No : - 8

Introduction
A collection of items of the same type stored in a hidden or inaccessible place. Cache is a small high-speed memory that stores data from some frequently used addresses (of main memory) Caches are used to bridge the increasing gap between processor speeds and memory access times. A cache is a small, fast memory that stores a subset of the main memory contents. It is located at or near the processor. Due to the principle of locality , most memory accesses can be serviced by the cache, although it is much smaller than main memory, thereby drastically improving the average latency of memory accesses Cache, which is pronounced "cash", stores recently used information so that it can be quickly accessed at a later time. Computers incorporate several different types of caching in order to run more efficiently, thereby improving performance. Cache hit: - Data found in cache. Results in data transfer at maximum speed. Cache miss: - Data not found in cache. Processor loads data from memory and copies into cache. This results in extra delay, called miss penalty

Cache line :- Cache is partitioned into lines(also called blocks). Each line has 4-64 bytes in it. During data transfer, a whole line is read or written.Each line has a tag that indicates the address in memory from which the line has been copied Each line has a tag that indicates the address in M from which the line has been copied.

Cache
Index Tag 0 2 1 0 Data ABC DEF

Main Memory

Index Tag 0 DEF 1 PQR 2 ABC 3 XYZ Cache hit is detected through an associative search of all the tags. Associative search provides a fast response to the query.
Common types of caches include browser cache, disk cache, memory cache, and processor cache. Browser cache - Most web browsers cache webpage data by default. For example, when you visit a webpage, the browser may cache the HTML, images, and any CSS or JavaScript files referenced by the page. When you browse through other pages on the site that use the same images, CSS, or JavaScript, your browser will not have to re-download the files. Instead, the browser can simply load them from the cache, which is stored on your local hard drive. Memory cache - When an application is running, it may cache certain data in the system memory, or RAM. For example, if you are working on a video project, the video editor may load specific video clips and audio tracks from the hard drive into RAM. Since RAM can be accessed much more quickly than a hard drive, this reduces lag when importing and editing files. Disk cache - Most HDDs and SSDs include a small amount of RAM that serves as a disk cache. A typical disk cache for a 1 terabyte hard drive is 32 megabytes, while a 2 TB hard drive may have a 64 MB cache. This small amount of RAM can make a big difference in the drive's performance. For example, when you open a folder

with a large number of files, the references to the files may be automatically saved in the disk cache. The next time you open the folder, the list of files may load instantly instead of taking several seconds to appears. A portion of RAM used to speed up access to data on a disk. The RAM can be part of the disk drive itself (sometimes called a hard disk cache or buffer) or it can be general-purpose RAM in the computer that is reserved for use by the disk drive (sometimes called a soft disk cache). Hard disk caches are more effective, but they are also much more expensive, and therefore smaller. Nearly all modern disk drives include a small amount of internal cache. A soft disk cache works by storing the most recently accessed data in the RAM cache. When a program needs to access new data, the operating system first checks to see if the data is in the cache before reading it from the disk. Because computers can access data from RAM much faster than from a disk, disk caching can significantly increase performance. Many cache systems also attempt to predict what data will be requested next so they can place that data in the cache ahead of time. Although caching improves performance, there is some risk involved. If the computer crashes (due to a power failure, for example), the system may not have time to copy the cache back to the disk. In this case, whatever changes you made to the data will be lost. Usually, however, the cache system updates the disk frequently so that even if you lose some data, it will not be much. Caches that work in this manner are called write-back caches. Another type of disk cache, called a write-thru cache, removes the risk of losing data because it only caches data for read operations; write operations are always sent directly to the disk. Processor cache - Processor caches are even smaller than disk caches. This is because a processor cache contains tiny blocks of data, such as frequently used instructions, that can be accessed quickly by the CPU. Modern processors often contain an L1 cache that is right next to the processor and an L2 cache that is slightly further away. The L1 cache is the smallest (around 64 KB), while the L2 cache may be around 2 MB in size. Some high-end processors even include an L3 cache, which is larger than the L2 cache. When a processor accesses data from a higher level caches, it may also move the data to the lower level cache for faster access next time.

Most caching is done in in the background, so you won't even notice it is happening. In fact, the only one of the above caches that you can control is the browser cache. You can open your browser preferences to view the cache settings and alter the size of your browser cache or empty the cache if needed. Cache Write Method Suppose the CPU wants to write a word to memory. If the memory unit uses the cache, it has several options. 2 commonly-used options: Write-through Write-back Write-Through On a write request by the CPU, check if the old data is in the cache. If the old data is in the cache (Write Hit), write the new data into the cache, and also into memory, replacing the old data in both cache and memory. If the old data is not in the cache (Write Miss), either: Load the line to cache, and write the new data to both cache and memory (this method is called write-allocate), or Just write the new data to memory, dont load the line to cache (this method is called no-write-allocate)

 A write-through cache forces all writes to update both the cache and the main memory. This is simple to implement and keeps the cache and memory consistent. The bad thing is that forcing every write to go to main memory, we use up bandwidth between the cache and the memory. Advantage: Keeps cache and memory consistent Disadvantage: Needs to stall for memory access on every memory write To reduce this problem, use a write buffer. When the CPU wants to write a word to memory, it puts the word into the write buffer, and then continues executing the instructions following the memory write. Simultaneously, the write buffer writes the words to memory.

Write-Back When the instruction requires a write to memory, If there is a cache hit, write only to the cache. Later, if there is a cache miss, and this line needs to be replaced, write the data back to memory. If there is a cache miss, either Load the line to cache, and write the new data to both cache and memory (this method is called write-allocate), or Just write the new data to memory, dont load the line to cache (this method is called no-write-allocate) In the Write-back method, we have a dirty bit associated with each cache entry. If the dirty bit is set to 1, we need to write back to memory when this

cache entry needs to be replaced. If the dirty bit is 0, we dont need to write back to memory, saving CPU stalls Disadvantage of Write-Back approach: Inconsistency - memory contains stale data Note: Write-allocate is usually used with write-back. No-write-allocate is usually used with write-through. In a write-back cache, the memory is not updated until the cache block needs to be replaced (e.g., when loading data into a full cache set). For example, we might write some data to the cache at first, leaving it inconsistent with the main memory as shown before. The cache block is marked dirty to indicate this inconsistency The advantage of write-back caches is that not all write operations need to access main memory, as with write-through caches. If a single address is frequently written to, then it doesnt pay to keep writing that data through to main memory. If several bytes within the same cache block are modified, they will only force one memory write operation at write-back time. The cache size also has a significant impact on performance. The larger a cache is, the less chance there will be of a conflict. Again this means the miss rate decreases, so the AMAT and number of memory stall cycles also decrease.