Caching is complex. There must be some technique for determining which information to place in the cache. Information must be selected and loaded into the cache(s) without significantly slowing down operations. Writes to cached data must be reflected both in the cache(s) and eventually (or sometimes immediately) in the main storage. There are two approaches to handling writing -- write through and write back. Write back is faster, but can be complex, so much so that reliability can be an issue. The algorithm used to find cached data must be fast and simple and should minimize flushing information at different, but similar, addresses. The latter is accomplished by a technique called "association". Association deals with two or more cached bytes mapping to the same cache address.
Caching can speed up reads. It is likely to slow write operations somewhat overall although individual programs may perceive their write operations as being faster.. Fortunately most applications do many more reads than writes. While memory caching is fast enough that write delays usually do not affect user perceptions, slower caches such as disk caches can have problems with cache flushing delays -- pauses while unwritten data is moved out of the cache and back to longer term storage. These, in turn can lead to a need for orderly OS shutdown if data is not to be lost. In some portable, industrial, or specialized applications, orderly shutdown can not be guaranteed.
Caches can exhibit some pathological problems including "thrashing" where two sets of data compete for cache space and the constant flushing of one by the other slows data access dramatically -- sometimes below the speed of an uncached system.
Overall one level of caching usually speeds up computer operation considerably. Multiple levels of caching may be less effective or even counterproductive as each level adds write delays but lower levels may not produce much read improvement if a higher level is caching most read information.
Return To Index Copyright 1994-2008 by Donald Kenney.