Recent Events
While processor speed has increased dramatically in recent years and the amount of processor storage has also grown substantially, the real bottleneck in system performance continues to be I/O. As an example, even a relatively slow 486 processor is easily 10 times as powerful as a 286 processor of a few years ago. And while great improvements in disk capacity have been made, access times to this vast amount of data have remained relatively constant.
In today's complex client/server computing environments the I/O bottleneck becomes magnified because an I/O request may have to be passed from a client to a remote server to be processed. A client application making too many requests to a server can cause the whole client/server network to degrade.
The answer, of course, is to minimize I/O. Many methods are available to reduce the performance mismatch between CPU and I/O. These include cache, buffering, short DASD strings, etc. IBM has suggested that the best method would be to place as much data in processor storage as possible and then retrieve the rest as quickly as possible.
Disk caching provides the easiest and most cost effective solution to this problem. Caching will reduce the amount of time spent waiting for an I/O to complete, resulting in improved on-line response time and reduced batch execution times.
Cache
In an un-cached system, an I/O request must move through the operating system to initiate the I/O request. The controller processes the request from the operating system and directs it to the correct device. The device then positions itself and begins the transfer of data to the controller. Eventually, the data is stored in processor storage and an interrupt notifies the system that the transfer has completed.
By using a hardware cache, the controller saves the Most Recently Used (MRU) data in local storage. This eliminates the need to communicate with the device completely, provided the requested data is in the cache. Removing the device from the loop usually results in savings of 60-80% in the time it takes to service an I/O request. This savings makes a un-cached I/O request 3-5 times slower than an cached request.
However, a software cache provides even greater performance improvements because it removes the I/O subsystem completely from the loop. When an I/O request is made, the system first checks the local cache and uses the data stored there, if available. Removing external hardware completely from the loop results in savings approaching 100% in the time it takes to service an I/O request. Even allowing for differing processor speeds, a request serviced by a software cache would be measured in micro-seconds, while an un-cached I/O request would be measured in milli-seconds.
Cache Basics
Most caches use simple Least Recently Used (LRU) methods for storing data. In an LRU cache the oldest data is discarded when new data is loaded into the cache. When an application makes a request for data, a search is made of the data stored in the cache to determine if the requested data is present. If the data is present, a cache HIT occurs and the data is copied from the cache to satisfy the request. If the data is not present in the cache, a cache MISS occurs and the request must be serviced by starting an I/O to retrieve the data from the disk.
Read caching is the easiest and safest form of caching. The most basic type of read cache is a read-behind cache. A read-behind cache places only the specific data requested into the cache. The disadvantage of this method is that only another request for exactly the same data will cause a cache hit to occur. However, greater performance can be achieved by using a read-ahead cache. In read-ahead caching a large block or track of data is loaded into the cache when satisfying a request. Reading multiple data records into the cache increases the chances another request can be satisfied with the data already in the cache.
Write caching is a more complex form of caching. There are two basic types of write caching: write-through, which simply updates the data in the cache as the write is being done and write-back, which defers the actual write until a later time. This results in quick (or fast) write times; but can result in lost data if a hardware error occurs before the cached data is written to the device.
Microsoft's SMARTDRV is an example of a good read-behind cache with optional write-back caching features. SMARTDRV's write-back caching feature has received a lot of press on the possibility of data loss after a system crash or power failure. IBM's 3990-3 DASD controller is a good example of a read-ahead/write-back hardware cache. The 3990-3 gets around the data loss problem by providing Non-Volatile Storage (NVS) to save data in case of system failures. Both of these products do a good job of caching data; but they make no decisions about whether or not the data being cached is actually "good" data to cache.
Choosing Data to Cache
Even the best cache facilities do not perform well when caching data with poor cache characteristics. Good cache characteristics start with files that have high read-to-write ratios or files where data is read more often than written and files that contain data with good locality of reference. Locality of reference refers to data accesses that occur in small or localized areas of a file. Sequential files can also have good cache characteristics provided the cache will de-stage a track after use and provide read-ahead of the next track to be used.
As examples, paging datasets have poor cache characteristics. They have low read-to-write ratios and bad locality of reference. File Allocation Tables (FATs) and Volume Table of Contents (VTOCs) have good cache characteristics, these characteristics include both high read-to-write ratios and good locality of reference.
Choosing the Size of the Cache
Many people subscribe to the "bigger is better" theory of cache sizing. However, studies have shown that past a certain point, increasing cache size yields little in the way of improved performance. Once all the data with good cache characteristics has been loaded into the cache, adding data without good cache characteristics does not improve performance to any significant degree.
In general, the optimum size of the cache is related to the amount of data on the disk with good cache characteristics. Knowing the exact "best" cache size would be nice; but is not really necessary. The key is to use the cache resources that are available as effectively as possible.
Optimal Cache Management
Determining the optimal cache strategy can be a difficult process due to the complex nature of the data stored on a typical disk subsystem. The problem is not dealing with ALWAYS or NEVER cache files; but what to do with the other 99% of the files, which fall into the MAY (or MAY not) cache category. Fortunately, a great deal of research has gone into dealing with this issue. By following some fairly simple rules, the software driving a cache management system can decide what to do about each file.
The minimum level for determining cache characteristics should be at the file level. However, for best results the determination should be made at the file's EXTENT level. Extent level determination allows distinction between data and indexes, or simply between multiple data extents with vastly different cache characteristics. The usual method of determining an extent's cache characteristics is to calculate the read hit ratio and compare it to the read hit ratio of the cache itself. An extent read hit ratio that is greater than that of the cache after subtracting an adjustment value (usually 5-15%) is considered an extent that may be cached.
In a typical cache management scheme, all files begin with a MAY cache status (except those files defined as ALWAYS or NEVER cache). This MAY cache status is adjusted periodically (normally every 100-1000 I/O requests) throughout the day. Once a file exhibits poor cache characteristics the MAY cache status is removed. After a period of time (normally 500-5000 I/O requests) the file is marked as MAY cache again and the evaluation process begins again. Therefore, as the files usage changes, so does a file's cache status.
Software caches make it simple to monitor file activity at the extent level allowing easy, automated management of the cache. They are also able to adjust the size of the cache based on user defined system paging thresholds. Utilities provided with the software cache make it easy to create user defined extents, alter the cache status of a file, stage data into the cache, or de-stage data from the cache.
In many respects, a software cache is a client/server application. The software running the cache itself acts as a server for the client applications requesting data.
Moral of the Story
While a hardware cache can provide good benefits, it does not eliminate all of the I/O overhead and is generally quite expensive. Software caches are probably the easiest to implement and provide the greatest I/O overhead reductions making them the most cost effective solution.
Overall, disk caching provides excellent performance benefits resulting in improved on-line response times and reduced batch execution times. The performance benefits can help you postpone hardware upgrades, a critical money saving strategy.
Regardless of whether you are running a small desktop PC, RSCS-based workstation, IBM mainframe or something in between, all systems should have some type of cache. When selecting a caching product be sure to choose a cache that will automatically cache the data best suited to being cached.
Copyright 1995 by Barnard Software, Inc.