In many applications, the program needs more memory than is available in RAM. Virtual memory and extended memory systems help to overcome this problem by paging disk space into memory. Only the most often addressed pages (or pages addressed most recently) are kept in memory, which improves access speed, and creates an illusion that the application program works with an essentially unlimited size of memory.
Virtual or extended memory operates at the system level, and the application program cannot control its parameters, or open several independent memories at the same time. The PAGER organization described in this section allows you to create and control several pagers simultaneously. Each pager operates independently, with resources controlled by the application program. The disk/page memory is addressed by a long int index, and is treated as raw memory (a string of bytes). It can be used to store objects (structures), but the size of the objects, and any pointers must be managed manually.
There is no need to save pager data to disk. The information stored in a pager remains on disk even after the end of the run (each pager has its own disk file), and can be accessed later. The pager is automatically persistent.
For example, you can maintain two pagers running simultaneously, one having page size=10kB and up to 200 pages in memory, the other with page size=1MB, and up to 3 pages in memory. Each has its name (id), and you code with each pager as if it were a huge memory resident array, which you index by a long int.
The figure above shows a slightly simplified internal organization of one pager. The figure does not show all the details of the hash table. It also shows the holderObject with only one pager. In reality, there is a linked list of pagers on each holderObject.
Inside the pager, each page has its own pageHead. Fast access to the pages is provided via hash table (exactly as described in Chap.11.13). Pages move automatically between disk and memory. The organization shown in the figure is completely transparent. When closing a pager, all memory resident pages move to disk, and the reserved memory is freed. When opening a pager, it automatically opens the old file if it exists, or else opens a new file.
| ZZ_HYPER_PAGER(id,TYPE); | creates a pager on object type TYPE. id is the pager identifier. |
| void form(TYPE *h,char *fileName,int pgSize,int numPgs,int init); | forms a new pager (or re-opens an old one) on the holderObject h. fileName is the name of the disk file, pgSize is the size of one page in bytes, numPgs is the maximum number of pages in memory, and init specifies the initialization for new pages (0=none,1= `\0',2=` '). |
| void io(TYPE *h,long ind,char *buff,int n,int mode); | reads/write a block of data from the pager on object h. Here ind is the byte address within the file, buff is a memory buffer (target or source object of the read/write operation), n is the size of the object in bytes, and mode specifies whether reading from disk (=0) or writing to it (=1). |
| char * addr(TYPE *h,long ind); | makes sure that the page corresponding to disk address ind is loaded in memory, and returns pointer ptr to the ind location. |
| void close(hp); | closes pager with id=id on object h moves all pages to disk, and frees the memory. |
| long fill(hp); | returns the highest disk address currently used by the pager. |
| void flush(hp); | flushes all dirty pages to disk. |
Under DOS or other systems with integers only 2 bytes long, the total internal memory per pager is limited to 32k (=32767). In case that the total of numPages*pageSize exceeds this limit when forming the pager, numPages is automatically reduced to fit into the limit, and a warning is printed about it. The run continues uninterrupted; the only disadvantage is more intensive paging.
The reason for this limitation is that all the memory used by the pager is, internally, allocated as a single memory buffer. This design decision allows the pager to handle objects that are bigger than one page, or which overlap page boundaries.
test29c.c and test29d.c.
 Previous Section 11.13 Hash Tables |
Next Section 11.15 Access to Type
Tables  |