Programming Techniques for Memory Management (Part 1) by Roger Donnay

This section written by Roger Donnay is the second part of CA-Clipper 5.x Memory Management by Roger Donnay & Jud Cole.

---

How you write your CA-Clipper code will greatly determine how the application uses memory. Once you understand and become grounded in the basics you will discover that memory management will become second nature to your programming style. You should never write code or use any third-party library without a basic understanding of the impact on memory usage, otherwise you can paint yourself into a memory-deficient corner that can not be easily undone.

Virtual Memory

CA-Clipper includes an automatic memory manager referred to as the Virtual Memory Manager or VMM. There is very little the CA-Clipper programmer needs to know about the VMM system other than it requires EMS memory or available disk space for the creation of swap files. After CA-Clipper uses up the available EMS for VMM, it will start creating swap files. In a network situation with no local hard drive or ram drive, this can slow down the application, therefore it is recommended that you set up your environment to allocate from 500k to 1 meg of EMS for average Clipper applications. You may monitor CA-Clipper's usage of EMS during your application by reporting the remaining EMS with the MEMORY(4) function. If the number reported falls below 100, then it is recommended that you increase available EMS by several hundred K.

If no EMS is available, then Virtual memory can be allocated from XMS memory by using a Third-party product named ClipXMS. See the section titled DOS Extenders for more information on this product.

Conventional Memory

The CA-Clipper free-pool (also sometimes referred to as the fixed-heap) is always allocated from conventional memory. I have found, from experience, that applications will perform poorly or run the risk of running out of memory completely if the amount of free-pool deteriorates to less than 50k. You can count on the fact that your application will have less free-pool available after running for awhile than it has at start-up time. The CA-Clipper MEMORY(0) function can be used to monitor the amount of memory remaining in the fixed heap. Opening databases has the most serious long-term affect on the use of conventional memory. See the section titled "The Affects of Databases on Memory Usage" for more details. To give the CA-Clipper application more conventional memory, you must start at DOS with ample conventional memory. After you load device drivers, network drivers, mouse drivers, TSR's, etc. your environment just may not have sufficient conventional memory remaining to run your Clipper application. The new generation of expanded memory managers are designed to not only create EMS/XMS memory but they also will load drivers and TSR's into the upper memory blocks (UMB) area of memory above 640k so they will not use up valuable conventional memory. I have used QEMM and 386MAX in the past to accomplish this task but have recently found that, on some systems, the MEMMAKER.EXE utility and the EMM386 driver supplied with DOS 6.x are adequate in creating a better memory environment.

SYMBOL Management

For a dynamic-overlay manager to work effectively, it is important that it load overlays at the smallest possible code-segment level, i.e., the procedure/function rather than the entire object. This requires managing the symbol table separately from the code and data, therefore, these linkers place the symbol table into the root memory area and eachfunction into a separate overlay segment.

Symbols cannot be overlayed or swapped to VMM, therefore it is important that you program in a manner consistent with producing the smallest number of symbols in your CA-Clipper-compiled objects. Here are some tips for reducing the symbol table size in your applications.

Use Constants instead of Memvars

All PRIVATE, PUBLIC and STATIC CA-Clipper memory variables are treated as "symbols". Refrain from using a memory variable if a constant is sufficient. For example, an unnecessary symbol can be eliminated by changing the code:

nEscapeKey := 27

DO WHILE INKEY() # nEscapeKey

* CA-Clipper code

ENDDO

to:

DO WHILE INKEY() # 27

* CA-Clipper code

ENDDO

or:

#define K_ESC 27

DO WHILE INKEY() # K_ESC

* Ca-Clipper code

ENDDO

Use Arrays Instead of Memvars

Every different CA-Clipper PRIVATE, PUBLIC, or STATIC memvar name creates a "symbol", whereas an array name creates only ONE symbol. The following example shows how to save considerable memory in a CA-Clipper application by reducing the symbol count with an array.

This code produces 5 symbols:

PRIVATE cName := customer->name

PRIVATE cAddress := customer->address

PRIVATE cCity := customer->city

PRIVATE cState := customer->state

PRIVATE cZip = customer->zip

@ 1,1 SAY 'Name ' GET cName

@ 2,1 SAY 'Address' GET cAddress

@ 3,1 SAY 'City ' GET cCity

@ 4,1 SAY 'State ' GET cState

@ 5,1 SAY 'Zip ' GET cZip

READ

This code produces 1 symbol:

PRIVATE aGets[5]

aGets[1] := customer->name

aGets[2] := customer->address

aGets[3] := customer->city

aGets[4] := customer->state

aGets[5] := customer->zip

@ 1,1 SAY 'Name ' GET aGets[1]

@ 2,1 SAY 'Address' GET aGets[2]

@ 3,1 SAY 'City ' GET aGets[3]

@ 4,1 SAY 'State ' GET aGets[4]

@ 5,1 SAY 'Zip ' GET aGets[5]

READ

Some programmers choose memvars over arrays because it makes their source code more readable. Source code that refers to hundreds of array elements can look very cryptic and can be hard to maintain. To reconcile this problem, use the CA-Clipper pre-processor to create a "symbolic" reference to each array element like in the following example.

STATIC aGets[5]

#define cNAME aGets[1]

#define cADDRESS aGets[2]

#define cCITY aGets[3]

#define cSTATE aGets[4]

#define cZIP aGets[5]

cNAME := customer->name

cADDRESS := customer->address

cCITY := customer->city

cSTATE := customer->state

cZIP := customer->zip

@ 1,1 SAY 'Name ' GET cNAME

@ 2,1 SAY 'Address' GET cADDRESS

@ 3,1 SAY 'City ' GET cCITY

@ 4,1 SAY 'State ' GET cSTATE

@ 5,1 SAY 'Zip ' GET cZIP

READ

Use the Same Name Memvars whenever possible

Again, every "different" PUBLIC, PRIVATE or STATIC CA-Clipper memvar in a module creates a symbol. If an object contains several procedures, use the same name for memvars even though they may not perform the same or similar functions. For example, procedure A and procedure B both need 5 memvars. If procedure A declares its memvars with 5 unique names and procedure B declares its memvars with 5 unique names, then 10 symbols are used in the linked application. To eliminate 5 symbols, make sure that procedure B assigns the same name to the memvars as procedure A. This is not possible of course, if the memvars need to be PUBLIC to both procedures and perform different functions, only if they are PRIVATE.

Use Complex Expressions instead of Memvars

The following three lines of code represents the method that most CA-Clipper programmers choose to accomplish most programming tasks. It makes sense to code this way for readability and debugging, but if you are writing a very large application, the complex expression technique can save some memory. The following three lines of code will read the disk file READ.ME into a memvar named cReadFile, save the changed code into a file named cEditFile, then write the changed code back to the disk file READ.ME.

cReadFile := MEMOREAD('READ.ME')

cEditFile := MEMOEDIT( cReadFile )

MEMOWRIT('READ.ME', cEditFile )

These three lines of code can be replaced by one complex expression which uses no symbols at all.

MEMOWRIT("READ.ME",MEMOEDIT(MEMOREAD("READ.ME")))

An additional advantage to coding this way is that less free-pool or VMM memory is used because the process of temporarily storing the text in cReadFile and cEditFile is completely eliminated. If you find that creating complex expressions such as this are unreadable and hard to maintain then use the CA-Clipper pre-processor to accomplish the same task as follows:

#define cReadFile MEMOREAD('READ.ME')

#define cEditFile MEMOEDIT(cReadFile)

MEMOWRIT('READ.ME',cEditFile)

The above code is nearly as readable as the original three lines of code but will not create any variables at compile time. Try compiling this code with your CA-Clipper compiler and use the /P switch to write the pre-processed code to a .PPO file, then look at the .PPO file to see what is actually compiled.

Use LOCALS Instead of PRIVATES

Sometimes it is just not possible or practical to write code without using symbols, so if you find yourself in this situation, CA-Clipper provides the feature of "LOCALIZING" symbols to the code segment which is currently being executed rather than placing the symbol in the main symbol table. LOCAL symbols are effectively "overlayed" because they are treated as part of the code segment rather than given a place in the main symbol table.

Not only does this save valuable memory but it also improves speed performance because the public symbol table does not need to be searched each time a LOCAL symbol is referenced in your code. Of course, if the symbol you are referencing is needed for the entire application or is used in a macro, then it must be declared as PRIVATE or PUBLIC. Symbols which are not declared at all are automatically assumed to be PRIVATE, so make sure you use the LOCAL declaration for all symbols in your code which you do not want to end up in the main symbol table. In the previous code example, the 5 PRIVATE memvars consume 110 bytes of memory, the single PRIVATE array consumes 22 bytes of memory, whereas 5 LOCAL declarations would consume 0 bytes of memory.

Use STATIC functions instead of PUBLIC functions

Every PUBLIC function and procedure name will occupy 22 bytes of the symbol table. Don't make functions and/or procedures PUBLIC unless they need to be called from anywhere within your application. STATIC procedures and functions are called only from within the source code module in which they are declared thereby eliminating the need to place the function name into the public symbol table.

Use PSUEDO functions or CONSTANTS instead of PUBLIC functions

A Psuedo-function is a function that does not really exist in the compiled code but exists only in your source code. Many programmers will use a large number of functions with different names even when each function may do something very similar like a conversion or a table lookup. I have seen a lot of code that looks like this:

SetColor( blueonwhite() + "," + redongreen() + "," + black() )

FUNCTION blueonwhite

RETURN 'B/W'

FUNCTION redongreen

RETURN 'R/G'

FUNCTION black

RETURN 'N/N'

Programmers will use this technique to make their code easy to read and maintain without understanding how this can bloat the size of the application. Probably the best technique to replace the above code would be to #define constants for each as follows:

SetColor( BLUEONWHITE+ "," + REDONGREEN + "," + BLACK )

#define BLUEONWHITE "B/W"

#define REDONGREEN "R/G"

#define BLACK "N/N"

Your existing source code may have "many" references to public functions and you don't want to risk changing all the function names to constants, or maybe the functions return something a little more complex than can be handled by a simple constant. In this case, you can #translate the functions into psuedo-functions as follows:

BEFORE:

FUNCTION BLOBget ( nPointer, nStart, nCount )

RETURN dbInfo( BLOB_GET, { nStart, nCount } )

FUNCTION BLOBput ( nPointer, xBlob )

RETURN dbInfo( BLOB_PUT, { nPointer, xBlob } )

FUNCTION BLOBExport( nPointer, cTargetFile, lMode )

RETURN dbInfo( BLOB_EXPORT, { nPointer, cTargetFile, lMode } )

AFTER:

#xTranslate BLOBget( , , ) => ;

dbInfo( BLOBGET, { , , } )

#xTranslate BLOBput( , ) => ;

dbInfo( BLOBPUT, { , } )

#xTranslate BLOBExport( , , ) => ;

dbInfo( BLOBEXPORT, { , , } )

In the BEFORE example above, three PUBLIC functions were created simply to provide a better way to call the same dbInfo() function, whereas in the AFTER example, no PUBLIC functions were needed to accomplish the exact same task. Not only did we save 3 symbol table entries but we also eliminated 3 LOCAL variables that get pushed onto the EVAL stack. Of course it must be remembered that since psuedo-functions don't actually exist they cannot be used in macros or index keys.

Don't use STATICs or PRIVATEs to pass parameters

Many CA-Clipper programmers will assign a variable as PRIVATE or STATIC so it can be accessed and changed within multiple procedures in the same source code module. Variables should be STATIC only if their value needs to be maintained throughout the program, not for the convenience of eliminating the need to pass parameters. By using the pass-by-reference symbol "@" you can change the value of LOCAL variables anywhere within your calling program as shown by the following example.

LOCAL cName, cAddress

MyFunction( @cName, @cAddress )

Return( { cName, cAddress } )

STATIC FUNCTION MyFunction ( cName, cAddress )

cName := "CA-Clipper"

cAddress := "New York, NY"

Return( nil)

The "Myth" of PUBLICs verses STATICs

There is no memory advantage to using a STATIC variable in lieu of a PUBLIC variable. Both types of variables use up permanent space in conventional memory. A STATIC variable will use space in DGROUP, whereas a PUBLIC variable will use space in the Symbol Table. In fact, there are situations in which a good case can be made for using a PUBLIC array instead of a STATIC array. Take the example of a system-wide color system in which the colors are defined in an array. If the array is STATIC, then extracting the information from the array requires a call to a PUBLIC function which exists in the same source code module as the array definition. This would actually add more symbols to the symbol table and take more processor time than if the color array were defined as PUBLIC thereby allowing direct access to the array without the need for the public function. A STATIC array may be a wiser choice in cases where the application is a library which is incorporated into another CA-Clipper application, thereby eliminating any possibility of symbol conflicts with the application.