Monday, January 26, 2009

Programming Techniques for Memory Management (Part 1) by Roger Donnay

This section written by Roger Donnay is the second part of CA-Clipper 5.x Memory Management by Roger Donnay & Jud Cole.

How you write your CA-Clipper code will greatly determine how the application uses memory. Once you understand and become grounded in the basics you will discover that memory management will become second nature to your programming style. You should never write code or use any third-party library without a basic understanding of the impact on memory usage, otherwise you can paint yourself into a memory-deficient corner that can not be easily undone.

Virtual Memory

CA-Clipper includes an automatic memory manager referred to as the Virtual Memory Manager or VMM. There is very little the CA-Clipper programmer needs to know about the VMM system other than it requires EMS memory or available disk space for the creation of swap files. After CA-Clipper uses up the available EMS for VMM, it will start creating swap files. In a network situation with no local hard drive or ram drive, this can slow down the application, therefore it is recommended that you set up your environment to allocate from 500k to 1 meg of EMS for average Clipper applications. You may monitor CA-Clipper's usage of EMS during your application by reporting the remaining EMS with the MEMORY(4) function. If the number reported falls below 100, then it is recommended that you increase available EMS by several hundred K.

If no EMS is available, then Virtual memory can be allocated from XMS memory by using a Third-party product named ClipXMS. See the section titled DOS Extenders for more information on this product.

Conventional Memory

The CA-Clipper free-pool (also sometimes referred to as the fixed-heap) is always allocated from conventional memory. I have found, from experience, that applications will perform poorly or run the risk of running out of memory completely if the amount of free-pool deteriorates to less than 50k. You can count on the fact that your application will have less free-pool available after running for awhile than it has at start-up time. The CA-Clipper MEMORY(0) function can be used to monitor the amount of memory remaining in the fixed heap. Opening databases has the most serious long-term affect on the use of conventional memory. See the section titled "The Affects of Databases on Memory Usage" for more details. To give the CA-Clipper application more conventional memory, you must start at DOS with ample conventional memory. After you load device drivers, network drivers, mouse drivers, TSR's, etc. your environment just may not have sufficient conventional memory remaining to run your Clipper application. The new generation of expanded memory managers are designed to not only create EMS/XMS memory but they also will load drivers and TSR's into the upper memory blocks (UMB) area of memory above 640k so they will not use up valuable conventional memory. I have used QEMM and 386MAX in the past to accomplish this task but have recently found that, on some systems, the MEMMAKER.EXE utility and the EMM386 driver supplied with DOS 6.x are adequate in creating a better memory environment.

SYMBOL Management

For a dynamic-overlay manager to work effectively, it is important that it load overlays at the smallest possible code-segment level, i.e., the procedure/function rather than the entire object. This requires managing the symbol table separately from the code and data, therefore, these linkers place the symbol table into the root memory area and eachfunction into a separate overlay segment.

Symbols cannot be overlayed or swapped to VMM, therefore it is important that you program in a manner consistent with producing the smallest number of symbols in your CA-Clipper-compiled objects. Here are some tips for reducing the symbol table size in your applications.

Use Constants instead of Memvars

All PRIVATE, PUBLIC and STATIC CA-Clipper memory variables are treated as "symbols". Refrain from using a memory variable if a constant is sufficient. For example, an unnecessary symbol can be eliminated by changing the code:

nEscapeKey := 27

DO WHILE INKEY() # nEscapeKey

* CA-Clipper code




* CA-Clipper code



#define K_ESC 27


* Ca-Clipper code


Use Arrays Instead of Memvars

Every different CA-Clipper PRIVATE, PUBLIC, or STATIC memvar name creates a "symbol", whereas an array name creates only ONE symbol. The following example shows how to save considerable memory in a CA-Clipper application by reducing the symbol count with an array.

This code produces 5 symbols:

PRIVATE cName := customer->name

PRIVATE cAddress := customer->address

PRIVATE cCity := customer->city

PRIVATE cState := customer->state

PRIVATE cZip = customer->zip

@ 1,1 SAY 'Name ' GET cName

@ 2,1 SAY 'Address' GET cAddress

@ 3,1 SAY 'City ' GET cCity

@ 4,1 SAY 'State ' GET cState

@ 5,1 SAY 'Zip ' GET cZip


This code produces 1 symbol:

PRIVATE aGets[5]

aGets[1] := customer->name

aGets[2] := customer->address

aGets[3] := customer->city

aGets[4] := customer->state

aGets[5] := customer->zip

@ 1,1 SAY 'Name ' GET aGets[1]

@ 2,1 SAY 'Address' GET aGets[2]

@ 3,1 SAY 'City ' GET aGets[3]

@ 4,1 SAY 'State ' GET aGets[4]

@ 5,1 SAY 'Zip ' GET aGets[5]


Some programmers choose memvars over arrays because it makes their source code more readable. Source code that refers to hundreds of array elements can look very cryptic and can be hard to maintain. To reconcile this problem, use the CA-Clipper pre-processor to create a "symbolic" reference to each array element like in the following example.

STATIC aGets[5]

#define cNAME aGets[1]

#define cADDRESS aGets[2]

#define cCITY aGets[3]

#define cSTATE aGets[4]

#define cZIP aGets[5]

cNAME := customer->name

cADDRESS := customer->address

cCITY := customer->city

cSTATE := customer->state

cZIP := customer->zip

@ 1,1 SAY 'Name ' GET cNAME

@ 2,1 SAY 'Address' GET cADDRESS

@ 3,1 SAY 'City ' GET cCITY

@ 4,1 SAY 'State ' GET cSTATE

@ 5,1 SAY 'Zip ' GET cZIP


Use the Same Name Memvars whenever possible

Again, every "different" PUBLIC, PRIVATE or STATIC CA-Clipper memvar in a module creates a symbol. If an object contains several procedures, use the same name for memvars even though they may not perform the same or similar functions. For example, procedure A and procedure B both need 5 memvars. If procedure A declares its memvars with 5 unique names and procedure B declares its memvars with 5 unique names, then 10 symbols are used in the linked application. To eliminate 5 symbols, make sure that procedure B assigns the same name to the memvars as procedure A. This is not possible of course, if the memvars need to be PUBLIC to both procedures and perform different functions, only if they are PRIVATE.

Use Complex Expressions instead of Memvars

The following three lines of code represents the method that most CA-Clipper programmers choose to accomplish most programming tasks. It makes sense to code this way for readability and debugging, but if you are writing a very large application, the complex expression technique can save some memory. The following three lines of code will read the disk file READ.ME into a memvar named cReadFile, save the changed code into a file named cEditFile, then write the changed code back to the disk file READ.ME.

cReadFile := MEMOREAD('READ.ME')

cEditFile := MEMOEDIT( cReadFile )


These three lines of code can be replaced by one complex expression which uses no symbols at all.


An additional advantage to coding this way is that less free-pool or VMM memory is used because the process of temporarily storing the text in cReadFile and cEditFile is completely eliminated. If you find that creating complex expressions such as this are unreadable and hard to maintain then use the CA-Clipper pre-processor to accomplish the same task as follows:

#define cReadFile MEMOREAD('READ.ME')

#define cEditFile MEMOEDIT(cReadFile)


The above code is nearly as readable as the original three lines of code but will not create any variables at compile time. Try compiling this code with your CA-Clipper compiler and use the /P switch to write the pre-processed code to a .PPO file, then look at the .PPO file to see what is actually compiled.


Sometimes it is just not possible or practical to write code without using symbols, so if you find yourself in this situation, CA-Clipper provides the feature of "LOCALIZING" symbols to the code segment which is currently being executed rather than placing the symbol in the main symbol table. LOCAL symbols are effectively "overlayed" because they are treated as part of the code segment rather than given a place in the main symbol table.

Not only does this save valuable memory but it also improves speed performance because the public symbol table does not need to be searched each time a LOCAL symbol is referenced in your code. Of course, if the symbol you are referencing is needed for the entire application or is used in a macro, then it must be declared as PRIVATE or PUBLIC. Symbols which are not declared at all are automatically assumed to be PRIVATE, so make sure you use the LOCAL declaration for all symbols in your code which you do not want to end up in the main symbol table. In the previous code example, the 5 PRIVATE memvars consume 110 bytes of memory, the single PRIVATE array consumes 22 bytes of memory, whereas 5 LOCAL declarations would consume 0 bytes of memory.

Use STATIC functions instead of PUBLIC functions

Every PUBLIC function and procedure name will occupy 22 bytes of the symbol table. Don't make functions and/or procedures PUBLIC unless they need to be called from anywhere within your application. STATIC procedures and functions are called only from within the source code module in which they are declared thereby eliminating the need to place the function name into the public symbol table.

Use PSUEDO functions or CONSTANTS instead of PUBLIC functions

A Psuedo-function is a function that does not really exist in the compiled code but exists only in your source code. Many programmers will use a large number of functions with different names even when each function may do something very similar like a conversion or a table lookup. I have seen a lot of code that looks like this:

SetColor( blueonwhite() + "," + redongreen() + "," + black() )

FUNCTION blueonwhite


FUNCTION redongreen




Programmers will use this technique to make their code easy to read and maintain without understanding how this can bloat the size of the application. Probably the best technique to replace the above code would be to #define constants for each as follows:



#define REDONGREEN "R/G"

#define BLACK "N/N"

Your existing source code may have "many" references to public functions and you don't want to risk changing all the function names to constants, or maybe the functions return something a little more complex than can be handled by a simple constant. In this case, you can #translate the functions into psuedo-functions as follows:


FUNCTION BLOBget ( nPointer, nStart, nCount )

RETURN dbInfo( BLOB_GET, { nStart, nCount } )

FUNCTION BLOBput ( nPointer, xBlob )

RETURN dbInfo( BLOB_PUT, { nPointer, xBlob } )

FUNCTION BLOBExport( nPointer, cTargetFile, lMode )

RETURN dbInfo( BLOB_EXPORT, { nPointer, cTargetFile, lMode } )


#xTranslate BLOBget( , , ) => ;

dbInfo( BLOBGET, { , , } )

#xTranslate BLOBput( , ) => ;

dbInfo( BLOBPUT, { , } )

#xTranslate BLOBExport( , , ) => ;

dbInfo( BLOBEXPORT, { , , } )

In the BEFORE example above, three PUBLIC functions were created simply to provide a better way to call the same dbInfo() function, whereas in the AFTER example, no PUBLIC functions were needed to accomplish the exact same task. Not only did we save 3 symbol table entries but we also eliminated 3 LOCAL variables that get pushed onto the EVAL stack. Of course it must be remembered that since psuedo-functions don't actually exist they cannot be used in macros or index keys.

Don't use STATICs or PRIVATEs to pass parameters

Many CA-Clipper programmers will assign a variable as PRIVATE or STATIC so it can be accessed and changed within multiple procedures in the same source code module. Variables should be STATIC only if their value needs to be maintained throughout the program, not for the convenience of eliminating the need to pass parameters. By using the pass-by-reference symbol "@" you can change the value of LOCAL variables anywhere within your calling program as shown by the following example.

LOCAL cName, cAddress

MyFunction( @cName, @cAddress )

Return( { cName, cAddress } )

STATIC FUNCTION MyFunction ( cName, cAddress )

cName := "CA-Clipper"

cAddress := "New York, NY"

Return( nil)

The "Myth" of PUBLICs verses STATICs

There is no memory advantage to using a STATIC variable in lieu of a PUBLIC variable. Both types of variables use up permanent space in conventional memory. A STATIC variable will use space in DGROUP, whereas a PUBLIC variable will use space in the Symbol Table. In fact, there are situations in which a good case can be made for using a PUBLIC array instead of a STATIC array. Take the example of a system-wide color system in which the colors are defined in an array. If the array is STATIC, then extracting the information from the array requires a call to a PUBLIC function which exists in the same source code module as the array definition. This would actually add more symbols to the symbol table and take more processor time than if the color array were defined as PUBLIC thereby allowing direct access to the array without the need for the public function. A STATIC array may be a wiser choice in cases where the application is a library which is incorporated into another CA-Clipper application, thereby eliminating any possibility of symbol conflicts with the application.

No comments:

Post a Comment

Welcome to Clipper... Clipper... Clipper

In 1997, then using Delphi 3, I had already created 32-bits Windows applications for HRIS, ERP and CRM. In 2007, using Ruby on Rails, an AJAX powered CRM site running on Apache & MySQL was created and I am now using Visual Studio .Net 2008 to create web-based projects and Delphi 7 for Win32 applications using SQL2005 & DBFCDX.

So, why then am I reviving the Original Clipper... Clipper... Clipper via a Blog as CA-Clipper is a programming language for the DOS world ? Believe it or not, there are still some clients using my mission-critical CA-Clipper applications for DOS installed in the late 80's and up to the mid 90's. This is testimony to CA-Clipper's robustness as a language :-)

With the widespread introduction of Windows 7 64-bits as the standard O/S for new Windows based PCs & Notebooks, CA-Clipper EXE simply will not work and it has become imperative for Clipper programmers to migrate immediately to Harbour to build 32/64 bits EXEs

Since 28th January 2009, this blog has been read by 134,389 (10/3/11 - 39,277) unique visitors (of which 45,151 (10/3/11 - 13,929) are returning visitors) from 103 countries and 1,574 cities & towns in Europe (37; 764 cities), North America (3; 373 cities) , Central America & Caribeans (6; 13 cities), South America(10; 226 cities), Africa & Middle-East (12; 44 cities) , Asia-Pacific (21; 175 cities). So, obviously Clipper is Alive & Well : -)

TIA & Enjoy ! (10th October 2012, 11:05; 13th November 2015)

Original Welcome Page for Clipper... Clipper... Clipper

This is the original Welcome Page for Clipper... Clipper... Clipper, which I am republishing for historical and sentimental reasons. The only changes that I have made was to fix all the broken links. BTW, the counter from is still working :-)

Welcome to Chee Chong Hwa's Malaysian WWW web site which is dedicated to Clipperheads throughout the world.

This site started out as a teeny-weeny section of Who the heck is Chee Chong Hwa ? and has graduated into a full blown web site of more than 140 pages (actually hundreds of A4 size pages) ! This is due to its growing popularity and tremendous encouragements from visiting Clipperheads from 100 countries worldwide, from North America, Central America, Caribbean, South America, Europe, Middle-East, Africa and Asia-Pacific. Thanx Clipperheads, you all made this happen !

What is Clipper ?

You may ask, what is this Clipper stuff ? Could Clipper be something to do with sailing as it is the name of a very fast sailing American ship in the 19th century ?

Well, Clipper or to be precise, CA-Clipper is the premier PC-Software development tool for DOS. It was first developed by Nantucket Corporation initially as a compiler for dBase3+ programs. Since then, CA-Clipper has evolved away from its x-base roots with the introduction of lexical scoping & pre-defined objects like TBrowse. As at today, the most stable version ofClipper is 5.2e while the latest version, 5.3a was introduced on 21 May 1996.

As at 11th November, 1996, an unofficial 5.3a fixes file was made available by Jo French. See the About CA-Clipper 5.3a section for more details. BTW, Jo French uploaded the revised 5.3a fixes file on 20th November, 1996.

Latest News

The latest news is that CA has finally released the long-awaited 5.3b patch on 21 May, 1997.

For 5.3b users, you must a take a look at Jo French's comments on unfixed bugs in 5.3b.

BTW, have you used Click ? If you're a serious Clipperprogrammer and need an excellent code formatter, Click is a natural choice. How to get it ? Simple, access Phil Barnett's site via my Cool Clipper Sites.

32-bits Clipper for Windows ?

Have you tried Xbase ++ ? Well, I have and compared to Delphi (my current Windows programming tool of choice), I'm still sticking to Delphi.

Anyway, you should visit the Alaska Home Page. Give it a chance and then draw your own conclusions !.

The Harbour Project

Is this the future of Xbase ? Take a look at at the Harbour Project

You are Visitor # ...

According to, you are visitor since 3 June 1996.

If you like or dislike what you see on this website, please drop me a line by clicking the email button at the bottom of this page or better still, by filling out the form in my guest book. If you are not sure what to write,click here to take a look at what other Clipperheads have to say.