DGScan-The DGROUP Usage Scanner For Clipper by Ian Day & Dave Pearson

By Ian Day and Dave Pearson

Background On DGROUP

(Portions of this section are taken, with the kind permission of Dark Black Software, from the MrDebug Norton Guide)

DGROUP is a 64K chunk of memory that, for a Clipper program, can be broken down into five distinct sections, each serving a specific purpose. DGROUP can be easily described as a 64K block of memory that is used by Clipper as a common area of memory that Cliper and third party products can rely on to find various items of information. There are five main parts to DGROUP:

1. The MemVar table (Dynamic)

This is used to store Clipper STATIC variables and ITEMs (declared through the ITEM API) with their contents if the contents and variable/ITEM definition can be held within 14 bytes, otherwise the variable/ITEM definition is held along with a pointer to a Virtual Memory segment where the variable/ITEM contents are held.

2. DS Available (Dynamic)

This is the amount of free memory available within the DGROUP. This amount can be seen from //INFO as DS AVAIL or from the Memory/Info window. This will be reduced during the execution of the program as the Eval Stack and the Memvar table may both 'grow' into this area.

3. Eval Stack (Dynamic)

This is where Clipper LOCAL variables are held (if the variable definition and contents take up 14 bytes or less), otherwise a pointer is stored to a Virtual memory segment that contains the variable contents.

4. CPU Stack (Fixed)

The CPU stack is used to store the function return addresses each time a new function is called, as well as local 'C' variables.

The size of the CPU stack is set when you link your program. It depends upon the linker that you are using and linker specific commands that you have in the link script that might increase or reduce the stack

5. Fixed C and ASM Data (Fixed)

This area is used by other third party libraries and other languages that do not use their own data segments and use the default data segment (otherwise known to us Clipperites as DGROUP) to store fixed pieces of text and global variables used by other languages

For example, inside Clipper, the un-recoverable error messages are stored within this section of the default data segment (DGROUP).

As you can see from the above, the DGROUP area is pretty important to the smooth running of your Clipper application, and a lack of DGROUP can cause your software to fall over with a number of internal errors.

In an effort to make sure this does not happen you should try to reduce the impact your code has on DGROUP. Many 3rd party libraries use large ammounts of DGROUP (in my experience they can bump up the fixed C and ASM data usage by quite a bit. If you need to use a number of 3rd party libraries this can become quite a problem.

However, given that your chances of changing the usage of 3rd party libraries are pretty slim, the only chance you have of making a difference is by improving your own code to reduce it's DGROUP usage.

What Can Be Done?

If you write nothing but pure Clipper code, you still have the ability to reduce your DGROUP usage. The main area you can address is your use of STATIC variables. STATIC variables are, without a doubt, a good thing, but, they come with a price.

Unlike the other variable types, each STATIC variable requires a fixed (14 byte) entry in the the MemVar table. On the surface this may not seem like a big deal. A STATIC only needs 14 bytes in DGROUP, what does it matter? To see how this matters, lets take the GETSYS.PRG code you can find in the SOURCE\SYS directory of your copy of Clipper as an example.

Close to the top of the source you will find the following code:


//
// State variables for active READ
//
STATIC sbFormat
STATIC slUpdated := .F.
STATIC slKillRead
STATIC slBumpTop
STATIC slBumpBot
STATIC snLastExitState
STATIC snLastPos
STATIC soActiveGet
STATIC scReadProcName
STATIC snReadProcLine


As you can see, there are 10 STATIC variables, each using 14 bytes in the MemVar Table. With a simple little trick we can reduce that 140 bytes of usage down to 14 bytes. Watch:


// Hold all the statics in one static.

Static _aSysStuff := { NIL, .F., NIL, NIL, NIL, NIL,; NIL, NIL, NIL, NIL }

// Translated static variable names.

#xtranslate sbFormat => _aSysStuff\[ 01 \] #xtranslate slUpdated => _aSysStuff\[ 02 \] #xtranslate slKillRead => _aSysStuff\[ 03 \] #xtranslate slBumpTop => _aSysStuff\[ 04 \] #xtranslate slBumpBot => _aSysStuff\[ 05 \] #xtranslate snLastExitState => _aSysStuff\[ 06 \] #xtranslate snLastPos => _aSysStuff\[ 07 \] #xtranslate soActiveGet => _aSysStuff\[ 08 \] #xtranslate scReadProcName => _aSysStuff\[ 09 \] #xtranslate snReadProcLine => _aSysStuff\[ 10 \]

With that simple little trick you will have reduced the DGROUP usage from 140 bytes to just 14 bytes.

If you have lots of code of your own that has a large number of file wide STATICs in a single file you can make quite a difference.

If you write your own C functions, then it's a wise move to take a look at the compiler options at your disposal, as some of these can drastically reduce the impact on DGROUP.

For example, take the following piece of code:


#include

CLIPPER YesNo( void ) // Return yes or no string { if ( _parl(1) ) // .T. passed to function? { _retc( "Yes" ); // Yes, so return "Yes" } else { _retc( "No" ); // No, so return "No" } }

Simple enough, but already we've just used 7 (or 8) bytes of DGROUP without thinking! Why? you ask. Answer: It's the strings. By default, C compilers will put constants (which means strings, doubles, floats and some arrays) into DGROUP. And when I say '7 (or 8)', it's because some compilers will align data on a WORD boundary as default, and you can usually expect the DGROUP usage to be a tad more than you thought.

You can change this by looking for a compiler option to reverse this default. For example, with the Microsoft C compilers, there is the /Gt switch and for Borland C compilers there is the -Ff switch. Given a number of 1, it will automatically put any constants larger than 1 length into a FAR segment, and out of DGROUP! This means, that no matter how much data is used by your C functions for strings and things, it won't make a difference to DGROUP.

With assembler code, we use the same principles. Don't use _DATA, _BSS, CONST or DGROUP for data storage if you can possibly help it.

So, if you have a module like this:


_DATA SEGMENT WORD PUBLIC 'DATA'
cYes DB 'Yes', 0
cNo DB 'No', 0
_DATA ENDS

DGROUP GROUP _DATA

YESNO_TEXT SEGMENT WORD PUBLIC 'CODE'

ASSUME cs:YESNO_TEXT, ds:DGROUP

YESNO PROC FAR PUSH bp MOV bp, sp

MOV ax, 1 PUSH ax CALL __parl ADD sp, 2

OR ax, ax JZ ZeroSoItsFalse

MOV ax, OFFSET cYes JMP SHORT ReturnString

ZeroSoItsFalse: MOV ax, OFFSET cNo

ReturnString: PUSH ax PUSH ds CALL __retc ADD sp, 4

POP bp RETF YESNO ENDP

YESNO_TEXT ENDS END

You've again used 7 (or 8) bytes of DGROUP without thinking, so in this case you would have to alter the segments and anything that used the data in them so that DGROUP can once again be saved:


YESNO_DATA SEGMENT WORD PUBLIC 'FAR_DATA'
cYes DB 'Yes', 0
cNo DB 'No', 0
YESNO_DATA ENDS

YESNO_TEXT SEGMENT WORD PUBLIC 'CODE' ASSUME cs:YESNO_TEXT, ds:NOTHING

YESNO PROC FAR PUSH bp MOV bp, sp

MOV ax, 1 PUSH ax CALL __parl ADD sp, 2

OR ax, ax JZ ZeroSoItsFalse

MOV ax, OFFSET cYes JMP SHORT ReturnString

ZeroSoItsFalse: MOV ax, OFFSET cNo

ReturnString: PUSH ax PUSH YESNO_DATA CALL __retc ADD sp, 4

POP bp RETF YESNO ENDP

YESNO_TEXT ENDS END

As you can see, if your application is tight on DGROUP, with the source to hand and a little bit of work you can free up some of that usage. Lets just hope you have the source code for the worst offending code in your application.

Using DGSCAN To Find DGROUP Usage

Back when I first became aware of this "problem" I started to work on my own library code and managed to reduce some of the DGROUP usage. However, working through all that code was getting to be pretty boring. That's when I decided to write DGSCAN. What I wanted was a tool that could scan OBJ and LIB files and find anything that might be taking up precious DGROUP space.

If you want to get and try out DGSCAN then pop over to Hagbard's World and grab a copy. Got it? Good.

Ok, lets take a simple example of using DGSCAN. Look at the following code:

Static xVar1
Static xVar2
Static xVar3
Static xVar4

Function Main()

? "I'm eating up DGROUP"

Return( NIL )

Compile it into an OBJ and then run DGSCAN over it:

F:\DGTEST>dgscan foo.obj

DgScan v3.00 - DGROUP Usage Scanner By Dave Pearson

File-------- Module--------------------------- Segment----------- Bytes----- FOO.OBJ FOO STATICS$ 56 FOO.OBJ *** Total *** 56

As you can see, DGSCAN has detected 56 bytes of possible DGROUP usage. This is because we have 4 STATIC variables, each with a 14 byte impact. If you applied the "array trick" we spoke about earlier you would see the usage reduced to 14 bytes.

To generate a report for all your OBJ files just do:

F:\DGTEST>dgscan *.obj


It's as simple as that, and the same goes for LIB files.

I won't say much more about DGSCAN, everything you need to know is covered by the file DGSCAN.TXT found in the DGSCAN archive. However, if you do have problems with the utility please feel free to mail me and I'll try to help you out.