How to Convert a Clipper 5.2e application to Harbour

This article is written in in response to a comment left on Clipper... Clipper.. Clipper by Terry Carrol on 3rd July 2011 who asked

I’d like to know if there is an easy migration from Clipper to something else that I could make my Clipper program work on a 64-bit machine..."

As you are probably aware, Clipper apps being 16-bits cannot run on 64-bits Windows 7. For it to do so, it must be converted to 32-bits by using the Harbour or  xHarbour. I had previously show how to do so in Migrating from CA-Clipper 5.2e to xHarbour (Part I), a 3-part series. So this article focuses on using Harbour 2.1

Step 1 - Convert your Clipper link (*.lnk) file to  Harbour Project File (*.hbp)
Step 2 - Use Hbmk2 to compile the .hbp file

Step 1

1.1 Let us start by looking a Blinker link file of one of my actual Clipper project

BLINKER INCREMENTAL OFF
BLINKER EXECUTABLE COMPRESS
BLINKER EXECUTABLE CLIPPER F99
BLINKER PROCEDURE DEPTH 70
BLINKER OVERLAY OPSIZE 30
STACK 7168

FILE FC_MAIN
OUTPUT FASCALL

BEGINAREA
    FILE WRPT_GLL
    FILE WRPT_ARL
    FILE WRPT_ARD
    FILE WRPT_ARS
    FILE WRPT_APL
    FILE WRPT_APD
    FILE WRPT_APS
    FILE \KSLIB\CLIPLIB\UTILITY

    FILE \CLIPLIB\COMIX\CLIPMORE\52\CM52.OBJ
    ALLOCATE \CLIPLIB\COMIX\CLIPMORE\52\CM52
    FILE \CLIPLIB\COMIX\COMIX\52\CMX52.OBJ
    ALLOCATE \CLIPLIB\COMIX\COMIX\52\CMX52
ENDAREA

LIBRARY CLIPPER
LIBRARY EXTEND
LIBRARY DBFCDX
NOBELL


1.2 Convert the fascall.lnk to fascall.hbp

-ofascall4w7.exe
-w0
-es2
-rebuild

fc_main
WRPT_GLL
WRPT_ARL
WRPT_ARD
WRPT_ARS
WRPT_APL
WRPT_APD
WRPT_APS
UTILITY

Step 2


hbmk2 FASCALL.HBP

Result

You are bound to have a number of unresolved Externals, ie functions not supported by Harbour.
Once you have found alternative functions, FASCALL4W7.EXE, a true 32-bit executable will run on Windows 7 64-bits.

That's it :-)

Migrating from CA-Clipper 5.2e to xHarbour (Part III)

Having managed to recompile an actual DOS-based CA-Clipper EXE to a XHarbour W32 EXE, the next question is does it work

1) Launch FAS.EXE... so far so good

2) Input User Name & press enter, it bombs with the message

Error BASE/1003 variable does not exist GETLIST
Error at ...: MAIN_PASS(1051) in Module: FAS_MAIN.PRG
Called from : PASS_SCR(234) in Module: FAS_MAIN.PRG
Called from : MAIN(125) in Module: FAS_MAIN.PRG


Solution by Michael Hagl via comp.lang.xharbour

Add LOCAL getlist:={} to MAIN_PASS.PRG

Status : FAS.EXE can now run but now getting a weird ALERT() positioning as reported to
http://groups.google.com/group/comp.lang.xharbour/browse_thread/thread/44d58ac09e95f332#

Solution :
// Ron Pinkas 27/2/09
SetMode( 25, 80 )

Updated 2nd March 2009

Conclusion :

We started with FAS4DOS (CA-Clipper 5.2e)


What I got was a true Win32 executable but it looks exactly the same as FAS4DOS (minus the effect of Funckys' background function CLS() ) :-(


So how did I get the following screen-shot ? ;-)

Notice that the Window is centred and displays the EXE name instead of the C:\PATH\EXENAME and looks bigger...

So, how did I do it ?

Migrating from CA-Clipper 5.2e to xHarbour (Part II)

From Migrating from CA-Clipper 5.2e to xHarbour (Part I), we have clicked Build and OMG, there are lots of Unresolved External Symbols that needed to resolve.

As usual, I posted to comp.lang.xHarbour and the registered user's xHarbour newsgroup.
Within a short while, I had a most useful response from Michael Hagl as follows :-


-HB_FUN_MOUSE
Michael Hagl :

Function Mouse(x)
Return SetMouse(x)

-HB_FUN_RANDOM
CCH :

Replace with Phil Barnett's Random()

-HB_FUN_APRINT
Michael Hagl:

very dirty.

Static saSetAttrib

Function Aprint(nRow,nCol,str,attr,nLength,nStart)
Local nHighStart := At("^",str)
Local nHighEnd
Local cAttrib, nAttrib

If nHighStart > 0
cAttrib := Substr(str,nHighStart+1,1)
nAttrib := GetAttrib(cAttrib)
If Valtype(nAttrib) == "C" .and. nAttrib == "^"
nHighStart := 0
str := Left(str,nHighStart-1)+Substr(str,nHighStart+1)
Else
str := Left(str,nHighStart-1)+Substr(str,nHighStart+2)
Endif
Endif
nHighEnd := At("^N",Upper(str))
If nHighEnd > 0
str := Left(str,nHighEnd-1)+Substr(str,nHighEnd+2)
Endif

If !(nStart == Nil)
str := Substr(str,nStart)
Endif
If !(nLength == Nil)
str := Untrim(str,nLength)
Endif
DispOutAt(nRow,nCol,str,NtoColor(attr,.t.),.f.)
If nHighStart > 0
If Valtype(nAttrib) == "N"
ColorWin(nRow,nCol+nHighStart-1,nRow,nCol+nHighStart-1,nAttrib)
Elseif Valtype(nAttrib) == "C"
// ???
Endif
Endif
Return Nil

Static Function saSetAttribInit()
If saSetAttrib == Nil
saSetAttrib := {{"B","B"},;
{"R","R"},;
{"F","F"},;
{"S",S_TEXT},;
{"E",S_TEXT+12},;
{"N",S_TEXT},;
{"^","^"}}
Endif
Return Nil

Function SetAttrib(chr,nAttr)
Local nPos
saSetAttribInit()
chr := Upper(chr)
nPos := Ascan(saSetAttrib,{|x|x[1]==chr})
If nPos > 0
saSetAttrib[nPos,2] := nAttr
Else
Aadd(saSetAttrib,{chr,nAttr})
Endif
Return .t.

Function GetAttrib(chr)
Local nPos
saSetAttribInit()
chr := Upper(chr)
nPos := Ascan(saSetAttrib,{|x|x[1]==chr})
If nPos > 0
Return saSetAttrib[nPos,2]
Endif
Return -1

-HB_FUN_FORM_BOD
CCH:
Added SuperLib s_foml.prg

-HB_FUN_SWPRUNCMD
Ron Pinkas
/-------------------------------------------------------------------------

---------------------------//
FUNCTION SwpRunCmd( cCommand, nuMem, cRunPath, cTempPath )

LOCAL nAt, cArgs, cPresetDisk, cPresetDir, nRet

#ifndef DIR_SEPARATOR
#ifdef __PLATFORM__Windows
#define DIR_SEPARATOR '\'
#else
#define DIR_SEPARATOR '/'
#endif
#endif

IF Empty( cRunPath )
nAt := At( ' ', cCommand )

IF nAt > 0
cArgs := SubStr( cCommand, nAt )
cCommand := Left( cCommand, nAt - 1 )
ENDIF

nAt := RAt( DIR_SEPARATOR, cCommand )

IF nAt > 0
cRunPath := Left( cCommand, nAt )
cCommand := SubStr( cCommand, nAt + 1 ) + cArgs
ENDIF
ENDIF

IF cRunPath[2] == ':'
cPresetDisk := DiskName()

IF ! DiskChange( cRunPath[1] )
RETURN Throw( ErrorNew( "Blinker", 0, 1001, ProcName(), "Could not
switch to drive: " + cRunPath[1], HB_aParams() ) )
ENDIF

cRunPath := SubStr( cRunPath, 3 )
ENDIF

IF ! Empty( cRunPath )
cPresetDir := DIR_SEPARATOR + CurDir()

IF DirChange( cRunPath ) != 0
RETURN Throw( ErrorNew( "Blinker", 0, 1002, ProcName(), "Could not
switch to folder: " + cRunPath, HB_aParams() ) )
ENDIF
ENDIF

__Run( cCommand, @nRet )
SwpErrLev( nRet )

IF ! Empty( cPresetDisk )
IF ! DiskChange( cPresetDisk )
RETURN Throw( ErrorNew( "Blinker", 0, 1003, ProcName(), "Could not
switch back to drive: " + cPresetDisk, HB_aParams() ) )
ENDIF
ENDIF

IF ! Empty( cPresetDir )
IF DirChange( cPresetDisk ) != 0
RETURN Throw( ErrorNew( "Blinker", 0, 1004, ProcName(), "Could not
switch back to folder: " + cPresetDir, HB_aParams() ) )
ENDIF
ENDIF

RETURN nRet == 0

FUNCTION SwpErrLev( nLev )

STATIC s_nLastLev := 0

IF ValType( nLev ) == 'N'
s_nLastLev := nLev
ENDIF

RETURN s_nLastLev
//----------------------------------------------------------------------------------------------------//

Michael Hagl :

/******************************************************************************
V32ShellExecute(<>,[ cVerb ],[ cParam ],[ cDir ],[ cState ],[
lModal ] )

Arguments
<> is a character string containing the path and file name to be
executed with its associated program. If this parameter is
missing,
no action is taken.
[ cVerb ] is the flag what type of action the associated program has to
take.
Default is 'open'. Windows defines default verbs as 'open',
'print',
'explore'.

[ cParam ] a character string representing parameters to be passed to
associated program. Default is empty string.

[ cDir ] optional directory path where the associated program searches for
the required components.

[ cState ] can be on of the these: Hide, Normal, Minimized, Maximized, Show.
Default is 'Normal'. This determines the state of associated
program window.

[ lModal ] logical to determine whether the invoked program becomes the
child process of the current application or the program flow
return to application immediately after invoking the associated
program. Default is FALSE means program flow returns immediately.
***************************************************************************

****/
Function V32ShellExecute(cFile,cAction,cParam,cDir,cState,lModal)
Local nShowCmd
Default cAction To "open"
Default cParam To ""
Default cDir To ""
Default cState To "NORMAL"
If Upper(cState) == "MINIMIZED"
nShowCmd := 2
Elseif Upper(cState) == "MAXIMIZED"
nShowCmd := 3
Elseif Upper(cState) == "NOACTIVATE"
nShowCmd := 4
Elseif Upper(cState) == "SHOW"
nShowCmd := 5
Elseif Upper(cState) == "SHOWMINNOACTIVE"
nShowCmd := 7
Elseif Upper(cState) == "SHOWNA"
nShowCmd := 8
Elseif Upper(cState) == "SHOWDEFAULT"
nShowCmd := 10
Else
nShowCmd := 1
Endif
ShellExecute(0,cAction,cFile,cParam,cDir,nShowCmd)
Return .f.

-HB_FUN_ATTRTOA
Michael Hagl :

Function Attrtoa(nAttr)
Local r := NtoColor(nAttr,.t.)
Return r


-HB_FUN_FRE
CCH:
Solved by removing obsolete DOS functions

-HB_FUN_CPUTYPE
CCH:
Solved by removing obsolete DOS functions

-HB_FUN_NDPTYPE
CCH:
Solved by removing obsolete DOS functions

-HB_FUN_SOUND
CCH:
Solved by removing obsolete DOS functions

-HB_FUN_ROMDATE
CCH:
Solved by removing obsolete DOS functions

-HB_FUN_ENCRYPT
Solved by Michael Hagl :

try:
Function xEncrypt(cStr,cKey)
Return cStr ^^ cKey
//Function is from Klas Engwall ?
// but there is a Problem, when cKey is only @
Function Encrypt(cString,cKey)
local i,cNewString := ""
i := int((len(cString)/len(cKey))+1)
cKey := replicate(cKey,i)
for i := 1 to len(cString)
cNewString += chr(numxor(asc(substr(cString,i,1)),;
asc(substr(cKey,i,1))+128))
next
Return cNewString

-HB_FUN_SEED
CCH :
Made this redundant using Phil Barnet Random()

-HB_FUN_ALLKEY
CCH:
Replaced with INKEY(0)


Phew, having got rid of the unresolved external references, FAS.EXE is now a text-based Win32 Apps ! Now to test... Results in the next article.

-HB_FUN_DECRYPT

-HB_FUN_CATERR
CCH: Remove redundant PRG containing CATERR function call

-HB_FUN_MYHELP
CCH: Changed MyQuery() to Superlib Smalls()

-HB_FUN_MYQUERY
CCH: Changed MyQuery() to Superlib Smalls()

-HB_FUN_STRTOKEN

-HB_FUN_ISPRN
CCH/David Smith:
FUNCTION ISPRN(nPort)
LOCAL cPort:="LPT"+str(nPort,1)+":"
RETURN ISPRINTER(PrinterPortToName(cPort))


More to come...

Migrating from CA-Clipper 5.2e to xHarbour (Part I)

I am going to attempt a migration of a Clipper 5.2e DOS application for Financial Accounting to a Win32 version using xHarbour. BTW, I had already moved this product to Win32 via Delphi 5 in 1997 :-)

Step 1

Test compile using Clipper 5.2e via blinkfas.rmk
Test link using Blinker5 via link script blinkfas.lnk

Status : FAS.EXE created

Nb. This is to verify that the 5.2e project can indeed be built

Step 2

Launch xHarbour Project Builder



Step 3

a) Update Project Root Folder
C:\xHB\Projects\FAS

b) Update Target Name
FAS.EXE

Step 4

a) Select Tab 2

b) Update Main Source File

Step 5

a) Select Tab 3

b) Add Project *.prg

c) Replace *.prg from xHarbour-compatible SuperLib

Step 6

a) Click Build

Result :

Fas_main.prg, Line 5, can't open include file chcass.ch

Solution : Block temp with //

b) Click Build

Result :

Unresolved external symbols

More to come...

How to add code created by Visual xHarbour to a xHarbour Application [Updated 25th Feb 2009]

As you are probably aware, CA-Clipper codes can readily be recompiled in Xharbour into a W32 executable. In my scenario, I have a complex processing engine originally written in CA-Clipper 5.2e which can then be called within my Delphi applications

I had earlier used CA-Clipper 5.2e +Fivewin to create a W16 executable which was then called via winexec() from my Delphi Application. Being 16-bits meant that NTVDM was called automatically by Windows. What I wanted was a true W32 executable which will not require the usage of NTVDM.

There are 2 scenarios involved
1) Automatic processing without user input
2) Need for user input before automatic processing takes place

For Scenario 1, all I needed was to recompiled in Xharbour to transform my 5.2e CA-Clipper Codes into a 32-bit true windows application. It is that easy :-)

For Scenario 2, I needed user to input a parameter. Problem is that XHarbour does not have a simple Window Dialog function such as Delphi's InputQuery. So what's the solution ?

The Solution

1) Use Visual xHarbour (VXH) to create a windows form with
a) label control named MessageLabel
b) edit control named InputValueEdit
c) button control named OKBtn
d) button control named CancelBtn

2) Saved the Windows Form as

//MsgGetForm.prg

GLOBAL EXTERNAL Application
GLOBAL cMessage
GLOBAL cCaption
GLOBAL cInputValue

#include "vxh.ch"
#include "MsgGetForm.xfm"
//---------------------------------------- End of system code ----------------------------------------//

//----------------------------------------------------------------------------------------------------//
METHOD CancelBtn_OnClick( Sender ) CLASS MsgGetForm
::close()
RETURN Self
//----------------------------------------------------------------------------------------------------//
METHOD MsggetForm_Form_OnActivate( Sender ) CLASS MsgGetForm
::MessageLabel:caption:=cMessage
::caption:=cCaption
::InputValueEdit:caption:=cInputValue

RETURN Self
//----------------------------------------------------------------------------------------------------//
METHOD OKBtn_OnClick( Sender ) CLASS MsgGetForm
cInputValue:= ::InputValueEdit:Caption
::close()
RETURN Self


MsgGet_XFM.prg (automatically generated by VXH)

#include "vxh.ch"
//---------------------------------------- End of system code ----------------------------------------//

//------------------------------------------------------------------------------------------------------------------------------------

CLASS __MsgGet INHERIT Application
// Components declaration
// Controls declaration
METHOD Init() CONSTRUCTOR

// Event declaration
ENDCLASS

METHOD Init( oParent ) CLASS __MsgGet
::Super:Init( oParent )


// Populate Components
// Properties declaration

::Create()

// Populate Children
RETURN Self

//MsgGetForm.xfm (text definition based on the window form created with VHB)
//---------------------------------------------------------------------------------------------------------------------------------
//
CLASS MsgGetForm INHERIT WinForm
// Components declaration
// Controls declaration
DATA MessageLabel EXPORTED
DATA InputValueEdit EXPORTED
DATA OKBtn EXPORTED
DATA CancelBtn EXPORTED
METHOD Init() CONSTRUCTOR

// Event declaration
METHOD MsggetForm_Form_OnActivate()
METHOD OKBtn_OnClick()
METHOD CancelBtn_OnClick()

ENDCLASS

METHOD Init( oParent ) CLASS MsgGetForm
::Super:Init( oParent )

::EventHandler[ "OnActivate" ] := "MsggetForm_Form_OnActivate"

// Populate Components
// Properties declaration
::Left := 10
::Top := 10
::Width := 534
::Height := 139
::Name := "MsgGetForm"
::Caption := "MsgGetFormCaption"
::ClipChildren := .T.
::ClipSiblings := .T.

::Create()

// Populate Children
WITH OBJECT ( ::MessageLabel := LABEL( Self ) )
:Left := 41
:Top := 8
:Width := 160
:Height := 16
:Name := "MessageLabel"
:Caption := "MessageLabel"
:Border := .F.
:ClipChildren := .T.
:ClipSiblings := .T.
:TabStop := .F.
:Create()
END //MessageLabel

WITH OBJECT ( ::InputValueEdit := EDIT( Self ) )
:Left := 41
:Top := 32
:Width := 445
:Height := 22
:Name := "InputValueEdit"
:ClientEdge := .T.
:Border := .F.
:ClipSiblings := .T.
:Create()
END //InputValueEdit

WITH OBJECT ( ::OKBtn := BUTTON( Self ) )
:Left := 178
:Top := 61
:Width := 77
:Height := 30
:Name := "OKBtn"
:Caption := "&OK"
:Border := .F.
:ClipChildren := .T.
:ClipSiblings := .T.
:EventHandler[ "OnClick" ] := "OKBtn_OnClick"
:Create()
END //OKBtn

WITH OBJECT ( ::CancelBtn := BUTTON( Self ) )
:Left := 268
:Top := 62
:Width := 79
:Height := 31
:Name := "CancelBtn"
:Caption := "&Cancel"
:Border := .F.
:ClipChildren := .T.
:ClipSiblings := .T.
:EventHandler[ "OnClick" ] := "CancelBtn_OnClick"
:Create()
END //CancelBtn

::Show()

RETURN Self


2) Next, I added a wrapper function MsgGet() function in my Main.prg

// function in msggetform.prg
FUNCTION MsgGet(cFormCaption,cFormMessage,cFormInputValue)
// GLOBALs in MsgGetForm.prg
cMessage:=cFormMessage
cCaption:=cFormCaption
cInputValue:=cFormInputValue
__MsgGet( NIL ):Run( MsgGetForm( NIL ), HB_aParams() )


RETURN cInputValue

3) In the calling prg

cRetVal:=MsgGet('Create Data','Input No. of Working Days', @mNo_of_days)

That's it !

What is my wish list for the xHarbour people ? Provide a standard windows function as part of xHarbour :-)

Comments from David A. Smith , comp.lang.xHarbour
Some options are:
1) make a DLL of your "scenario" code, and call it as an unloadable library. (Not sure if there are limitations on collecting user input in a DLL.)
2) call your "scenario" code via OLE / xBScript
3) do it the way you have been doing it, just translate / recompile.

Original Welcome Page for Clipper... Clipper... Clipper

This is the original Welcome Page for Clipper... Clipper... Clipper (3rd June 1996) , which I am republishing for historical and sentimental reasons. The only changes that I have made was to fix all the broken links.

Welcome to Chee Chong Hwa's Malaysian WWW web site which is dedicated to Clipperheads throughout the world.
This site started out as a teeny-weeny section of Who the heck is Chee Chong Hwa ? and has graduated into a full blown web site of more than 140 pages (actually hundreds of A4 size pages) ! This is due to its growing popularity and tremendous encouragements from visiting Clipperheads from 100 countries worldwide, from North America, Central America, Caribbean, South America, Europe, Middle-East, Africa and Asia-Pacific. Thanx Clipperheads, you all made this happen !

What is Clipper ?

You may ask, what is this Clipper stuff ? Could Clipper be something to do with sailing as it is the name of a very fast sailing American ship in the 19th century ?
Well, Clipper or to be precise, CA-Clipper is the premier PC-Software development tool for DOS. It was first developed by Nantucket Corporation initially as a compiler for dBase3+ programs. Since then, CA-Clipper has evolved away from its x-base roots with the introduction of lexical scoping & pre-defined objects like TBrowse. As at today, the most stable version ofClipper is 5.2e while the latest version, 5.3a was introduced on 21 May 1996.
As at 11th November, 1996, an unofficial 5.3a fixes file was made available by Jo French. See the About CA-Clipper 5.3a section for more details. BTW, Jo French uploaded the revised 5.3a fixes file on 20th November, 1996.

Latest News

The latest news is that CA has finally released the long-awaited 5.3b patch on 21 May, 1997.
For 5.3b users, you must a take a look at Jo French's comments on unfixed bugs in 5.3b.
BTW, have you used Click ? If you're a serious Clipperprogrammer and need an excellent code formatter, Click is a natural choice. How to get it ? Simple, access Phil Barnett's site via my Cool Clipper Sites.

32-bits Clipper for Windows ?

Have you tried Xbase ++ ? Well, I have and compared to Delphi (my current Windows programming tool of choice), I'm still sticking to Delphi.
Anyway, you should visit the Alaska Home Page. Give it a chance and then draw your own conclusions !.

The Harbour Project

Is this the future of Xbase ? Take a look at at the Harbour Project

Edited on 13th Nov 2015 to remove broken links :-)

Linking Techniques for Memory Management (Part 1) by Roger Donnay

This section written by Roger Donnay is the third part of CA-Clipper 5.x Memory Management by Roger Donnay & Jud Cole. If you like to have the whole book in .WRI format, click here to visit Roger's Site

The Role of Linkers in developing large applications has taken a new turn in the past few years. In addition to providing segment-linking, they are now also expected to provide the best "memory image" possible. This means that the linker must be sophisticated in it's management of code segments during the running of an application. The physical size of a CA-Clipper application is no longer representative of it's memory image, because the linker "overlays" the code segments to use the smallest possible amount of memory. Some type of code (such as CA-Clipper P-CODE) is easy to overlay while other types (ASM) may be more difficult. Applications that are almost all CA-Clipper-compiled code will usually produce the best memory image regardless of the physical size of the .EXE.

The subjects of this book are the four most commonly-used linkers available on the market today for linking CA-Clipper 5.2 and 5.3 applications. They are:

RTLINK - A "Real Mode" CA-Clipper-5.2 specific linker developed by Pocketsoft, Inc. This is the linker that is bundled with CA-Clipper 5.2.

BLINKER - A "Real Mode" Third-Party linker published by Blink, Inc. This is the linker that is bundled with CA-Clipper 5.3. A "Dual-Mode" version of Blinker is also available from Blink, Inc.

EXOSPACE - A "Protected Mode" linker developed by CA and sold as an add-on to Clipper 5.2. This linker is bundled with Clipper 5.3.

CAUSEWAY - A third-party "Protected Mode" linker developed by Michael Devore for use with Clipper 5.2 and 5.3.

The CA-Clipper compiler has the responsiblity of converting source code to "machine-readable" code. This .OBJect code can not be "executed" by the machine however until it is "linked" to other .OBJect code segments in the CA-Clipper libraries. This is the responsibility of the linker. Linking or "fixing up" references made by your code to external variables and functions is necessary to insure that your compiled program will address or call other programs, including functions in the CA-Clipper libraries and DOS functions.

A linker's basic job is to combine objects which have been compiled separately. Objects contain symbols of three types:

a. PUBLIC symbols - those symbols which are callable from other objects

b. STATIC symbols - those symbols which are callable only from within the same object

c. UNDEFINED symbols - symbols which exist in other objects and are referenced or used by this object ((also referred to as EXTERNAL symbols).

DYNAMIC Overlaying of CA-Clipper Code

Very large CA-Clipper applications can consist of more code than can be loaded into the available amount of DOS memory and may not be able to execute due to insufficient memory. CA-Clipper 5.x has a "Dynamic Overlay Manager" actually built into the CA-Clipper libraries that helps prevent this problem by loading the code into memory only as it is needed by the application that is running. The linker helps prepare the application to use this capability of CA-Clipper 5.2 by splitting the CA-Clipper-compiled application code into fixed-size "pages" that share a pre-allocated area of memory. These pages of code are then written to a seperate .OVL file or to the end of the .EXE file as determined by your linker options. When a procedure or function is called, the "dynamic-overlay manager" first checks to see if it is in memory. If it is, then it jumps to the called function. If it is not in memory, it loads the function from disk into any "free" location available in the overlay pool. If no free space can be found in the overlay pool, the overlay manager discards pages that have not been called recently to free space to load the called procedure.

The larger the overlay pool, the less often the overlay manager needs to load from disk and the faster your program will run. The size of the dynamic overlay pool needed for your application is automatically determined by CA-Clipper.

The great thing about CA-Clipper 5.x's dynamic overlaying system is that you, the programmer, don't need to do anything to create overlays. It is all done automatically by the linker. Only CA-Clipper-compiled modules, not C/ASM modules, are overlayed "dynamically", although since much of the CA-Clipper libraries has been compiled in CA-Clipper, even large portions of the CA-Clipper libraries will be dynamically overlayed.

Application libraries and third-party libraries written in CA-Clipper will also be automatically overlayed by RTLINK and BLINKER. In many cases it is more beneficial to write functions in CA-Clipper (using LOCAL memvars) than in C or ASM because it will not increase the executable memory model. Even CA-Clipper code that has been linked into to Pre-link libraries will be dynamically overlayed.

INTERNAL/EXTERNAL Overlays

INTERNAL overlays are dynamic pages which are written to the end of the .EXEcutable program file. During the running of the application, the dynamic overlay manager reads the disk and loads dynamic pages from this file. This is the default overlay method of RTLINK and BLINKER and it is often advantageous to use internal overlays to reduce the number of file handles being opened by your application. In some cases however, you may find that using multiple overlay files will increase the speed performance of your application because extracting code from several small files which are all opened at the same time is often faster that extracting code from one large file.

EXTERNAL overlays are sets of dynamic pages which are written to a seperately designated file more commonly referred to as an overlay file. The positional command /DYNAMIC[:] or the freeformat command DYNAMIC INTO will tell RTLINK to write the dynamic pages to a file named . You may wish to create a seperate overlay file when your application .EXE is too large to fit on a distribution disk.

Examples:

Rtlink> DYNAMIC INTO myprog.ovl Blinker> SECTION INTO myprog.ovl

Rtlink> DYNAMIC INTO myprogs.ovl MYPROG1, MYPROG2 DYNAMIC INTO myfuncs.ovl MYFUNC1, MYFUNC2 Blinker> SECTION INTO myprogs.ovl MYPROG1, MYPROG2 SECTION INTO myfuncs.ovl MYFUNC1, MYFUNC2

DYNAMIC Overlaying of C / ASM Code

BLINKER also provides the ability to dynamically overlay code written in C or Assembly. This is the most common code found in many third-party libraries. RTLINK also provides for overlaying of C/ASM but it is more complex and requires understanding more about the code being overlayed. See STATIC OVERLAYS and RELOADABLE OVERLAYS for more information on overlaying C/ASM code with RTLINK.

In the previous section, we discussed that application code compiled with the CA-Clipper compiler is dynamically overlayed using the overlay manager built inside the CA-Clipper libraries. This CA-Clipper-specific internal overlay manager is incapable of overlaying C/ASM code, so it is the responsibility of the linker to provide this capability. Each linker has it's own methods for overlaying this kind of code. BLINKER uses an overlay pool that is allocated at the start of the application. The size of this pool is determined at link-time. The larger the overlay pool, the faster the application will run because overlays will not be reloaded into memory from disk so often.

The dynamic-overlay capabilities of BLINKER require that the objects being overlayed are "well-behaved". After much experimentation with our own libraries and many third-party libraries, it is now more clear what constitutes well-behaved modules:

a. Well-behaved code uses the CA-Clipper EXTEND API registers or the stack for parameter passing and does not use undocumented features of CA-Clipper.

b. All CA-Clipper-compiled .OBJects are dynamically-overlayable.

Modules which CANNOT be dynamically-overlaid are:

a. Routines which handle interrupts.

b. Modules in which the DATA area is changed during runtime rather than allocating memory in the root area.

Dynamic overlaying of third-party libraries can be an aggravating learning experience if you don't get support from the third party vendor in trying to accomplish this task. Most of the third-party vendors now break their libraries into separate overlayable and non-overlayable .LIB files to make the task much easier.

Once you have determined which .OBJ files and/or .LIB files are acceptable for overlaying, the task of telling the linker to do this is quite simple. A BLINKER link-script will look similar to an RTLINK script except there is always only one (1) overlay area because all code uses the same overlay "pool". Example of a BLINKER script for overlaying C/ASM:

# Allocate 50k to the overlay pool

BLINKER OVERLAY OPSIZE 50

BEGINAREA

# CA-Clipper-compiled application code

FILE myprog1.obj, myprog2.obj

# C/ASM files

FILE ccode.obj, asmcode.obj

# C/ASM libraries

LIB ccode.lib, asmcode.lib

ENDAREA

Using EXPANDED or UMB RAM for OVERLAYS

BLINKER provides a several very useful feature for increasing the amount of available memory for CA-Clipper applications, i.e., the use of Expanded (EMS) or Upper Memory Blocks (UMB).

EMS Memory

Many computers have EMS memory. EMS memory managers such as QEMM, 386MAX, and DOS's own EMM386 create a memory area above conventional memory referred to as the "page frame". This 64k block of memory is mapped to an area of ram that can execute 8086 code. Blinker takes advantage of this feature by allocating the 64k EMS page frame rather than conventional DOS memory for their runtime overlay pool. This frees an additional 64k of memory for use by CA-Clipper because the overlays are loaded into the page frame rather than conventional ram. If no EMS is present when the application starts, then the overlay pool will be established in normal (conventional) ram. To enable this feature, use the command:

BLINKER OVERLAY PAGEFRAME ON

UMB Memory

Expanded memory managers will also create an area between the 640k and 1 meg memory address called UMB (upper memory blocks). This is the area where you would normally load your memory-resident programs such as DOS, network-drivers, mouse-drivers, etc. If your enviroment gives you about 30k - 60k of contiguous memory available in the UMB area, Blinker will load the overlay pool into the UMB area rather than conventional area, thus freeing more conventional memory for your CA-Clipper application. To enable this feature, use the Blinker command:

BLINKER OVERLAY UMB ON

Note: In today's modern environments, this UMB area is usually consumed by so many TSR's that the possiblity of finding a contiguous block larger than 30k is virtually impossible, so it recommended that you do not rely on the availability of this memory.

STATIC Overlays

STATIC overlays are supported by RTLINK only. They provide a method of overlaying C and ASM code which cannot be overlayed by the dynamic-overlay manager. C and ASM code is not as easily relocatable as CA-Clipper compiled PCODE (pseudo-code) therefore it cannot be overlayed "dynamically". A function which is dynamically loaded and unloaded from a pool is usually found in different places in memory at different times during the running of the program. When a C or assembly routine is called by an application linked with RTLINK it must be loaded in memory at the same address every time. This is accomplished by designating overlay AREAS where many different modules will occupy the same memory space, but only one-at-a-time. Designing Static overlays requires a more thorough understanding of the structure of your program. Static overlay segments usually look like this in your link file:

# area 1

BEGINAREA

SECTION FILE A

SECTION FILE B

SECTION FILE C

ENDAREA

# area 2

BEGINAREA

SECTION FILE D

SECTION FILE E

SECTION FILE F

ENDAREA

You must be sure that you properly place your objects in the overlay segments to prevent computer "lock-up" or "crashing" at runtime. In the above example, any procedure or function in FILE A may call any procedure or function in FILE D, E or F because they are in different areas, however if a function in FILE A calls a procedure or function in FILE B, then your program will crash as soon as the program returns to FILE A because FILE A was removed from memory to load FILE B.

Overlay management with "static overlays" is the most time-consuming because it requires extensive analysis of the structure of the program to insure that the modules are placed properly in the overlay areas.

RELOADABLE Overlays

The following overlaying technique should be applied at your own risk because it is an "unsupported" and "undocumented" feature of the RTLINK linker supplied with CA-Clipper-5.2. I feel however, that it is important to cover this subject because without an understanding of how to use this feature it may be impossible to develop certain types of CA-Clipper applications without resorting to a third-party linker.

I started RELOADING modules from the CA-Clipper libraries years ago using RTLINK and the Summer 87 version of Clipper, so when I received my copy of CA-Clipper-5.2, the first thing I attempted to do was to overlay some of the larger C/ASM modules in the CA-Clipper libraries using the RELOAD command in my linker script file. The CA-Clipper development team had the foresight to insure that their C and ASM compiler assigned a UNIQUE name to each CA-Clipper C/ASM object in the CA-Clipper libraries, thus allowing, you, the CA-Clipper programmer to use both the RELOAD command and MODULE command in your .LNK script files.

So what are "Reloadable Overlays"? These are a form of STATIC overlay in which segments of code are placed into overlay sections that occupy the same memory space at runtime, however the "calling module" is automatically "reloaded" into memory when returning from the "called module".

Reloadable overlay segments usually look like this in your link file:

RELOAD FAR 200

# area 1

BEGINAREA

SECTION FILE A

SECTION FILE B

SECTION FILE C

ENDAREA

# area 2

BEGINAREA

SECTION FILE D

SECTION FILE E

SECTION FILE F

ENDAREA

Not all CA-Clipper applications are going to need to use these reloadable static overlays, but if your application relies heavily on third-party libraries written in C/ASM, then it is recommended that you learn this technique.

The main advantage of "reloadable overlays" over conventional "static overlays" is that you are never in danger of "lockup" in the event that a procedure or function in FILE A calls a function or procedure in FILE B because FILE A will be reloaded into memory on return from FILE B. Make sure when you use the RELOAD command that you always use it as follows:

RELOAD FAR

where the is the amount of memory in hex bytes to use for saving addresses. My experience is that most applications will run just fine with a stack size of 200. If your application does recursive nesting, however, you may need to increase the stack size. Recursive nesting is described as follows: Procedure A calls Procedure B which calls Procedure A which calls Procedure B, etc, etc, etc. Each time a procedure is called, its return address is "pushed" onto the stack, and each time you return to the calling procedure the address is "popped" from the stack. In the above example, if this recursive condition were allowed to continue for a large number of iterations, the stack would be overrun and the program may crash.

Although your application will not crash using RELOAD, it may slow down a bit if overlay segments are not structured properly, because each time an overlay is reloaded the application must go to disk. In the above example if a function in FILE A repetitively calls a function in FILE B then your application will be very "disk intensive" and will run slowly. If a function in FILE A repetitively calls a function in FILE D there will be no slowing at all because both modules will remain in memory.

Overlay management of C/ASM code with "reloadable overlays" is less time-consuming and more reliable than simple conventional "static overlays" and provides an additional advantage of allowing more modules to be overlayed, thereby saving additional memory usage.

Which RDD to use ? - DBFCDX versus DBFNTX

CA-Clipper 5.2e/5.3 comes with the following replaceable database drivers or RDDs

DBFNTX - Native Clipper

DBFCDX - Foxpro 2 compatible

DBFMDX - DBase IV compatible

DBFNDX - dBase III plus compatible

DBFPX - Paradox compatible

You should note that the DBFCDX driver that comes with 5.2e is the Successware SIX driver while 5.3/5.3a comes with the Lodestone COMIX driver.

The most popular RDDs as can be seen from posts on comp.lang.clipper are DBFCDX & DBFNTX. As for me, I use both drivers as appropriate. Therefore I shall limit this section to the DBFCDX & DBFNTX drivers.

The following information is extracted from NGs that came with CA-Clipper 5.2e

• RDD Architecture
• DBFNTX - CA-Clipper Native RDD
• DBFCDX - Foxpro 2.0 Compatible RDD

CA-Clipper's RDD Architecture

CA-Clipper supports a driver architecture that allows CA-Clipper applications to use Replaceable Database Drivers (RDDs). The RDD system makes CA-Clipper applications data-format independent. Such applications can, therefore, access the data formats of other database systems, including the dBASE IV (.mdx), FoxPro (.cdx), and Paradox (.db) formats on a variety of equipment. This driver architecture can even support database drivers that are not file-based, although all of the drivers supplied with CA-Clipper 5.2 are file-based.

The concept of replaceable drivers is not new to this version of CA-Clipper. In previous versions, the use of the default database driver (DBFNTX.LIB) was hidden by the fact that it was automatically linked into your application. In fact, this is still the case. The DBFNTX driver has been replaceable since it was first introduced in version 5.0. Before this version, the DBFNTX driver was the only RDD supplied as part of the system.

With the introduction of the new RDDs, CA-Clipper provides many new and enhanced commands and functions that access and manipulate databases. These language elements can enable your applications to access data regardless of the RDD under which it is ordered. There are also commands and functions that give you specific information about the RDDs in use.

DBFNTX - CA-Clipper Native RDD

DBFNTX is the default RDD for CA-Clipper. This new database driver replaces the DBFNTX database driver supplied with earlier versions of CA-Clipper and adds a number of new indexing features. With DBFNTX, you can:

Create conditional indexes by specifying a FOR condition

Create indexes using a record scope or WHILE condition, allowing you to INDEX based on the order of another index

Create both ascending and descending order indexes

Specify an expression that is evaluated periodically during indexing in order to display an index progress indicator

As an update of the default database driver, DBFNTX is linked into and used automatically by your application unless you compile using the /R option.

New Features

The replaceable driver lets you create and maintain (.ntx) files using features above and beyond those supplied with the previous DBFNTX driver. The new indexing features are supplied in the form of several syntactical additions to the INDEX and REINDEX commands. Specifically you can:

Specify full record scoping and conditional filtering using the standard ALL, FOR, WHILE, NEXT, REST, and RECORD clauses

Create an index while another controlling index is still active

Monitor indexing as each record (or a specified record number interval) is processed using the EVAL and EVERY clauses

Eliminate separate coding for descending order keys using the DESCENDING clause

Compatibility

Index files (.ntx) created with the original DBFNTX driver are compatible with DBFNTX and can be used in new applications without reindexing. Index files (.ntx) created with this version of DBFNTX will also work with previous CA-Clipper applications provided that you use no FOR, WHILE, , or DESCENDING clauses.

Important! Indexes produced with DBFNTX using FOR or DESCENDING are incompatible with earlier version (.ntx) files. If you attempt to access them with the original DBFNTX database driver or programs compiled with versions earlier than CA-Clipper 5.2, you will get an unrecoverable runtime error. In CA-Clipper, this generates an "index corrupted" error message, causing the application to terminate.

New Locking Scheme

The DBFNTX database driver implements a new locking scheme to resolve several problems identified in previous versions of CA-Clipper and to prevent potential problems that might arise when running CA-Clipper applications in a network environment. This section discusses these changes and their implications, including compatibility issues.

Lock Time-outs

Problem: Index locking in previous versions of CA-Clipper was handled automatically by the database driver, and had no time-out provision. This created the potential for problems in network environments if a workstation died while holding a lock. If this situation occurred all other workstations waiting for an index lock would appear to freeze while waiting to obtain their lock. This could also happen if a user placed a CA-Clipper application in the background on a multitasking system without sufficient processing time allocated to it. Eventually, most network operation systems would clear a connection that had no activity for a specified period of time. This would free the lock and everything would resume as normal, but frustrated users may have rebooted their machines possibly causing file corruption.

Solution: In CA-Clipper 5.2 the NTX driver will generate a recoverable runtime error if it fails to lock the index after a predetermined number of retries. The default error handler for this system simply returns (.T.) to retry the operation. This emulates the behavior of previous CA-Clipper versions.

Error Handling

Time out handling: The handling of this error is problematic because the lock is issued from various internal index routines. Therefore the only safe recoveries are to retry or quit. Choosing to default from the error or issuing a break will more than likely leave the index in a corrupted state. If either of the options is employed, the application should immediately recreate the index. The preferred way to handle a time out such as this is to alert the user of the situation so they don't think their machine has hung, and then have the network administrator determine what workstation is causing the problem. When the problem workstation is cleared, the users that have timed out can select retry and continue processing.

NTXERR.PRG: The file NTXERR.PRG contains the source code for the default error handler INIT procedure. This error handler can be modified to allow user-defined error handling for index lock time-outs. Care should be exercised when modifying the error handler as detailed above.

Compatibility: The lock time-out capability when used in conjunction with the default error handler is totally compatibility with previous versions of CA-Clipper. No changes are made to the NTX file structure and no action is required by the developer to activate the time-out functionality.

New Lock Offset

Problem: Index locking, which is transparent to the developer, uses a single-byte semaphore locking system. This semaphore was placed at a virtual offset (beyond the physical end of file) in the index file. In previous versions of CA-Clipper, this offset was located at one billion (1,000,000,000) which was adequate at the time. But many systems today are capable of producing indexes that are large enough to cause the actual data present at the lock offset to become physically locked. This leads to problems when trying to read or write to the data at that offset.

Solution: The solution is to move the offset where locking occurs to a location at a greater offset. We have chosen FFFFFFFF hex, which is the largest offset possible under the DOS operating system. The problem with this solution is that new applications using the index will be locking this new byte while old applications using the same index will lock the old position. Clearly this would cause both applications to fail because each could have a lock on the file at the same time.

To avoid this, the signature of the index (in the index header) is modified to prevent pre-CA-Clipper 5.2 applications from being able to open the index. CA-Clipper 5.2 applications can detect the correct offset to use by the flag in the header and will automatically use the correct one. In Figure 7-1 below, each bit represents a flag:

BIT 7 6 5 4 3 2 1 0 FLAG R R R O P I I C

R Reserved I Index type - both bits set (NTX) C Index created with a Condition, condition in header T Created as a Temporary index O New Offset for exclusive (semaphore) lock

Figure 7-1: Bit Field for the Signature Byte of a CA-Clipper 5.2 NTX File

Activation

If CA-Clipper 5.2 automatically modified the signature in the header when it created indexes, programs with automatic reindexing routines would be creating indexes that appeared corrupt to pre-CA-Clipper 5.2 applications. This has an obvious problem with backward compatibility. Therefore, in order to create indexes with the new signature, the developer must link in the module NTXLOCK2.OBJ with the full knowledge that this will create indexes that older applications will not be able to access.

Header Changes

The signature byte of a .NTX file is 6 for an unenhanced NTX index. The inclusion of the NTXLOCK2.OBJ will cause the signature to become 26 hex. (6 hex ORed with 20 hex). See Figure 7-1 for an illustration of all the possible values for the signature byte.

Error Handling

CA-Clipper 5.2 applications will automatically recognize the signature byte of the header, and depending on the signature value, will use the correct index lock location. Applications built with previous versions of CA-Clipper, however, do not have the capability to detect the optional new information in the signature byte. Therefore, when an order application tries to open a file that has been created with the NTXLOCK2.OBJ linked in it will produce a Corruption Detected error.

Compatibility

The new locking location, if used, is not backward compatible with applications compiled with previous versions of CA-Clipper.

Indexes created by applications built with a previous version of CA-Clipper can be used by CA-Clipper 5.2 using the new location and will not be modified unless the index is recreated in application.

Since older applications have no knowledge of the new index locking scheme nor of the significance of the header signature, these applications will assume the index is corrupt and will produce an Index Corrupted error.

Conditional Indexing

Conditional indexes are a feature of the DBFNTX driver. This section discusses this feature of the DBFNTX driver in some detail, giving you specific information about the implementation of conditional indexes. Compatibility issues are also discussed.

Conditional indexes are produced by using a FOR condition in the index creation process. These indexes are made fully maintainable by storing the FOR condition in the index header. This condition is subsequently retrieved and compiled each time the index in opened. During updates, items are added to the index only if they meet the criteria of the condition.

Since older applications do not have the ability to recognize and use the condition stored in the header, they must be prevented from opening the index since they corrupt the index. This is accomplished by modifying the signature of the index (in the index header) preventing pre-CA-Clipper 5.2 applications from being able to open the index. CA-Clipper 5.2 applications can detect the flag in the header and will automatically use the stored FOR condition correctly.

Temporary Indexes

Temporary indexes are produced by using any scoping clause other than the FOR condition in the index creation process. These indexes are not automatically maintainable because the condition is not stored for later use. These indexes can be made maintainable if the condition can be expressed as a FOR condition and is added using the FOR clause. But the main use of temporary indexes is for fast creation of indexes for read- only browses or reports that operate on a subset of the database.

Since older applications would not operate properly with indexes that do not contain all the keys in a given database, they must be prevented from using them. This is accomplished by modifying the index signature to prevent pre-CA-Clipper 5.2 applications from being able to open the index.

EOF()

If an index is created with a FOR condition and an attempt is made to update the index with a key that does not match the condition, the update is suppressed and the index is placed at EOF(). This is consistent with the current behavior for indexes created with the unique flag when an update is attempted with a non-unique key.

Also if a navigational action is attempted (SKIP) and the current record is not found in the index, the index will place the record pointer at EOF(). This is true for both conditional and temporary indexes.

Installing DBFNTX Driver Files

DBFNTX is supplied as the file DBFNTX.LIB.

The CA-Clipper installation program installs this driver as the default in the \CLIPPER5\LIB subdirectory on the drive that you specify, so you need not install the driver manually.

Important! Before installing CA-Clipper, you may want to rename the DBFNTX.LIB that currently resides in your \CLIPPER5\LIB directory to DBFNTX.001. The new version, when installed, will overwrite DBFNTX.LIB. If you do not rename or otherwise protect the old version of DBFNTX.LIB, you will lose it.

Linking the DBFNTX Database Driver

Since DBFNTX is the default database driver for CA-Clipper, there are no special instructions for linking. Unless you specify the /R option when you compile, the new driver will be linked into each program automatically if you specify a USE command or DBUSEAREA() function without an explicit request for another database driver. The driver is also linked if you specify an INDEX or REINDEX command with any of the new features.

Using the DBFNTX Database Driver

In applications written for the new DBFNTX driver, you can use the INDEX and REINDEX commands exactly as you have used them in the past. The index files (.ntx) you create and maintain in this way are completely compatible with those created using previous versions of the driver.

Changes to existing code are necessary only if you use the new indexing features. The (.ntx) files you create using the new features will have a slightly different header file and cannot be used by programs linked with a previous version of the driver.

Using (.ntx) and (.ndx) Files Concurrently

You can use (.ntx) and (.ndx) files concurrently in a CA-Clipper program like this:

// (.ntx) file using default DBFNTX driver

USE File1 INDEX File1 NEW

// (.ndx) files using DBFNDX driver

USE File2 VIA "DBFNDX" INDEX File2 NEW

Note, however, that you cannot use (.ntx) and (.ndx) files in the same work area. For example, the following does not work:

USE File1 VIA "DBFNDX" INDEX File1.ntx, File2.ndx

Compatibility with dBASE III PLUS

The default DBFNTX driver makes CA-Clipper programs behave differently than traditional dBASE programs. Some of these differences are discussed below.

Supported Data Types

The DBFNTX database driver supports the following dBASE III PLUS- compatible data types for key expressions:

Character

Numeric

Date

Logical

Supported Key Expressions

When you create (.ntx) files using the DBFNTX driver, you can use all CA-Clipper or user-defined functions compatible with dBASE III PLUS as well as other functions accepted by the extended CA-Clipper functionality.

Error Handling

The indexing behavior of DBFNTX and DBFNDX in a CA-Clipper application is identical unless otherwise noted. With the default DBFNTX driver, you can handle most errors using BEGIN SEQUENCE...END SEQUENCE as illustrated in the next section.

FIND vs SEEK

In CA-Clipper, you can use the FIND command only to locate keys in indexes where the index key expression is character data type. This differs from dBASE III PLUS where FIND supports character and numeric key values.

Note: In CA-Clipper programs, always use the SEEK command or the DBSEEK() function to search an index for a key value.

The DBFNTX driver lets you recover from data type errors raised during a FIND or SEEK. However, since Error:canDefault, Error:canRetry or Error:canSubstitute are set to false (.F.), you should use BEGIN SEQUENCE...END to handle such SEEK or FIND data type errors. Within the error block for the current operation, issue a BREAK() using the error object that the DBFNTX database driver generates, like this:

bOld := ERRORBLOCK({|oError| BREAK(oError)}) . . . BEGIN SEQUENCE SEEK xVar RECOVER USING oError // Recovery code END . . .

ERRORBLOCK(bOld)

Sharing Data on a Network

The DBFNTX driver provides file and record locking schemes that are different from dBASE III PLUS schemes. This means that if the same database and index files are open in CA-Clipper and in dBASE III PLUS, CA-Clipper program locks are not visible to dBASE III PLUS and vice versa.

Warning! Database integrity is not guaranteed and index corruption will occur if CA-Clipper and dBASE III PLUS programs attempt to write to a database or index file at the same time. Therefore, concurrent use of the same database (.dbf) and index (.ndx) files by dBASE III PLUS and CA-Clipper programs is strongly discouraged and not supported by Computer Associates.

DBFCDX - Foxpro 2.0 Compatible RDD

DBFCDX is the FoxPro 2 compatible RDD for CA-Clipper. As such, it connects to the low-level database management subsystem in the CA-Clipper architecture. When you use the DBFCDX RDD, you add a number of new features including:

FoxPro 2 file format compatibility

Compact indexes

Compound indexes

Conditional indexes

Memo files smaller than DBFNTX format

Overview of the DBFCDX RDD

The DBFCDX driver lets you create and maintain (.cdx) and (.idx) files with features different from those supplied with the original DBFNTX driver and is compatible with files created under FoxPro 2. The new features are supplied in the form of several syntactical additions to the INDEX and REINDEX commands. Specifically, you can:

Create indexes smaller than those created with the DBFNTX driver. The key data is stored in a compressed format that substantially reduces the size of the index file.

Create a compound index file that contains multiple indexes (TAGs), making it possible to open several indexes under one file handle. A single (.cdx) file may contain up to 99 index keys.

Create conditional indexes (FOR / WHILE / REST / NEXT).

Create files with FoxPro 2 file format compatibility.

Compact Indexes

Like FoxPro 2, The DBFCDX driver creates compact indexes. This means that the key data is stored in a compressed format, resulting in a substantial size reduction in the index file. Compact indexes store only the actual data for the index keys. Trailing blanks and duplicate bytes between keys are stored in one or two bytes. This allows considerable space savings in indexes with much empty space and similar keys. Since the amount of compression is dependent on many variables, including the number of unique keys in an index, the exact amount of compression is impossible to predetermine.

Compound Indexes

A compound index is an index file that contains multiple indexes (called tags). Compound indexes (.cdx)'s make several indexes available to your application while only using one file handle. Therefore, you can overcome the CA-Clipper index file limit of 15. A compound index can have as many as 99 tags, but the practical limit is around 50. Once you open a compound index, all the tags in the file are automatically updated as the records are changed.

Once you open a compound index, all the tags contained in the file are automatically updated as the records are changed. A tag in a compound index is essentially identical to an individual index (.idx) and supports all the same features. The first tag (in order of creation) in the compound index is, by default, the controlling index.

Conditional Indexes

The DBFCDX driver can create indexes with a built-in FOR clause. These are conditional indexes in which the condition can be any expression, including a user-defined function. As the database is updated, only records that match the index condition are added to the index, and records that satisfied the condition before, but don't any longer, are automatically removed.

Expanded control over conditional indexing is supported with the revised INDEX and REINDEX command options as in the new DBFNTX driver.

Installing DBFCDX Driver Files

The DBFCDX driver is supplied as the file, DBFCDX.LIB.

The CA-Clipper installation program installs this driver in the \CLIPPER5\LIB subdirectory on the drive that you specify, so you need not install the driver manually.

Linking the DBFCDX Database Driver

To link the DBFCDX database driver into an application program, you must specify DBFCDX.LIB to the linker in addition to your application object files (.OBJ).

1. To link with .RTLink using positional syntax:

C>RTLINK ,,,DBFCDX

2. To link with .RTLink using freeformat syntax:

C>RTLINK FI LIB DBFCDX

Note: These link commands all assume the LIB, OBJ, and PLL environment variables are set to the standard locations. They also assume that the CA-Clipper programs were compiled without the /R option.

Using the DBFCDX Database Driver

To use FoxPro 2 files in a CA-Clipper program:

1. Place REQUEST DBFCDX at the beginning of your application or at the top of the first program file (.prg) that opens a database file using the DBFCDX driver.

2. Specify the VIA "DBFCDX" clause if you open the database file with the USE command.

-OR-

3. Specify "DBFCDX" for the argument if you open the database file with the DBUSEAREA() function.

-OR-

4. Use RDDSETDEFAULT( "DBFCDX" ) to set the default driver to DBFCDX.

Except in the case of REQUEST, the RDD name must be a literal character string or a variable. In all cases it is important that the driver name be spelled correctly.

The following program fragments illustrate:

REQUEST DBFCDX . . . USE Customers INDEX Name, Address NEW VIA "DBFCDX"

-OR-

REQUEST DBFCDX RDDSETDEFAULT( "DBFCDX" ) . . . USE Customers INDEX Name, Address NEW

Using (.idx) and (.ntx) Files Concurrently

You can use both (.idx) and (.ntx) files concurrently in a CA-Clipper program like this:

// (.ntx) file using default DBFNTX driver

USE File1 INDEX File1 NEW

// (.idx) files using DBFCDX driver

USE File2 VIA "DBFCDX" INDEX File2 NEW

Note, however, that you cannot use (.idx) and (.ntx) files in the same work area. For example, the following does not work:

USE File1 VIA "DBFNTX" INDEX File1.ntx, File2.idx

Using (.cdx) and (.idx) Files Concurrently

You may use (.cdx) with (.idx) files concurrently (even in the same work area); however, in most cases it is easier to use a single (.cdx) index for each database file or separate (.idx) files. When using both types of index at the same time, attempting to select an Order based on its Order Number can be confusing and will become difficult to maintain.

File Maintenance under DBFCDX

When an existing tag in a compound index (.cdx) is rebuilt using INDEX ON...TAG... the space used by the original tag is not automatically reclaimed. Instead, the new tag is added to the end of the file, increasing file size.

You can use the REINDEX command to "pack" the index file. REINDEX rebuilds each tag, eliminating any unused space in the file.

If you rebuild your indexes on a regular basis, you should either delete your (.cdx) files before rebuilding the tags or use the REINDEX command to rebuild them instead.

DBFCDX and Memo Files

The DBFCDX driver uses FoxPro compatible memo (.fpt) files to store data for memo fields. These memo files have a default block size of 64 bytes rather than the 512 byte default for (.dbt) files.

DBFCDX memo files can store any type of data. While (.dbt) files use an end of file marker (ASCII 26) at the end of a memo entry, (.fpt) files store the length of the entry. This not only eliminates the problems normally encountered with storing binary data in a memo field but also speeds up memo field access since the data need not be scanned to determine the length.

Tips For Using DBFCDX

1. Make sure index extensions aren't hard-coded in your application. The default extension for DBFCDX indexes is (.idx), not (.ntx). You can still use (.ntx) as the extension as long as you specify the extension when you create your indexes. The best way to determine index extensions in an application is to call ORDBAGEXT().

For example, if you currently use the following code to determine the existence of an index file:

IF .NOT. FILE("index.ntx") INDEX ON field TO index ENDIF

Change the code to include the INDEXEXT() function, as follows:

IF .NOT. FILE("index"+ORDBAGEXT()) INDEX ON field TO index ENDIF

2. If your application uses memo fields, you should convert your (.dbt) files to (.fpt) files.

There are some good reasons for using (.fpt) files. Most important is the smaller block size (64 bytes). CA-Clipper's (.dbt) files use a fixed block size of 512 bytes which means that every time you store even 1 byte in a memo field CA-Clipper uses 512 bytes to store it. If the data in a memo field grows to 513 bytes, then two blocks are required.

When creating (.fpt) files, the block size is set at 64 bytes to optimize it for your needs. A simple conversion from (.dbt) files to (.fpt) files will generally shrink your memo files by approximately 30%.

3. Add DBFCDX.LIB as a library to your link command or link script.

Can a Clipper 5.2x Application run on Windows NT [Nov 1995] ?

Here's all of the replies I got from comp.lang.clipper to my post " Can a Clipper 5.2x Application run on Windows NT ? My actual post reads as :

Hi to all the clipper Gurus

An existing client of my intends to migrate from Novell LAN to Windows NT. They are presently using several of our Clipper 5.2e apps and asked whether they will run on Windows NT. I will appreciate hearing from anyone as regards their experiences with Windows NT & their clipper apps.

---

From: mike@mantis.co.uk (Mike Jones)
Date: 2 Nov 1995 14:55:51 -0000
Organization: Mantis Consultants Ltd, Cambridge, UK

Our clipper app runs fine with the database on a Windows NT 3.50 server. However we had concurrency problems when using Window NT 3.50 clients (newly appended records being overwritten) with both NT and Novell servers. This problem went away when we upgraded to NT 3.51 client. So our advice is 3.50+ is fine as a server, but use 3.51 as a client.

Mike.

---

From: ewoudenberg@envirolink.org
Date: 2nd November 1995

I vaguely remember running a test of a Clipper app we used on an NT server. Didn't have any problems. The response wasn't quite as fast as I would have liked but I don't remember having any difficulties with the system running. We only tested it with about 3 or four users at the same time so I wouldn't say it was a rigorous test...

Esko

---

From: Terry Carmen
Date: 4th November 1995

Clipper apps work just fine with NT.

NT is stable as a rock. In fact, it's more stable than DOS wor Windows every thought of being.

Each DOS app gets it's own virtual DOS machine to run in, and has about 580 K of free RAM, including loading the Netware and CD-ROM drivers.

Terry

---

From: chips@eskimo.com (:crp:)
Date: Sun, 5 Nov 1995 02:48:29 GMT
Organization: Eskimo North (206) For-Ever

Make sure that the client is running NT3.51 . If all the clients are using windoz, you can use Uniform Naming Conventions for your low level fuctions. Do not expect any i

What's fixed in CA-Clipper 5.2e ?

There is no doubt that the most stable version of CA-Clipper is...5.2e. In fact, most FiveWin programmers prefer 5.2e to 5.3 in developing 16 bit Windows Applications.

Here's a list of known problems resolved by CA-Clipper 5.2e:

o General DBFCDX RDD stability problems.

o DBFCDX RDD compability problems with FoxPro 2.6 during record locking and record append locking.

o 1210 errors with DBFCDX RDD after a reindex.

o Corrected problem with APPEND BLANK (dbappend()) if optional argument was not passed to function.

o Corrected General Protection Fault (GPF) in PICTURE clause.

o Corrected GPF in ERRORSYS when constantly selecting "Retry" in Error Alert box.

o Corrected GPF in SET ORDER TO (OrdSetFocus()) when used in a loop.

o Corrected Virtual Memory Integrity Failure (VMIF) error in TBrowse stabilization.

o Corrected GPF in TBrowse STABILIZE() method with repeated CTRL-PGDN/CTRL-PGUP key events.

o Added reserved words in NG file and Reserved.ch file.

o Added error handling code to NTXERR.PRG.

o Enhanced NG file for TBrowse, Get and Error Classes.

o And last, but not least... enjoy the added samples

Nb. This article was first posted by me on 30th July 2006

Clipper - Brief Version History

this page is suggested by Viktor Szel of Budapest, Hungary who also provided most if not all of the following materials.He posed a this question to me ... How about some "back to the good old days" history page including the very first versions of Nantucket Clipper ?

By the way, you may have noticed that there are some missing dates... Anyone out there like to fill in the missing dates ?

Here comes a brief version list:

Nantucket Clipper'85 Winter - 1985

Nantucket Clipper'86 Autumn - 31 October 1986

Nantucket Clipper'87 Summer - 21 December 1987

Nantucket Clipper 5.00 - 1990

Nantucket Clipper 5.01 - 15 April 1991

CA Clipper 5.01a - ?

CA Clipper 5.20 - 15 February 1993

CA Clipper 5.2a - ?

CA Clipper 5.2b - 25 June 1993

CA Clipper 5.2c - ?

CA Clipper 5.2d - 25 March 1994

CA Clipper 5.2e - 07 February 1995

CA Clipper 5.30 - 26 June 1995

CA Clipper 5.3a - 20 May 1996

Here comes some command line help screen from previous versions:

• The Clipper Compiler, Winter '85
• The Clipper Compiler, Summer '87
• Clipper (R) Version 5.01
• Clipper (R) Version 5.2

The Clipper Compiler, Winter '85

Copyright (c) 1985, 1986 Nantucket Inc., All Rights Reserved.

Developed by Brian Russell        (Happy Birthday Sean)
Richard McConnell

Rick Spence

David Dodson

Kevin Shepherd

Jon P. Rognerud

options:

-n   (enhanced ative code)

-l   (no ine numbers)

-s   (syntax check only)

@list.clp (list of files to compile)


A>clipper b:main.prg -l

A>clipper @list.clp -n

The Clipper Compiler, Summer '87

---

Copyright (c) Nantucket Corp 1985-1987. All Rights Reserved. Microsoft C Runtime Library Routines, Copyright (c) Microsoft Corp 1984-1987. All Rights Reserved.

Richard McConnell & Brian Russell, Sr. VP

Kevin Shepherd

Dennis Dias

Rick Spence

David Dodson

Jon P. Rognerud

Fred Ho

options: -l (no ine numbers)

-m (compile one odule only)

-o (bject file drive and/or path)

-p (

ause for disk change)

-q (uiet)

-s (syntax check only)

-v (ariables are assumed M->)

@list.clp (list of files to compile)

A>clipper a:main.prg -p -ob: C>clipper @list.clp -l -q

Clipper (R) Version 5.01

---

Copyright (c) Nantucket Corp 1985-1991. All Rights Reserved. Microsoft C Floating Point Support Routines Copyright (c) Microsoft Corp 1984-1987. All Rights Reserved.

Designed by Rich McConnell

Written by Denny Dias, Basil Hosmer and Rich McConnell

Options:

/a automatic memvar declaration

/b debug info

/credits credits screen

/d[=] #define

/i #include file search path

/l suppress line number information

/m compile module only

/n no implicit starting procedure

/o object file drive and/or path

/p generate pre-processed output (.ppo) file

/q quiet

/r[] request linker to search (or none)

/s syntax check only

/u[] use command def set in (or none)

/v variables are assumed M->

/w enable warnings

@ compile list of modules in

Clipper (R) Version 5.2

---

Copyright (c) 1985-1993, Computer Associates International, Inc. Microsoft C Floating Point Support Routines Copyright (c) Microsoft Corp 1984-1987. All Rights Reserved.

Options:

/a automatic memvar declaration

/b debug info

/credits credits screen

/d[=] #define

/es[] set exit severity

/i #include file search path

/l suppress line number information

/m compile module only

/n no implicit starting procedure

/o object file drive and/or path

/p generate pre-processed output (.ppo) file /q quiet

/r[] request linker to search (or none)

/s syntax check only

/u[] use command def set in (or none)

/v variables are assumed M->

/w enable warnings

@ compile list of modules in

Contributed by Steve Read, USA in July 1999

Some additional info for your 'Brief Version History' section

Sorry about any line wrapping.

*====  Nantucket Clipper versions  ====*

198? Nantucket Software founded

Clipper            Winter 84       (Shipped on 25 May 1985)

Clipper            Winter 85       (1985)       ProLocked version

Clipper            Autumn 86       (10/31/86 02:00:00)
PLink86plus      2.21

Clipper            Summer 87       (12/21/87)
PLink86plus      2.24

Clipper            5.0   Rev. 1.03 USASCII (09/15/90)
RTLink           3.11

Clipper            5.01            (04/15/91)
RTLink           3.13
Clipper            5.01  Rev. 1.29
Clipper            5.01  Rev. 1.47
Clipper            5.01a

Clipper Tools II   2.0
Clipper Tools II   2.5

CA-Clipper Compiler Kit for dBASE IV
2.0             (11-06-93 02:00:00)
(formerly Matt Whelan's Compiler Kit, dBASE IV
Edition)


According to the book written by Larry Heimendinger, former
President of
Nantucket Software - "Advanced Clipper", the first version was
Clipper
Winter '84, shipped on May 25, 1985.

In "Advanced Clipper" by Larry Heimendinger is a complete history of
the
development of Clipper and Nantucket is outlined.

The very first Clipper version was Winter '84 (shipped May '85) and
had
a manual of.... 54 pages (knowledge of dBASEIII was assumed). Winter
'85: 200 pages of documentation, additions of VALID clause and
SAVE/RESTORE SCREEN commands. Autumn '86 added the "==" operator,
one-dimensional arrays, and usage of functions as commands. Summer
'87
(released after Christmas) brought further enhancements and a
completely
revised documentation. I don't know about any patches, I know about
them
only from the BBS's...

Clipper 5.0 was announced at the June 1989 Clipper Developer's
Conference in Los Angeles, by Nantucket.

*====  Computer Associates Clipper versions  ====*

1992 Computer Associates International purchased Nantucket Corp.

Clipper Tools II      2.?

CA-Clipper            5.01a
CA-Clipper            5.01ae          (clipper.lib 16/10/92 6:00)

CA-Clipper            5.20 (Rev. 196) (02-15-93 05:20)
CA-Clipper            5.2a (Rev. 197) (03-15-93 05:21)
CA-Clipper            5.2b (Rev. 202) (06-10-93 05:22)
CA-Clipper            5.2c (Rev. 204) (08-06-93 05:23)
CA-Clipper            5.2d (Rev. 215) (03-25-94 05:24)
CA-Clipper            5.2e (Rev. 216) (02/07/95 05:25)

CA-Clipper Tools III  3.0             (02-23-93 03:00) CT30
patched files)
(09-01-95 03:52) Clip52
patched files)
(09-01-95 03:53) Clip53
patched files)
CA-Clipper Tools III  3.0a
CA-Clipper Tools III  3.0b

CA-Clipper            5.3             (06-26-95 05:30)
CA-Clipper            5.3a            (05-20-96 05:31)
CA-Clipper            5.3b (Rev. 338) (05-20-97 05:32)

*====  Computer Associates Visual Objects versions  ====*

CA-Visual Objects     1.0   (16-bit) released in December 1994
CA-Visual Objects     1.0a
CA-Visual Objects     1.0b
CA-Visual Objects     1.0c
CA-Visual Objects     1.0d  stable version of 1.0x
CA-Visual Objects     2.0   (32-bit)
CA-Visual Objects     2.0.0 (Build 770)
CA-Visual Objects     2.0b3
CA-Visual Objects     2.5   to be released in July 1999

*===*  Notes  *====*
Visual Objects        1.x   The 16-bit versions. the early versions
of
VO1 was very flaky but is now stable
(VO1d).

Visual Objects        2.0   The 32-bit versions. Vo2 is superb and can  match anything on the market. The
advance news of the next release, 2.5, looks exciting