Saturday, September 28, 2013

Locking schemes used with DBF files by Przemyslaw Czerpak

Locking schemes used with DBF files.
Przemyslaw Czerpak (druzus/at/poczta.onet.pl)


Exclusive/shared modes (DENY READ, DENY WRITE) in POSIX systems.
================================================================

Exclusive modes (DENY READ, DENY WRITE) do not exist in POSIX systems.
They can be emulated by fcntl() or flock() locks. Harbour uses flock()
in systems where it does not create conflicts with fcntl() locks used
to synchronize file access by different processes in shared mode.
Currently it's enabled by default only in Linux (tests with *BSD systems
and MacOSX showed that it interacts with fcntl locks causing deadlocks).
On other POSIX systems Harbour uses fcntl() read and write locks at
address 0x7fffffff (size 1) to simulate shared and exclusive mode.

CLIP and FlagShip use fcntl() locks to simulate shared/exclusive modes.
CLIP shared/exclusive flag:     0x7fffffff : 1 (the same as Harbour)
FlagShip shared/exclusive flag: 0x7ffffbfc : 1




File access synchronization (record and file locking) in shared mode.
=====================================================================

FlagShip uses different record and index locking schemes
(i.e. for RLOCK it locks record area) and it is not described
here.


In documentation CLIP can use Clipper like or FoxPro like locking
schemes and it can be controlled by SET LOCKSTYLE [TO] ... command
or Set( _SET_LOCKSTYLE, 0 | 1 ) function.
This switch only changes the RLOCK/FLOCK offset.
For:
   SET LOCKSTYLE [TO] CLIPPER | CLIPPER50 | CLIPPER53
or:
   Set( _SET_LOCKSTYLE, 0 )
this offset is set to 1'000'000'000. This is header lock address
and records are locked at above address + record number.
This is compatible with standard Clipper locking.
Commands:
   SET LOCKSTYLE [TO] CLIPPER52 | FOXPRO | SIX
or:
   Set( _SET_LOCKSTYLE, 1 )
set this offset to 0x10000000 keeping the same algorithm
for record locking what is incompatible with any other locking
schemes known by me. For sure it's not compatible with [V]FP
locking (see details about this locking scheme below).
For more information look at CLIP source code.




(*) In Harbour index read locks are shared on platforms which support
    shared (read) locks.

Clipper DBFNTX locking:
=======================
all locks are exclusive (*)
DBF HEADER LOCK: @1000000000 : 1 (exclusive)
DBF RECORD LOCK: @1000000000 + recNO : 1 (exclusive)
DBF FLOCK SIZE:  1000000000
   => maximum records: 3'294'967'295
      maximum file size in non POSIX systems: 1'000'000'000

NTX READ LOCK:   @1000000000 : 1 (exclusive)
NTX WRITE LOCK:  @1000000000 : 1 (exclusive)

In Harbour it's DB_DBFLOCK_CLIPPER.


Clipper new DBFNTX locking (ntxlock2.obj):
==========================================
all locks are exclusive (*)
DBF HEADER LOCK: @4000000000 : 1 (exclusive)
DBF RECORD LOCK: @4000000000 + recNO : 1 (exclusive)
DBF FLOCK SIZE:  294967295
   => maximum records: 294'967'295
      maximum file size in non POSIX systems: 4'000'000'000

NTX READ LOCK:   @1000000000 : 1 (exclusive)
NTX WRITE LOCK:  @1000000000 : 1 (exclusive)

In Harbour it's DB_DBFLOCK_CLIPPER2.


CL52 DBFCDX, SIX3 SIXCDX, SIX3 SIXNSX, [V]FP CDX
================================================
all locks are exclusive (*)
when there is no structural index:
DBF HEADER LOCK: @0x40000000 : 1
DBF RECORD LOCK: @0x40000000 + record offset : 1 (some implementations
                                                  locks whole record)
DBF FLOCK SIZE:  0x3ffffffd
   => maximum file size: 1'073'741'823

when structural index is open:
DBF HEADER LOCK: @0x7ffffffe : 1
DBF RECORD LOCK: @0x7ffffffe - recNO : 1
DBF FLOCK SIZE:  0x07ffffff
   => maximum records: 134'217'727
      maximum file size in non POSIX systems: 2'013'265'919

index file locking is the same for production and normal indexes:
CDX READ LOCK:   @0x7ffffffe : 1
CDX WRITE LOCK:  @0x7ffffffe : 1

In Harbour it's DB_DBFLOCK_VFP.


CL53 DBFCDX, COMIX (hyper locking)
==================================
all locks are exclusive (*), in index shared locks are emulated

DBF HEADER LOCK: @1000000000 : 1
DBF RECORD LOCK: @1000000000 + recNO : 1
DBF FLOCK SIZE:  1000000000
   => maximum records: 1'000'000'000
      maximum file size in non POSIX systems: 1'000'000'000

CDX READ LOCK:   @0xffff0000 + random value from 0x0 to 0xffff : 1
CDX WRITE LOCK:  @0xfffeffff : 0x10001
    if fail then @0xfffeffff : 1
    prepare the index modification in memory and before writing to
    index file lock @0xffff0000 : 0x10000
  to eliminate starvation effect caused by many readers
  on each 16-th read lock reading process tries to lock write
  area @0xfffeffff : 1 instead and then sets normal read lock

In Harbour it's DB_DBFLOCK_COMIX.


HARBOUR 32-bit locking
======================
all locks are exclusive (*), in index shared locks are emulated

DBF HEADER LOCK: @4000000000 : 1 (exclusive)
DBF RECORD LOCK: @4000000000 + recNO : 1 (exclusive)
DBF FLOCK SIZE:  294967295
   => maximum records: 294'967'295
      maximum file size in non POSIX systems: 4'000'000'000

CDX READ LOCK:   @0xffff0000 + random value from 0x0 to 0xffff : 1
CDX WRITE LOCK:  @0xfffeffff : 0x10001
    if fail then @0xfffeffff : 1
    prepare the index modification in memory and before writing to
    index file lock @0xffff0000 : 0x10000
  to eliminate starvation effect caused by many readers
  on each 16-th read lock reading process tries to lock write
  area @0xfffeffff : 1 instead and then sets normal read lock

In Harbour it's DB_DBFLOCK_HB32.


HARBOUR 64-bit locking
======================
all locks are exclusive (*), in index shared locks are emulated

DBF HEADER LOCK: @0x7fffffff00000001 : 1
DBF RECORD LOCK: @0x7fffffff00000001 + recNO : 1
DBF FLOCK SIZE:  0xfffffffeUL
   => maximum records: 4'294'967'294
      maximum file size: no limits created by locks

CDX READ LOCK:   @0x7fffffff00000000 + random value from 0x1 to 0xffff : 1
CDX WRITE LOCK:  @0x7fffffff00000000 : 0x10001
    if fail then @0x7fffffff00000000 : 1
    prepare the index modification in memory and before writing to
    index file lock @0x7fffffff00000001 : 0x10000
  to eliminate starvation effect caused by many readers
  on each 16-th read lock reading process tries to lock write
  area @0x7fffffff00000000 : 1 instead and then sets normal read lock

In Harbour it's DB_DBFLOCK_HB64.

No comments:

Post a Comment

Welcome to Clipper... Clipper... Clipper


In 1997, then using Delphi 3, I had already created 32-bits Windows applications for HRIS, ERP and CRM. In 2007, using Ruby on Rails, an AJAX powered CRM site running on Apache & MySQL was created and I am now using Visual Studio .Net 2008 to create web-based projects and Delphi 7 for Win32 applications using SQL2005 & DBFCDX.

So, why then am I reviving the Original Clipper... Clipper... Clipper via a Blog as CA-Clipper is a programming language for the DOS world ? Believe it or not, there are still some clients using my mission-critical CA-Clipper applications for DOS installed in the late 80's and up to the mid 90's. This is testimony to CA-Clipper's robustness as a language :-)

With the widespread introduction of Windows 7 64-bits as the standard O/S for new Windows based PCs & Notebooks, CA-Clipper EXE simply will not work and it has become imperative for Clipper programmers to migrate immediately to Harbour to build 32/64 bits EXEs

Since 28th January 2009, this blog has been read by 134,389 (10/3/11 - 39,277) unique visitors (of which 45,151 (10/3/11 - 13,929) are returning visitors) from 103 countries and 1,574 cities & towns in Europe (37; 764 cities), North America (3; 373 cities) , Central America & Caribeans (6; 13 cities), South America(10; 226 cities), Africa & Middle-East (12; 44 cities) , Asia-Pacific (21; 175 cities). So, obviously Clipper is Alive & Well : -)


TIA & Enjoy ! (10th October 2012, 11:05; 13th November 2015)


Original Welcome Page for Clipper... Clipper... Clipper

This is the original Welcome Page for Clipper... Clipper... Clipper, which I am republishing for historical and sentimental reasons. The only changes that I have made was to fix all the broken links. BTW, the counter from counter.digits.com is still working :-)

Welcome to Chee Chong Hwa's Malaysian WWW web site which is dedicated to Clipperheads throughout the world.

This site started out as a teeny-weeny section of Who the heck is Chee Chong Hwa ? and has graduated into a full blown web site of more than 140 pages (actually hundreds of A4 size pages) ! This is due to its growing popularity and tremendous encouragements from visiting Clipperheads from 100 countries worldwide, from North America, Central America, Caribbean, South America, Europe, Middle-East, Africa and Asia-Pacific. Thanx Clipperheads, you all made this happen !


What is Clipper ?

You may ask, what is this Clipper stuff ? Could Clipper be something to do with sailing as it is the name of a very fast sailing American ship in the 19th century ?

Well, Clipper or to be precise, CA-Clipper is the premier PC-Software development tool for DOS. It was first developed by Nantucket Corporation initially as a compiler for dBase3+ programs. Since then, CA-Clipper has evolved away from its x-base roots with the introduction of lexical scoping & pre-defined objects like TBrowse. As at today, the most stable version ofClipper is 5.2e while the latest version, 5.3a was introduced on 21 May 1996.

As at 11th November, 1996, an unofficial 5.3a fixes file was made available by Jo French. See the About CA-Clipper 5.3a section for more details. BTW, Jo French uploaded the revised 5.3a fixes file on 20th November, 1996.

Latest News

The latest news is that CA has finally released the long-awaited 5.3b patch on 21 May, 1997.

For 5.3b users, you must a take a look at Jo French's comments on unfixed bugs in 5.3b.

BTW, have you used Click ? If you're a serious Clipperprogrammer and need an excellent code formatter, Click is a natural choice. How to get it ? Simple, access Phil Barnett's site via my Cool Clipper Sites.

32-bits Clipper for Windows ?

Have you tried Xbase ++ ? Well, I have and compared to Delphi (my current Windows programming tool of choice), I'm still sticking to Delphi.

Anyway, you should visit the Alaska Home Page. Give it a chance and then draw your own conclusions !.

The Harbour Project

Is this the future of Xbase ? Take a look at at the Harbour Project

You are Visitor # ...

According to counter.digits.com, you are visitor since 3 June 1996.

If you like or dislike what you see on this website, please drop me a line by clicking the email button at the bottom of this page or better still, by filling out the form in my guest book. If you are not sure what to write,click here to take a look at what other Clipperheads have to say.