Monday, February 23, 2009

Xbase++ vs Harbours: Clarifications on Multi-threading by Steffen F. Pirsig

Sorry to use your blog to comment on the comments of others, as I think this is not the idea of a blog. Anyway please allow me to clarify my statement. I said "clean and easy to use way of multithreading", i didn’t say the Harbours don’t support Multithreading. My statement is still true, even so Harbour and xHarbour have implemented the ability to execute code in multiple threads and have implemented some of the interfaces Xbase++ provides in one way or another. They are still far away from the idea and concepts of Xbase++ in that area. In addition Harbour and xHarbour implemented different semantics of isolation. Which makes porting of complex MT applications for sure a mess. Let me clarify that.

Multithreading as the ability to execute code in different code paths is a feature of modern OS sinces decades. The problem with MT is that it adds another dimension of complexity to the developers task. While with single threaded apps. the developer needs only to think in a more or less sequential way with MT each execution path adds a new dimentions to the equation of programm complexity. Development languages supporting MT such as Delphi, .NET (C#,VB) or Harbour and xHarbour support MT thats correct, but they do not remove the burden of correctness from the programmer. It is in the sole responsibility of the programmer to ensure programm correctness in two different areas; data-consistency and algorithm isolation.

The problem of data consistency occurs as soon as more than one thread is accessing the same data - such as a simple string or an array. Besides nuances in terms of single or multiple readers/writers the consistency of the data must be ensured, so developers are forced to use mutex-semaphores or other higher level concepts such monitors, guards... to ensure data-consistency.

Algorithm isolation is somewhat related to data-consistency, it becomes obvious that a linked-list accessed from multiple threads must be protected otherwise dangling pointer occurs. But what about a table/relation of a database. The problem here is that concurrency inside the process can be resolved - but this type of "isolation" does break the semantics of the isolation principles which are already provided by the underlying dbms (sql-isolation-levels, record or file locks, transactions). Therefore algorithm isolation/correctness is a complete different beast as it is located at a very high semantic level of the task.

Alaska Software has put an enormous amount of research efforts into that area and we have more than a decade of practical experience with that area based on real world customers and real world applications. From that point of view I would like to reiterate my initial statement "As of today there is still no tool available in the market which provides that clean and easy to use way of multithreading".

Lets start with xHarbour, its MT implementation is not well thought, as it provides MT features to the programmer without any model, just the features. xHarbour even allows the usage of a workarea from different threads which is a violation of fundamental dbms isolation principles. In fact xHarbour ist just a system language in the sense of MT and makes life not really easier compared with other system languages. Therefore there is no value in besides being able to do MT. Also keep in mind due to the historical burden of the VM and RT core the MT feature is implemented in a way making it impossible to scale in future multi-core scenarios (see later-note).

Harbour is better here because if follows more the principles of Xbase++, while I am not sure if the Harbour people have decided to adapt the Xbase++ model for compatibility reasons or not I am glad to see that they followed our models point of view. The issues with Harbour however is that it suffers from the shortcoming of its runtime in general, the VM design and of course the way how datatypes - the blood of a language - are handled. It is still in a 1980 architectual style centered around the original concept how Clipper did it. This is also true for xHarbour, so both suffer from the fact that MT was added I think in 2007, while the VM and RT core is from 1999 - without having MT in mind.

This is in fact one of the biggest differences between
Xbase++ and the "Harbours" from a pure architectual point of view, we designed a runtime architecture from the beginning to be MT/MP and Distributed, they designed a runtime based on the DOS Clipper blueprint.

In fact, I could argue on and on, specifically it it comes to dedicated implementations of the Harbour runtime core or the Harbour VM but sharing these type of technical details is of course definitively not what I am paid for -;) Anyway allow me to make it clear in a general terms.

First, any feature/functionality of Xbase++ is reentrant there is not a single exception of this rule. Second, any datatype and storage type is thread-safe regardless of its complexity so there is no way to crash an Xbase++ process using multithreading. Third, the runtime guarantees that there is no possibility of a deadlock in terms of its internal state regardless what you are doing in different threads. There is a clean isolation and inheritance relationship of settings between different threads. In practical terms that means, you can output to the console from different threads without any additional code, you can execute methods or access state of GUI (XbasePARTS) objects from different threads, you can create a codeblock which detaches a local variable and pass it to another thread, you are performing file I/O or executing a remote procedure call and in the meanwhile the async. garbagge collector cleans up your memory - and the list goes on... But in Xbase++ you can do all that without the need to think about MT or ask a question such as "Is the ASort() function thread safe" or can I change the caption of a GUI control from another thread. Thats all a given, no restrictions apply, the runtime does it all automatically for you.

Anyway, I like Harbour more than xHarbour in terms of MT support. However the crux is still there, no real architecture around the product, leading to the fact that MT is supported form a technical point of view but not from a 4GL therefore leading to a potential of unnecessary burden for the average programmers, and of course that was and is still not the idea of Clipper as a tool.

Btw, the same is true for VO or so, they left the idea of the language and moved to something more like a system -language, while I can understand that somewhat I strongly disagree with that type of language design for a simple reasons; its not practical in the long term - we will see that in the following years as more and more multi core system will find their way in the mainstream and developers need to make use of them for performance and scaleability reasons. In 10 - 15 years from now we will have 100 if not thousands cores per die - handling multithreading , synchronisation issues by hand becomes then impossible, the same is true for offloading tasks for performance reasons. So there is a need for a clean model in terms of the language - thats at least into what we believe at Alaska Software. It goes even further, the current attempty by MS in terms of multicore support with VS2010 or NET 4.0 are IMO absolutely wrong, as they force the developer to write code depending on the underlaying execution infrastructure alias cores available. In other words, infrastructure related code/algorithms get mixed with the original algorithm the developers writes and of course the developer gets payed for. Thats a catastrophic path which for sure does not contribute to increased productivity and reliability of software solutions.

Funnily enough, the most critical, and most difficult aspect in that area; getting performance gains from multi core usage is even not touched with my technical arguments right now. However it adds another dimension of complexity to the previous equation as it needs to take into account the memory hierarchy which must be handled by a 4GL runtime totally different as it is with the simple approach of Harbour/xHarbour. Their RT core and VM needs a more or less complete rewrite and redesign to go that path.

In other words, Xbase++ is playing in the Multithreading ballpark since a decade. Harbour is still finding its way into the MT ballpark while xHarbour is in that context at a dead-end.

I would bet that Xbase++ will play in the multicore ballpack while the Harbours are still with their MT stuff.

In a more theoretical sense, it is important to understand that a programming language and its infrastructure shall not adapt any technical feature, requirement or hype. Because then the language and infrastucture are getting more and more complicated up to an point of lost control. Also backward compatibility and therefore protection of existing investments becomes more and more a mess with Q&A costs going through the roof.

Nor is it a good idea to provide software developers any freedom - the point here is, a good MT modell does smoothly guide the developer through the hurdels and most of the time is even not in the awareness of the developer. The contrary is providing the developer all freedom, but this leads to letting him first build the gun-powder, then the gun to finally shoot a bullet -;)

Therefore let me rephrase my initial statement to be more specific;

As of today there is still no tool available in the market which provides that clean and easy to use way of multithreading, however there are other tools which support MT - but they support it just as an technical feature without a modell and thats simple wrong as it leads to additional dimensions in code complexity - finally ending in applications with lesser reliability and overall quality.

Just my point of view on that subject - enough said

Steffen F. Pirsig

No comments:

Post a Comment

Welcome to Clipper... Clipper... Clipper


In 1997, then using Delphi 3, I had already created 32-bits Windows applications for HRIS, ERP and CRM. In 2007, using Ruby on Rails, an AJAX powered CRM site running on Apache & MySQL was created and I am now using Visual Studio .Net 2008 to create web-based projects and Delphi 7 for Win32 applications using SQL2005 & DBFCDX.

So, why then am I reviving the Original Clipper... Clipper... Clipper via a Blog as CA-Clipper is a programming language for the DOS world ? Believe it or not, there are still some clients using my mission-critical CA-Clipper applications for DOS installed in the late 80's and up to the mid 90's. This is testimony to CA-Clipper's robustness as a language :-)

With the widespread introduction of Windows 7 64-bits as the standard O/S for new Windows based PCs & Notebooks, CA-Clipper EXE simply will not work and it has become imperative for Clipper programmers to migrate immediately to Harbour to build 32/64 bits EXEs

Since 28th January 2009, this blog has been read by 134,389 (10/3/11 - 39,277) unique visitors (of which 45,151 (10/3/11 - 13,929) are returning visitors) from 103 countries and 1,574 cities & towns in Europe (37; 764 cities), North America (3; 373 cities) , Central America & Caribeans (6; 13 cities), South America(10; 226 cities), Africa & Middle-East (12; 44 cities) , Asia-Pacific (21; 175 cities). So, obviously Clipper is Alive & Well : -)


TIA & Enjoy ! (10th October 2012, 11:05; 13th November 2015)


Original Welcome Page for Clipper... Clipper... Clipper

This is the original Welcome Page for Clipper... Clipper... Clipper, which I am republishing for historical and sentimental reasons. The only changes that I have made was to fix all the broken links. BTW, the counter from counter.digits.com is still working :-)

Welcome to Chee Chong Hwa's Malaysian WWW web site which is dedicated to Clipperheads throughout the world.

This site started out as a teeny-weeny section of Who the heck is Chee Chong Hwa ? and has graduated into a full blown web site of more than 140 pages (actually hundreds of A4 size pages) ! This is due to its growing popularity and tremendous encouragements from visiting Clipperheads from 100 countries worldwide, from North America, Central America, Caribbean, South America, Europe, Middle-East, Africa and Asia-Pacific. Thanx Clipperheads, you all made this happen !


What is Clipper ?

You may ask, what is this Clipper stuff ? Could Clipper be something to do with sailing as it is the name of a very fast sailing American ship in the 19th century ?

Well, Clipper or to be precise, CA-Clipper is the premier PC-Software development tool for DOS. It was first developed by Nantucket Corporation initially as a compiler for dBase3+ programs. Since then, CA-Clipper has evolved away from its x-base roots with the introduction of lexical scoping & pre-defined objects like TBrowse. As at today, the most stable version ofClipper is 5.2e while the latest version, 5.3a was introduced on 21 May 1996.

As at 11th November, 1996, an unofficial 5.3a fixes file was made available by Jo French. See the About CA-Clipper 5.3a section for more details. BTW, Jo French uploaded the revised 5.3a fixes file on 20th November, 1996.

Latest News

The latest news is that CA has finally released the long-awaited 5.3b patch on 21 May, 1997.

For 5.3b users, you must a take a look at Jo French's comments on unfixed bugs in 5.3b.

BTW, have you used Click ? If you're a serious Clipperprogrammer and need an excellent code formatter, Click is a natural choice. How to get it ? Simple, access Phil Barnett's site via my Cool Clipper Sites.

32-bits Clipper for Windows ?

Have you tried Xbase ++ ? Well, I have and compared to Delphi (my current Windows programming tool of choice), I'm still sticking to Delphi.

Anyway, you should visit the Alaska Home Page. Give it a chance and then draw your own conclusions !.

The Harbour Project

Is this the future of Xbase ? Take a look at at the Harbour Project

You are Visitor # ...

According to counter.digits.com, you are visitor since 3 June 1996.

If you like or dislike what you see on this website, please drop me a line by clicking the email button at the bottom of this page or better still, by filling out the form in my guest book. If you are not sure what to write,click here to take a look at what other Clipperheads have to say.