Friday, September 26, 2008

A Sink Programming.

In this post I demonstrated how you can use an Anonymous Method to "synchronize" a background thread with the main UI thread. However there are times where you don't want to block execution of the thread but still want to have something happen in the main UI thread, asynchronously. For several releases of Delphi there has been the Queue() method on TThread. This allows you to schedule a TThreadMethod (just like Synchronize) to execute on the main UI thread. The difference is that Synchronize will block the caller until the UI thread completes the call, while Queue will return immediately. The problem with the Queue() method is that there is no simple way to know when the "queued" event is done executing. Queue() really only works in a "queue it and forget it" scenario. It is still a form of Async programming has limited usefulness. New to D2009, a new Queue() overload was added just like Synchronize() that takes "reference to procedure" type (the underlying type to which you can assign an Anonymous Method). Let's see if we can use the Queue() method as the underlying mechanism for doing some "A Sink" programming :-).

Since the point of this exercise is to create something that we can use to asynchronously run some code on the UI thread and also know when it is complete, we need something that will represent a particular scheduled event. For this we'll declare an interface:

IAsyncResult = interface
function GetAsyncHandleObject: THandleObject;
function GetCompletedSynchronously: Boolean;
function GetIsCompleted: Boolean;
property AsyncHandleObject: THandleObject read GetAsyncHandleObject;
property CompletedSynchronously: Boolean read GetCompletedSynchronously;
property IsCompleted: Boolean read GetIsCompleted;

The next thing I'm going to do is a trick I'm going to use to "extend" the existing framework using a class helper. This allows us to syntactically add something to the framework we will eventually fold into the class itself. If you looked closely we did this for the Windows Vista extensions to VCL for D2007, then in D2009, all those added methods and properties were folded directly into the classes themselves and the helpers were removed. Any consumers of the D2007 code could remain unchanged. But, I digress...

  TControlHelper = class helper for TControl
function BeginInvoke(const AProc: TProc): IAsyncResult; overload;
function BeginInvoke<T>(const AFunc: TFunc<T>): IAsyncResult; overload;
procedure EndInvoke(const ASyncResult: IAsyncResult); overload;
function EndInvoke<T>(const AsyncResult: IAsyncResult): T; overload;

The non-generic versions of the methods are used for normal procedure calls where you're only interested in calling some method that doesn't have a return value. The non-generic EndInvoke() method simply blocks until the method call is complete and returns. If the call is complete before calling EndInvoke, then it will return immediately. If you call BeginInvoke from the UI thread, then the Proc is executed synchronously, and CompletedSynchronously returns true. The generic versions of these methods allow you to asynchronously call a function and get the return value once it becomes available. Using the AsyncHandleObject property on IAsyncResult, you can dispatch several different async calls with BeginInvoke() and then wait for one or all of them to complete using the new class function THandlObject.WaitForMultiple() in SyncObjs.pas. Another thing is that if an exception is raised during the execution of the Proc, it will be caught, saved off and then re-raised when you call EndInvoke(). So you should always call EndInvoke() at some point.

So that's an overview of the exposed interfaces. In this example, this is the execute method of a TThread descendant that searches a disk folder for all the files and fills up a list box the results. An edit box on the form holds the path to the folder, which is read asynchronously. Yes, a regular "Synchronize" could have been used in this case, but I needed a quick little example:

procedure TBeginInvokeTestForm.TSearchThread.Execute;
SR: TSearchRec;
SH: Integer;
AR: IAsyncResult;
if not Terminated then
AR := FForm.BeginInvoke<string>(TFunc<string>(function: string
Result := FForm.Edit1.Text;
FFolder := FForm.EndInvoke<string>(AR);
SH := FindFirst(IncludeTrailingPathDelimiter(FFolder) + '*.*', faAnyFile, SR);
while (SH = 0) and not Terminated do
//Sleep(10); // this makes the background thread go a little slower.
AR := FForm.BeginInvoke(procedure
if not Terminated then
SH := FindNext(SR);

This shows both the usage of the generic and non-generic BeginInvoke/EndInvoke functions. In the next post we'll start to look at the implementation behind all of this. We'll also start to look at how this same pattern can be used for async IO using I/O Completion ports and a simple thread pool. There is also a little "gotcha" in the above code using this technique that I'll explain.

Yes, this this looks remarkably like what is available in .NET. Hey they co-opted a bunch of ideas from Delphi and VCL... why not return the favor :-).

NOTE: The TFunc<string>() cast in the first BeginInvoke<string>() call is to work around an overload resolution bug that is slated to be fixed in a future service pack.


  1. > "in D2009, all those added methods and properties were folded directly into the classes themselves and the helpers were removed"

    Can you elaborate? I thought the reason for implementing them as helpers in the first place was because you couldn't roll them into the framework for compatibility reasons.

    Also, I seem to recall that the 'helpers' weren't always entirely compatible with C++ Builder - the auto-generated HPP files didn't include the helpers, and needed to be hacked manually to have them added. Have I got that bit right?

    - Roddy

  2. Nice blog and a very informative one Allen ... as usually !

    There is any chance to see all these goodies when stabilized (additionally to Barry's one) in future VCL/RTL (maybe added at the time of RAD2009) ? Many will benefit a lot !

    Regarding the class helpers I have some questions:
    1) It is possible to use helpers for interfaces and for records ?
    2) Since it seams that just a single helper is allowed for a specific class (ex: TMyClass, but can be added to the classes that extendens TMyClass) how can multiple helpers could be added to the "same" class (TMyClass) ?
    (by inheriting the original one ?)
    3) How we can use RTTI to know that a specific method/property was added using a helper ?
    4) How we can use RTTI to know if a specific class has a helper, and witch type that helper is ?
    4) The helpers are just a "source level" (compile time) feature ? Since you used helpers in D2007 to add various (non breaking) Vista features, how we can add to T(Custom)Frame (compiled) another helper in our own code ?

    10x for any clarifications & keep posting on the nice stuff that we can do with Delphi !

  3. "Hey they co-opted a bunch of ideas from Delphi and VCL why not return the favor"

    And how!

    Next time someone throws up the "CodeGear is just copying .NET" line, this should be the standard answer.


Please keep your comments related to the post on which you are commenting. No spam, personal attacks, or general nastiness. I will be watching and will delete comments I find irrelevant, offensive and unnecessary.