Tuesday, September 25, 2007

Who is going where to attend what!? The ISO C++ committee meetings.

Yep.  That's right.  This weekend I'll be boarding a plane to attend the ISO C++ meetings next week.  So why is the long-term Delphi guy going to attend something for C++?  Since I do things for CodeGear across the product lines in different capacities, it isn't too much of a stretch.  But there are a couple of really interesting things happening within the C++ community surrounding the standards that may be of interest on the Delphi side of things.  This is concurrency and garbage collection.  I'll be attending these meetings along with Alisdair Meredith, the new C++Builder product manager, and one of the C++ compiler engineers.  Apparently among the many topics of discussion will be on the concurrency side with debates on memory models, variable volatility, and fence-posting.  Some discussion will certainly ensue surrounding the OpenMP folks and their requirements.  On the GC front, things are a ways off from being finalized as there are, apparently, several camps aligned with different views on this subject.  This is interesting in light of the fact that we've actually had several internal conversations about GC on the Delphi/Win32 side (nothing to see here folks, move along, move along ;-).  I'd like to see how they're going to handle adding GC to a language with such a long legacy of deterministic resource manipulation.

So if you're going to be in Kona, Hawaii from September 29th through October 6th, let me know and maybe we can meet up and chat.  Oh this is going to be such a rough trip... ;-).

Friday, September 21, 2007

Super editor zoom - proof of concept.

David Clegg mentions in this post a new super secret, "super zoom" feature.  First of all the reason this "feature" isn't on by default is that it was implemented as a very quick hack to more or less simply be a proof of concept for the team to play with.  As such, it is riddled with problems that if you start using it you may encounter.  It is actually somewhat ironic that this comes up now because for the last couple of days I've been looking at this feature and how it should actually be implemented.

The way it is currently implemented is to simply hide/show the dock sites on the editor form.  This causes a couple of major issues.  First of all, when in the zoomed mode, you cannot easily get back to the other views without unzooming first (and you must to use the mouse for this operation).  Zoom the editor, then select View|Project Manager... nothing happens.  That is because the form is visible, but the site into which it is docked is not.  Try switching desktop states, by starting the debugger or manually from the desktop state dropdown... then it gets really interesting.  Again, it was only for some internal research and a proof of concept.

Similar to my post about adding auto-save and recovery of files to the IDE, I'll go through and outline the behavior I'd expect this feature to adhere to.  As I mentioned in the previous paragraph, this feature takes the somewhat naive approach of hiding and showing the dock sites.

The first modification to this code is to iterate through all the dock sites on the editor form and then go through all the docked clients within each side and hide them individually.  This will allow the docking system to properly track their state.  It will also allow you to still selectively re-show specific views.  Selecting View|Project Manager will now properly display the Project Manager view in its properly docked position.   The next change is how the "unzoom" is handled.  As there can be several views that are not visible when the view is zoomed, you want to make sure they remain hidden when unzoomed unless they were explicitly shown while zoomed.  Another requirement is that for all "unpinned" views, they should remain unpinned and accessible.  So there could still be a thin strip on the sides and bottom where the views can still pop up.  Finally, any large desktop changes, such as switching states, loading a new project with a saved desktop, etc should cancel the zoom state.

I'm sure the next question on everyone's mind is why wasn't this feature just finished and done right for RAD Studio?  The simple truth is that it fell too low on the priority list to make it into the product.  Things like this are always going to happen where good intentions just sometimes don't make it because of the reality of the schedule and other priorities.  Another side effect of this is that we can also get some direct feedback from the field as to the overall usefulness of the concept (irrespective of the implementation) before devoting actual resources to it.  By all means, follow the instructions from David and play with it.  Even given its limitations and shortcomings there is enough there to get the idea of what it would do.

Wednesday, September 19, 2007

Wading in the shallow end of the pool - Thread Pools

A few weeks ago I was following with interest the announcement by Intel where they open-sourced their Thread Building Blocks (TBB) library. OK, so it is written in C++, makes heavy use of templates, and is generally like nearly every C++ library out there with heavy macro usage and umpteen levels of nest macros... Besides all of that, either I've gotten way better at reading and understanding C++ or these libraries are simplifying things, I was able to understand it quite well. This got me thinking a bit and so I decided to take a quick look at Windows Thread Pools.

A thread pool is simply a way to more efficiently utilize the CPU(s) while incurring as little thread context switching overhead as possible. If your application needs to do lots of quick little tasks and can do them concurrently, the build-up and tear-down overhead of threads may actually cancel out any gains from trying to push these tasks into threads. Think of thread pooling as simply a way to queue up a bunch of work-items and allow them to run on an number of available threads that are created once and used over and over. Thus you can amortize the build-up and tear-down of the threads across all the work they do. Once a work item is assigned to a thread, that work items has exclusive access to that particular thread until it is finished and allows the thread to return to the pool. A thread pool is not the solution for everything. For instance, if you have a thread that is blocked most of the time, you should not do that on a thread pool thread.

Off I went on a discovery mission to see how Windows thread pooling works. I wanted to visually see what was going on as I scheduled or queued work items. I came up with a simple TThreadPool class. Here's the declaration of what I came up with:

  TThreadPool = class
      TUserWorkItem = class
        FSender: TObject;
        FWorkerEvent: TNotifyEvent;
    class procedure QueueWorkItem(Sender: TObject; WorkerEvent: TNotifyEvent; Flags: ULONG); overload; static;
    class procedure QueueWorkItem(Sender: TObject; WorkerEvent: TNotifyEvent); overload; static;
    class procedure QueueIOWorkItem(Sender: TObject; WorkerEvent: TNotifyEvent); static;
    class procedure QueueUIWorkItem(Sender: TObject; WorkerEvent: TNotifyEvent); static;

You'll notice that this is designed to be a singleton class since all methods are class static. The main function of interest is the QueueWorkItem. What this does is simply schedule the WorkerEvent to be called from a thread pool thread whenever that is. It is up to you to make sure that the instance on which the WorkerEvent event is called is still valid at the time it is called. The other two methods simply correspond to some of the flags you can pass to QueueUserWorkItem. They're not used right now. Sender is passed through to the event handler specified by WorkerEvent, so that object should contain the context in which that task item is to work.

Now here's the implementation of that class:

function InternalThreadFunction(lpThreadParameter: Pointer): Integer; stdcall;
  Result := 0;
      with TThreadPool.TUserWorkItem(lpThreadParameter) do
        if Assigned(FWorkerEvent) then
    // Eventually this will need to somehow synchronously notify the main thread and either reraise the exception over there or
    // otherwise provide some information about the exception to the main thread.

{ TThreadPool }

class procedure TThreadPool.QueueWorkItem(Sender: TObject; WorkerEvent: TNotifyEvent);
  QueueWorkItem(Sender, WorkerEvent, WT_EXECUTEDEFAULT);

class procedure TThreadPool.QueueIOWorkItem(Sender: TObject; WorkerEvent: TNotifyEvent);
  QueueWorkItem(Sender, WorkerEvent, WT_EXECUTEINIOTHREAD);

class procedure TThreadPool.QueueUIWorkItem(Sender: TObject; WorkerEvent: TNotifyEvent);
  QueueWorkItem(Sender, WorkerEvent, WT_EXECUTEINUITHREAD);

class procedure TThreadPool.QueueWorkItem(Sender: TObject; WorkerEvent: TNotifyEvent; Flags: ULONG);
  WorkItem: TUserWorkItem;
  if Assigned(WorkerEvent) then
    IsMultiThread := True;
    WorkItem := TUserWorkItem.Create;
      WorkItem.FWorkerEvent := WorkerEvent;
      WorkItem.FSender := Sender;
      if not QueueUserWorkItem(InternalThreadFunction, WorkItem, Flags) then

To see just what is going on I wrote this little application:

The numbers in the list box represent the thread ID for the thread that is currently running. The band of colors visually show how the threads are scheduled. What is interesting is that this is what it looks like after about the 3rd or 4th time it runs. The first time it runs, each color is painted in sequence in a clearly serialized manner. Subsequent iterations seem to interleave more and more. This is running on a dual core system. You can get the whole application in Code Central.

As multi-core systems become more and more mainstream (aren't they already??), your applications really should begin to take advantage of them. The problem is that multi-threaded, or concurrent, programming is not very easy since we humans tend to think serially and so it is conceptually a little tricky to understand all the various nuances of concurrency. This is where CodeGear is looking to help. By providing simple, easy to understand, tools and libraries we can help bring multi-core programming out of the realm of voodoo and black magic and into the hands of developers of all skill levels. This will involve providing both library and compiler/tool support.

Friday, September 14, 2007

Cleggy joins the CodeGear team

The ever-present David Clegg, has just joined the CodeGear crew working on the Developer Network! Welcome to the crucible (it does get a little "hot" around here at times! :-)  Be sure to stop by his blog and say, "Hi."

Thursday, September 13, 2007

A Generic Playground, an intro to parameterized types in Delphi for .NET

The other day, I presented an introduction to the new parameterized types language feature in the latest release of Delphi for .NET at the Developer Days event. First of all, I'm not what one would call a consummate expert in the realm of language generics. In fact a lot of it just makes my head hurt. For real hard-core, on the edge, looks at the kinds of things you can do with generics, you should be looking here or here. I'm sure you may recognize those guys that run those blogs.

So given that disclaimer, I'm going to start presenting some topics on using the new parameterized types in the Delphi language. Keep in mind that for this release this only applies to Delphi for .NET. Check out the roadmap for some guidance on when this will be coming to the Win32 side of the house. I'm going to approach this from the perspective that as a Delphi programmer, you've never used generics or parameterized type. You may not even know what they are, how to use them, and why. That's ok... Me neither. :-)

In many ways,"generic" programming takes the code reuse tenet of Object Oriented Programming (OOP) to a whole new level. When OOP was becoming more prominent throughout the '80s, a key advantage to the concept was this notion of being able to reuse and extend already written code in ways not considered by the original author. Instead of reusing code via inheritance and polymorphism (although parameterized types strongly leverage OOP), it is handled by separating the type of the data being operated on from the algorithm.

Suppose you've written a killer new sorting algorithm. Now you want to use that algorithm to sort a variety of different types of items. In one case you want to sort a simple list of integers. Other times you may want to sort some complex data structure based on a collating sequence unique to that data. Without "generics" you probably would have written this algorithm using traditional OOP polymorphism. So when you wanted to use your killer sort function, you'd have to create a unique descendant of the base class that implements the algorithm, override a few methods, and finally create an instance of this new type to use it.

Generics allow you to, in many cases, forgo the sometimes tedious, often mind-numbing, task of creating yet another descendant class just to sort that new data structure you just defined. With a parameterized type, you write your algorithm as if you were writing it for a specific data-type (yes, I know there is a little more you have to consider, this is the basic gist of it). So let's start with something simple.

So I find myself constantly needing to reverse the items in an array. I have this awesome algorithm that I keep writing over and over. Sometimes I need to reverse an array of integers and other times it's strings. So, how could I use parameterized types to only write this killer algorithm only once and then tell the compiler that I need a version to work with integers, or strings, or some other structure only when needed. Here's what I came up with:

TArrayReverser<T> = class
procedure ReverseIt(var A: array of T);

procedure TArrayReverser<T>.ReverseIt(var A: array of T);
I: Integer;
E: T;
for I := Low(A) to High(A) div 2 do
E := A[I];
A[I] := A[High(A) - I];
A[High(A) - I] := E;

Eat your heart out, Mr. Knuth ;-). Now I can actually use the above parameterized type anyplace that I want to have the order of an array reversed, regardless of what the element types and sizes are. I don't have to create a descendant class and override some methods. I just instantiate the above type using the type of the array elements as the parameter. For example:

I: Integer;
IntegerArray: array of Integer;
IntegerArrayReverser: TArrayReverser<Integer>;

SetLength(IntegerArray, 10);
for I := Low(IntegerArray) to High(IntegerArray) do
IntegerArray[I] := I * 10;
IntegerArrayReverser := TArrayReverser<Integer>.Create;
for I := Low(IntegerArray) to High(IntegerArray) do
Writeln('IntegerArray[', I, '] = ', IntegerArray[I]);

So there is a quick intro to using parameterized types in Delphi for .NET. I'm sure you can see some of the interesting things you can do with this new found power. Another interesting fact about using generics is that in many cases the code is actually more compact and faster than if you'd tried to make a general purpose class using traditional OOP techniques. This is because the body of the ReverseIt procedure is compiled (at runtime in .NET, at compile-time for Win32) as if 'T' were actually declared as an 'Integer'. We could take the class I wrote above and use it to reverse an array of TPoint or TRect structures. Any type you declare or an intrinsic language type can be used to create a concrete version of TArrayReverser. If you have any questions about this pose, please post them in a comment.

Wednesday, September 12, 2007

Oh bugger! Comments are being moderated...

I know a lot of you have posted comments and then sent me email directly since your comment didn't show up... well it turns out that I'm still learning this new blog engine and comments are all moderated by default. I also noticed that the spam has already started :-(. So for now, I'll leave the moderation on. I'll try to moderate as often as I can. I'll also review all the commenting options and see if there are any settings that would allow comments to go unmoderated...

UPDATE:  Ok, I think I see what is going on.  Most comments will get posted immediately.  However if your post has 2 or more links in it, it goes into the moderation queue.  So if you post with a link to your blog and a link in the body of the comment, it gets moderated.  I've upped this to 3 or more links (may bump this up if I see more "normal" posts getting moderated).  This makes sure those silly SEO folks who like to post comments with a huge list of links will get moderated.  Nice feature!  Oh, and if you use profanity, it goes into moderation... I felt a little strange typing in all those words you're mother would wash your mouth out with soap if you ever said them.

Tuesday, September 11, 2007

IDE Autosave and recovery

I mentioned it in this post, which got me thinking about a new RAD Studio IDE feature; autosave. So what do you think, is some sort of time-based autosave feature something nice to have in a development tool?

Let's spec this thing out. I think you should be able to configure the idle delay, the "modified" state isn't cleared, and the original file isn't overwritten (nor are the __history backups cycled). So if you're working on MyUnit.pas, and it autosaves to a file called MyUnit.pas.autosave. Once you explicitly save the file, the MyUnit.pas.autosave file is deleted. Of course the normal __history backups are cycled and the working file is overwritten. So if certain random sub-atomic particles crash into your memory and cause a meltdown, you at least can recover the autosaves. When the IDE opens a file, it should detect that an orphaned autosave file is out there when you open a file and will ask if you want to recover the autosave by overwriting main working file. You should even be allowed to view the differences before you make that decision. How's that for really designing something on the fly? Anything else I missed? The compare tool should be pluggable/configurable, of course. Maybe allow you to enable this only for only certain files and file types?

New blogging server and engine...

So this is officially my first post on the live version of the Wordpress based blog-server. Lots of really cool new toys and features to play with. I'll miss some things like customizations and personal themes, but I'm promised that it's coming. No longer will I have to upload an image to my homepages.borland.com account and then reference them from my blog. This one allows me to upload attachments and images then inject the link. It even does thumbnailing for me.

OK, OK... so this stuff is expected these days for a blog engine... but after using, ancient by today's internet time, .Text, this is a welcome relief. Of course all of this won't make me any better of a blogger... I'll still suck as bad as I ever did as far as content and topics. :-) Oh and I just noticed.. the builtin post editor autosaves after a timeout... This sounds like an interesting IDE feature...

Friday, September 7, 2007

Steve Shaughnessy on Blackfish SQL and dbExpress4.

If you're doing any kind of database work with Delphi and/or C++Builder (I'd say that a very large number of you are), then you have got to start reading Steve Shaughnessy's blog. Yesterday he posted an excellent introduction to the new Blackfish SQL database with some interesing information about its lineage. Then today there is some great information on the new rich metadata support in the latest version of dbExpress coming in RAD Studio 2007. Be sure to post some comments on his blog and encourage him to keep posting more information!

Wednesday, September 5, 2007

Just in case...RAD Studio 2007

Just in case you missed all the other CodeGear bloggers, the press release, and the article on the CodeGear Developer Network... I'll go ahead an mention it here; CodeGear has announced RAD Studio 2007. If you've been waiting for the next studio release to upgrade, now is the time to do it. This release will also include Update 3 for the previously released Delphi and C++Builder 2007. If you already have Studio Level Software Assurance, you'll be getting this release automatically. If you still haven't signed up for SA, now is the time to do it. By maintaining your annual SA account, you are assured that you'll have all the latest versions and features available.

While RAD Studio 2007 includes many updates for the Delphi and C++Builder 2007 personalities, the main addition is the Delphi for .NET personality. This includes the new language feature, parameterized types (generics). This allows you to both create and consume generic data types as supported by the latest .NET 2.0 framework. And updated version of VCL for .NET is included with all the same level of support for Windows Vista. You can also create ASP.NET web applications.

Ok.. the commercial is over... :-)