Various other links:
Letter to our customers, partners, and fans from Ben Smith, CEO CodeGear.
Borland Spins Off Its Tools Unit.
Borland Launches CodeGear to Supply Developers with Tools of the Trade.
And you thought it'd never happen! :-) However, as of today, I'm officially a CodeGear employee. CodeGear is the new developer company that is born of Borland. We're about, for, and by developers. In the coming days and weeks, we'll be talking more what we're about, where we're going to take this new venture and how we're going to get there. Delphi, C++Builder, JBuilder, InterBase are all the core products around which this company will be built. But that isn't all we're about. Be sure to stay tuned as we roll out more and more information. You can read the press release here. Also you can Digg this link as well. Full disclosure, I did submit the story to Digg.
There were quite a few interesting comments made to this post from the other day that seemed to indicate that there is a little confusion out there regarding exceptions. More specifically, the try...finally construct.
I've seen some rather interesting twists on the use of try...finally that have made me pause and wonder why it was done that way. When programming with exceptions, you always have to be aware that at nearly any point and without warning, control could be whisked away to some far-off place. In many ways you have to approach and think about the problem very similarly to how you would in a multi-threaded scenario. In either case, you always have to keep in the back of your mind two (or more in the multi-threaded case) potential code paths. The first one is easy since that is the order in which you're writing the code statements that define the overall logic and intent of your program. The other, and often forgotten code path is the exception execution path.
When an exception is raised (or "thrown" in the parlance of C++ and other similar languages) a lot of behind the scenes work is set into motion. I won't be going into the details of exactly what is going on since that tends to be platform and language dependant. I'll focus more on what happens from the programmers perspective and even more specifically the try...finally construct. One way to think of the try...finally block is that it is the programmer's way of "getting in the way" of that secondary code execution path. However, it is also unique in that it also "gets in the way" of the normal code execution path. In the grand scheme of things, the try...finally block is one of the most used (and often mis-used) block types for programming with exceptions. I'd venture to say that in the typical application the ratio of try...finally blocks to try...except blocks is on the order of 100:1. But why use them at all and what are they good for? It's all about resource management. Memory, files, handles, locks, etc... are all examples of the various resources your application uses and interacts with. The whole point of the try...finally block is to ensure an acquired resource is also properly handed back regardless of which execution path the application takes.
Let's look at some common misuses of the try...finally block and examine them more closely.
var
Obj: TMyClass;
begin
try
Obj := TMyClass.Create;
...
finally
Obj.Free;
end;
end;
There's a subtle problem here... Let's follow the normal execution flow first. Control enters the try block, an instance of TMyClass is allocated and the constructor is called, control returns and the local variable, Obj, is assigned. Some operations are done with the Obj instance (the "..."), then control enters the finally block and the Obj instance is freed. So far so good, right? I mean, the memory is allocated and freed and all is well with the heap along with any other resources needed by the TMyClass instance (assuming it is a properly designed class, that is).
Now let's look at the other possible flow of control. Control enters the try block, an instance of TMyClass is allocated and the constructor is called. Here is where something can go horribly wrong. When programming with exceptions, the possibilities are nearly endless as to what can happen during the call to the constructor. The most obvious is the case where the memory manager is unable to allocate enough space on the heap to hold the instance data for the new instance. An "Out of Memory" exception is raised. Hey, but that's OK because the finally block will get in the way of the the exception control flow, right? Yep, that's right. So memory was never allocated, the constructor was never called, local variable Obj was never assigned, and control is passed to the finally block which contains.... uh... Obj.Free; Do you see it now? Yep, that's right, the Obj reference was never properly set. Because of that, the call to Obj.Free; is not good for the health of your application. Chances are that another exception is going to be raised which will supercede the original exception (most likely a more fatal and nasty one).
So how do we fix this? Hey I know! What if we just made sure to pre-initialize the Obj reference to nil (Obj := nil;)? Sure. You could do that, but that is just adds another line of code to your function. How can we arrange the above code to ensure that every time control is passed to the finally block regardless of which path of execution is used to get there? It's actually very simple. Here's the same block with that subtle change:
var
Obj: TMyClass;
begin
Obj := TMyClass.Create;
try
...
finally
Obj.Free;
end;
end;
But now the construction of the TMyClass instance isn't protected! It doesn't have to be and let's examine why. From some of my previous posts regarding exception safety and the use of Assigned, we alluded to the fact that while the constructor of an object is executing, if any exception is raised the destructor will automatically be called, the object will be freed and the exception is allowed to continue on its merry way. There are essentially two main areas were things can go wrong. The first is during the actual allocation of the memory for the instance. If the memory manager is unable to find a block of free memory large enough to hold that instance, it will raise an "Out of Memory" exception. Since the instance was never actually allocated, there is no need to ever execute the Obj.Free; line of code. Since the try...finally block was never entered, Obj.Free will never be called. The other place where things can go wrong is in the object's constructor. In this case the memory manager allocated the memory and then control was passed off to the the constructor that would begin to setup the instance. If something fatal happened along the way there, we know from those past articles that the destructor will automatically be called and the instance memory handed back to the memory manager for later re-use. Since the object was already freed and, again, control never entered the try...finally block, the Obj.Free; line is never executed. So in both of those scenarios, there was no need to touch the local Obj variable reference since the resources associated with the TMyClass instance were taken care of. If the memory allocation succeeds, the constructor runs to completion, then control will return and the local variable Obj will be assigned after which control will then enter the try...finally block. It is only at this point that you always want to make sure the TMyClass instance referenced by Obj is freed regardless of what happens next.
There are a lot of other interesting "twists" on the usage of try...finally and try...except blocks that I've seen over the years. The above case stands out in my mind as the most common mistake made. I'll address some of the other cases in following posts. If you have any questions about certain common idioms and patterns that you're not sure are exception aware, post some comments and maybe I'll address them in future posts.
Back in late 1992 or 1993 or so, we had a dilemma. We wanted to add exceptions to the Turbo Pascal (what was soon to become the basis for Delphi's programming language). WindowsNT was under full-swing development. With this new OS came a new-fangled thingy called Structured Exception Handling (SEH). So what was the dilemma here? If you'll recall, the first release of Delphi was targeting Windows 3.1, aka. Win16. There was no OS-level support for SEH. Also, WindowsNT wasn't going to be released as a mass-market consumer-based OS. Again, what's the problem? Just add your own implementation of SEH to the language and move on. The problem was all about "safety." So we, of course, added our own specific implementation of SEH to the 16bit version of the Delphi language. Aren't exceptions suppose to make your code more safe? Ok, I'm being obtuse and a little evasive here.
The fundemental problem we faced was the notion of a partially constructed object. What if halfway through an object's constructor an exception was raised? How do you know how far into the execution of the constructor you got by the time the exception handler is executed and the object's destructor is called? The constructor is excuting merrily along, initializing fields, constructing other objects, allocating memory buffers, etc... Suddenly, BAM! One of those operations fail (can't open a file, bad pointer passed in as a constructor parameter, etc...). Since the compiler had already injected an implicit exception handler around the execution of the constructor, it catches the exception and promptly and dutifully calls the destructor. Once that is complete and the partially constructed object is destroyed and all the resources are freed, the exception is allowed to continue, in other words, is re-raised. The problem in this scenario is the fact that the destructor really has absolutely no clue why it got called (sure it could see that an exception is currently in play, but so what?). The destructor doesn't know if the instance was ever fully constructed and if not, how much got constructed.
The solution turned out to be remarkably simple and somewhat clever at the same time. Since all classes in Delphi have an associated virtual method table (VMT) each instance must be initialized to point to that table. Since Delphi classes allow you to make virtual method calls from within the constructor, that VMT pointer has to be initialized before the constructor is allowed to execute. If the VMT pointer has to be set before the constructor executes, why not just initialize the entire instance to some known state? This is exactly what is done. The entire instance is initialized to zero (0), the VMT pointer is set, if the object implements interfaces, those VMT pointers are also set. Because once the user's code in the object constructor begins to execute you know that the instance data is in a known state. By using this fact, the destructor can easily tell how far in the object's initialization sequence things got before the world went haywire. Remember yesterday's post where I mentioned the FreeAndNil procedure? Another item to note is the non-virtual TObject.Free method. Because you can assume that if an instance field contains a non-nil or non-zero value, it must have been successfully initialized, it should also be OK to de-initialize it. This is more important for any memory allocations or object constructions that took place in the constructor (or any other object method for that matter). The destructor has to know when a valid pointer is in that field. So a non-nil value means, go ahead and free the memory or destroy the object.
We realized too, that it would be very tedious and error-prone for the user to always have to remember to always do this pattern: if FField <> nil then FField.Destroy; Enter TObject.Free. If you opened System.pas and looked at the implementation of Free, it simply does if Self <> nil then Destroy; So you can safely call the Free on a nil, or unassigned instance. That is because the check is done within that method. All you need to do is FField.Free; and your destructor is now "exception safe." The same thing can be done for memory allocated with GetMem. You can safely call FreeMem(FField), even if FField is nil. It just returns. Finally, it should be noted that certain "managed types" such as strings, interfaces, dynamic arrays and variants are also automatically handled through some compiler generated meta-data. So just before an object instance's memory is freed, an RTL function is called that will take the instance and this meta-data table which contains field types and offsets so this RTL function knows how to free certain instance fields. Again, if a particular field is nil, it is simply passed over with no action needed.
What about the FreeAndNil thingy? For the astute among you, you've probably noticed that the implementation of that procedure actually sets the passed in reference to nil first and then destroys the instance. Shouldn't the name actually be NilAndFree? Yeah, probably. But it just doesn't roll of the tougue very well and is equally confusing. "So if you set the referene to nil first... how can you destroy it?" So why was it implemented in this manner? Exception safety is big one reason. Another significantly more obscure reason involves intimately linked objects. Suppose you have one object that holds a list of other objects which in-turn hold a reference back to the "owner" object? Depending on the order in which the objects get destroyed, they may need to notify other objects of their impending doom. Since the owner and the owned objects are intimately linked, they directly call methods on each other throughout their lifetime. However, during destruction, it could be very dangerous to willy-nilly call methods on the other instance while it is in the throws of death. By setting the instance pointer to nil before destroying the object, a simple nil-check can be employed to make sure no method calls are made while the other instance is being destroyed.
So there you have it; a few little tips on ensuring your objects are "exception safe" and a couple of hints into when you should use FreeAndNil. By peeking under the hood and examining the code, you can get a better understanding of why and how things are implemented. So, you could always use the if FField <> nil then FField.Destroy pattern buy why when calling FField.Free; does all the work for you? Using the pattern, if FField <> nil then FField.Free; is a redundant, as is, if Assigned(FField) then FField.Free;
There's a rather interesting discussion taking place in the news://forums.borland.com/borland.public.delphi.non-technical newsgroups about whether or not the Assigned is better than simply testing for nil. There's also some discussion in that same thread going on about FreeAndNil, which I can cover in another post. As for Assigned, I thought I'd shed some light on why it was even introduced. This is also a bit of a peek under the hood (bonnet for those of you across the pond) into some inner workings of the Delphi (BDS) IDE VCL designer.
'if PointerVar <> nil then' has been used to check if a pointer variable holds a useful value (assuming, of course it had been initialized to nil previously). This statement generates reasonably good machine code, is clear, and serves the intended purpose just fine. So why add "Assigned"? Starting with Delphi 1, we introduced to the language the notion of a "method pointer" or some call it a "closure" (although it is not *quite* what a "closure" in the pure sense actually is). A method pointer is simply a run-time binding consisting of a method and, this is very key, a specific instance of an object. It is interesting to note that the actual type of the object instance is arbitrary. The only type checking that needs to take place is to make sure the method signature matches that of the method pointer type. This is how Delphi achieves its delegation model.
OK, back to Assigned. The implementation of a method pointer for native code (Win16 and Win32, and the upcomming Win64 implementation), consists of two pointers. One points to the address of the method and the other to an object instance. It turns out that the simple if methodpointer <> nil then statement would check that both pointers were nil, which seems logical, right? And it is. However, there is a bit of a hitch here. At design-time, the native VCL form designer pulls a few tricks out of its hat. We needed a way to somehow assign a value to a component's method pointer instance, but also make sure the component didn't actually think anything was assigned to it and promptly try and call the method (BOOM!!). Enter, Assigned. By adding the standard function, Assigned, we could preserve the <> nil sematics, and also introduce a twist. It turns out, and for the adventurous among you can verify this, that Assigned only checks one of the two pointers in a method pointer. The other pointer is never tested. So what happens is that when you "assign" a method to the method pointer from within the IDE's Object Inspector, the designer is actually jamming an internally created index into the other (non-Assigned-tested) pointer within the method pointer structure. So as long as your component uses if Assigned(methodpointer) then before calling through the method pointer, your component code will never misbehave at design-time. So ever since Delphi 1, we've diligently drilled into everyone's head, "Use Assigned... Use Assigned... Use Assigned...." It think it worked. For component writers it is critical that Assigned be used when testing the value of published method pointers before calling through them. For all other cases, it is a little muddier and less defined. Personally, I still use the <> nil pattern all the time. Maybe it's because I'm "old skool", I don't know... I do, however, always use Assigned for testing method pointers, for the reasons given above.
I've seen some rather strange arguments as to why you should always use Assigned, even on normal instance references and pointers. One, that I found kinda strange was that "in the future, 'nil' may be defined as something other than a pointer value with all zeros (0)" If that were going to be the case, don't you think that the "value" of nil will also change to reflect that condition? Another factor is, why on earth would you ever need to use a non-zero valued nil? That, to me, should be one of those immutable laws of computing, like PChars (char*) are always terminated with a zero (0), nil should always be a zero (0) value. Now, I do realized that the Pascal definition of 'nil' doesn't necessarily mean zero (0), but it has been so convienient and consistent, that for any new platform or architecture that would want to change it would need to have an extremenly compelling reason to redefine it. At some point, I'll set the WABAC machine and we can all go back to investigate some other obscure factoid and try to dig up the the reasoning behind certain decisions and designs related to Delphi, VCL, and the IDE. For now, keep on using "Assigned(someinstance)" or "<> nil" with impunity... however for method pointers, it's if Assigned(methodpointer) then all the way!
Be sure to fill out the newly posted survey. Your feedback is highly valued and greatly appreciated. I would suggest you block out about 30-45 minutes to completely fill out this survey. I'd say that it's one of the more comprehensive surveys we've had in a while.
I use Firefox almost exclusively, and anytime I use IE is in the context of the Firefox plugin "IE Tab." I've also made sure the whole family is using Firefox as well, mainly because the number of nefarious exploits out there for Firefox are significantly less than for IE (that is slowly changing as FF gains popularity). Well I just installed FF 2.0 and so far it's been working great. The UI updates are very subtle, but I can tell the overall performance is definately better. Memory footprint is about the same, though.
I happened across this posting where the Internet Explorer team sent a cake to the Firefox team congratulating them on the release of FF 2.0. Of course Slashdot is filled with a myriad of conspiracy theories, negative spin, and outright paranoia... however my initial thought was how classy that was for the IE team to do that. The FF folks should be proud of their accomplishment and the fact that a competitive team from MS is recognizing the tireless efforts of the team working on an open source project, speaks volumes... about both teams.
After a long dry spell... you post twice in one day? No, that's not it.
After fighting with the product build, unit tests, and automated smoke tests for two days and you finally get the build working and start allowing commits to the repository... the first commit after all of that breaks the build again...Gahhh!! It's a good thing I was there to warn the culprit before the engineer who has spent the last two days (and nights) getting everything back in shape, came storming into this guy's office... armed to the teeth with one of the monster Nerf cannons laying around here. The hallways sure did light up for a few moments ;-).
I'm highlighting all of this because no matter what processes you put into place, and how many emails, wiki posts, meetings you have, ultimately it will boil down to simple human error. Just like the old adage, the more foolproof you make something, the more the world seems to hand you smarter fools :-). I know I've certainly been just as guilty as the next guy of breaking the build. Are there any of you out there with some great (infamous, funny, or just plain scary) stories of how against all odds, a member of your team managed to bypass all the safeguards and promptly broke the build? We all have stories of some "former team member" who did some of the craziest stuff, but what about your most embarrasing moments? These are the kinds of moments that, while infuriating at the time, you will look back on and find the shear humor and irony of the entire situation. What is your WTF story?
It's been a while since I've been posting to my blog, so I'd like to appologize for that... not that anyone is really out there hanging on my every word ;-). So this post is going to be a bit of a grab-bag of things.
The first and most obvious thing to comment on is actually something I cannot comment on so any comments I make should in no way be construed as commenting on anything that is worth commenting on at all. Huh??!! Right. Now that that is out of the way...
Duh! Of course they are important. How else can you deliver your product and ensure that it is placed properly on the end-user's system and is configured correctly? The problem is that many companies tend to think of installers as an afterthought. "We just built this great killer-app... now how do we deliver it?" Because of this tendency to push it off to late in the process, it is often short-changed. A flexible build, delivery and install process is key to making sure that your "integration" team does not become a bottleneck.
How you're going to deliver your product is something that needs to that needs to have a decent amount of up-front consideration. It can, and often does, affect how you actually architect and build the product itself. You have to ask questions like:
Nearly all of those questions above, and probably a few others I can't think of right now, are ones we have to ask ourselves all the time. "Integration" is the process of bringing together all the various bits and peices from other teams and external third-parties and "integrating" them into an installable image. For many years there have been a plethora of tools out there that serve to ease the construction of these installation images. However, not many (if any at all) actually help in streamlining the whole integration process. Not that they should, mind you, but there is often a lot more to delivering a product than simply pressing F9, grabbing the resulting executable, jamming it into an installer project, build the installer and burn a CD or upload to your shop site. For many folks those steps are actually all that is needed due to the limited complexity of the product. When you compare that to what we in the DTG group have to do, that is far from an adequate process.
Over the years we've developed quite a few internal tools (built with Delphi and sometimes InterBase of course!) that help make this process easier to maintain and to make it, and this is a critical point, repeatable. If you cannot ensure that each time the process is run that it goes through the same steps in the same order every single time, how can you be sure of anything in the output. For instance, no "offcial" build is ever delivered from a developer's machine. Official builds are always done on a single-purpose isolated system that is dedicated to that one and only task of building the product. Developers rarely, if ever, have direct access to this system. The "integration" team is responsible for the care and feeding of that system. That team is also responsible for maintaining what we call the "delivery database" which is exactly what it sounds like. It is a huge centralized database that describes which files get "delivered" where and what kind of processing needs to be done to them on the way. By extension this same database is also used to generate scripts that are used by the installer software to create an installation image.
I'll try and talk a little more about this whole process in some upcoming posts, but for now I'd like to hear from folks what kinds of processes you've setup to streamline and speed the delivery and building of installers? Also, what installers out there on the market are your favorites? We've used for several releases InstallShield from Macrovision, but there are many others and we're always evaluating new versions or new entries into the market. A few of the others we've looked at over the years is Wise, InstallAnywhere, and a new interesting entrant into the whole installer domain, InstallAware. There are many others; some free, some open source and other commercial offerings. What is interesting about InstallAware is that it is written entirely in Delphi! That's really cool!
Huh? What's that? It's a term used to describe a group of bicycle riders, usually in some kind of race, such as the Tour De France. Peloton is also the code name for the upcoming release of JBuilder. (What! This is a Delphi blog! Why are you talking about Java??!). Yeah, yeah, I know. However, since the formation of the Developer Tools Group here at Borland, I've had the privilege of working closer with the Peloton team here in Scotts Valley, CA. In many ways my role has expanded a bit and I now keep an eye on all the various projects going on in the DTG. Closely watching these other teams (this include the InterBase team) has been very enlightening.
The Peloton team had a daunting task ahead of them. They had to, essentially, recreate almost the whole JBuilder experience on top of the open source Eclipse platform! The face and character of the Java market is shifting dramatically and the move is on to open-source. The maturity and usefulness of all the various bits and pieces of open-source are finally reaching a critical mass. The problem, however, is that this new world order in the Java space doesn't really have "order." It's probably best characterized as being the "wild-west." The possibilities are nearly endless, but the dangers and pitfalls are many. There needs to be a trailblazer and a regional "marshal" to help establish some order and sanity to this new frontier. Take the Eclipse effort. Clearly the Eclipse IDE was first and foremost built to be a Java IDE. However, some of that is changing with the uptake in the adoption of the RCP (Rich Client Platform) for creating non-IDE type products. There is also the CDT (C/C++ Developer Toolkit), which is a cross platform C/C++ IDE tool.
However there is a bit of chaos. Enterprises don't like chaos. Even many small to medium developer shops cannot afford added chaos to their development teams. By chaos, I'm referring to the whole open nature of the Eclipse platform. This is not an argument against open source at all, but merely an observation of reality. We've been speaking with many of the current and past JBuilder customers; For the ones that have moved to Eclipse, they are finding that "free" is by no means "free!" There are even some companies that have some of their top talent dedicated to defining the official Eclipse platform and plug-ins the development teams are going to use. That is amazing to me! You now have companies where development tools are not their core business, and most of them, software isn't even their core business. They're banks, investment firms, hospitals, government agencies. They're not in the business of selling software let alone assembling or building development tools! Software is merely a necessary tool that the company uses to remain competitive in their respective markets. This is where Peloton (aka. JBuilder) will step in and fill the gap. By providing a certified and preassembled Eclipse environment, we can help many of these companies get back to focusing on what makes their businesses successful.
Peloton is not just a simple thin coating on top of the Eclipse you can go download for free right now, but it is also the evolution of the whole JBuilder product line itself! It is interesting to note that the first 3 versions of JBuilder were actually built on top of the Delphi IDE! (Hah!! There's your Delphi reference!!). Starting with version 4 (there was a 3.5 in there... but let's keep this simple), JBuilder shifted to what was called the PrimeTime platform, which was an all Java IDE platform. So from version 4 up to JBuilder 2006, it was based on the PrimeTime core. Now, we're just doing the same thing again and moving to a new IDE core again and this time we've chosen the Eclipse platform. There was significant effort and emphasis placed on making sure that your existing JBuilder projects can be easily and seamlessly carried over to this new version. The DTG has always been about making sure our customers were not left behind and are provided an avenue to the latest technologies. Just like Delphi and C++Builder, JBuilder has held to this equally as well. If you're at all interested in the upcoming release of JBuilder or have been using a older version, I would strongly suggest you look into purchasing JBuilder 2006 with Software Assurance. Everyone who is on SA when Peloton is released will get it as part of their SA agreement.
There has been some recent bruhaha surrounding the DTG's BDS published roadmap. I will say that we're currently in the process of review, re-alignment, adjustment and/or clarification of the currently publised roadmap. Industry landscapes can change, priorities can shift, so maintaining a stagnant roadmap is not in anyone's best interest. Likewise, radical departures from an existing publised roadmap are equally, if not more, damaging. "Balance" is the watchword of the day... Very soon we'll be announcing the opening of another online Delphi survey. Nick Hodges has been frantically gathering input about what to place on it from all those involved from the various teams. Changes are not made on a whim, but we are actively evaluating everything we're doing and where we're going. This is, in fact, a continuous process and something we've done for many, many years.
I just want to reiterate that the DTG continues to be filled with a flurry of activity. For instance, we've just hired a new head of marketing who is fully and solely dedicated to the DTG! Yeah, folks it's been a while on this one... We've also opened many positions on the documentation team. So if you're a technical writer, here is one of the positions to send in your resume' or CV. We also recently filled a position on the Delphi compiler team. There are some openings available on our integration team, so if you're experienced in writing installers, and would like to help define new, modern ways to install, update, and deliver Borland Developer Studio, be sure to send your resume here.