Saturday, February 24, 2007

How to add a "published" property without breaking DCU compatibility

First off, if you are an OOP or framework purist and have a weak stomach, you can stop reading now.  In the upcoming Delphi 2007 for Win32, we took an unprecidented approach to adding functionality to the VCL.  If you've used Delphi for more than a couple of versions, you can fully appreciate that when the VCL framework changes, you have to also get updates to your third-party components.  Now I'm not going to discuss the reasons why, since I've covered this in detail over the years as to why this happens.  Basically it boils down to having the flexibility to make changes to existing classes and the compiler itself as we see fit to provide the best overall experience that we can.  So, for all of you who will make the knee-jerk comments about “why don't you just do this for every release!”  Had we done that, you'd never have packages, interfaces, class variables, inline functions, and in a post-Delphi 2007 release, generics.  Just browse through some of my past posts for more details.

With that said, we decided to take a slightly different approach for Delphi 2007.  D2007, is being touted as a “non-breaking” release.  What exactly does this mean?  In a nutshell, it simply means that you should be able to take most of your components for BDS2006 and install them into D2007.  I say most because there will, of course, be some fringe cases that this isn't possible.  For example, if you have a component package that likes to reach around the existing design-time interfaces and muck with things behind the scenes, that is out of the scope of the non-breaking change.  As long as your existing component design-time package sticks with the existing published interfaces it should work without a recompile.

Uh... so what's new then??  It is true that a lot of the changes we've made have been in the “implementation” section only and so there is no DCU breakage there.  But there are also quite a few changes that manifest as either whole new components or in one case a brand-new published property on TForm!  The rules for type compatibility with the Delphi compiler is deceptively simple.  Basically, for any structured type (class, record, object, etc...), the “version” of that type or symbol is derived from all the embedded symbols and types.  So you cannot modify an existing class type or method declaration without causing this “version” to change.  This is the cause of the error message “xxx” was compiled with a different version of “yyy.”

For the astute among you that have been following along so far, you are probably already jumping ahead here and thinking.. of course, class helpers!  Well, yes, you'd be right on that count, to a point.  I did say that the new property is “published” which means it also shows up in the Object Inspector in the IDE.  So there is a little more gong on than just a class helper.  As it turns out, using a class helper will make this new property appear from the user's source code to be on  TCustomForm, which is the desired effect.  So we just added a property without actually touching the original TCustomForm declaration.  Then there is the streaming and the storage of the property in the instance.  This is where you should avert your eyes if you're a little queasy ;-).  We simply take and overload an existing, little-used, private field on the TCustomForm instance and create an internal (implementation section only) structure that has a field of the type of this overloaded field to now hold its value along with whatever fields are going to store the class helper's property data.  Then we go through all the places in Forms.pas that access this field and to some casting tricks to use that field as a pointer to that structre.  Since it is a private field, there is no effect of any descendants outside the Forms.pas unit.  This is actually a place where the OOP tennet of encapsulation helped us acheive this.  We were able to make this change and no code or classes that use an instance of this class are any more the wiser.

That takes care of the storage of the data behind the scenes. Now how about streaming this data since I did actually say that this was suppose to be a “published” property?  This is someplace where some past changes to the framework helped us.  TCustomForm had already overridden the DefineProperties method, so now it was a simple matter of adding some more code that method to handle the streaming of the property data.  This is also where the class helper came in handy because we could put the read and write methods on it.  Again, without changing the TCustomForm class.  So now we have a new property on the TCustomForm class, the streaming is all hooked up and you can now access this property on any TCustomForm or decendant instance.  The final piece now is how to get this property to show up in the Object Inspector.

This is where the ISelectionPropertyFilter interface comes in.  You may or may not have seen this little one-method interface that was introduced in BDS2006 in the DesignIntf unit.  For those of you with BDS2006, you can open DesignIntf.pas and read the usage comments there about where and how to implement this interface.

  { ISelectionPropertyFilter
    This optional interface is implemented on the same class that implements
    ISelectionEditor.  If this interface is implemented, when the property list
    is constructed for a given selection, it is also passed through all the various
    implementations of this interface on the selected selection editors.  From here
    the list of properties can be modified to add or remove properties from the list.
    If properties are added, then it is the responsibility of the implementor to
    properly construct an appropriate implementation of the IProperty interface.
    Since an added "property" will typically *not* be available via the normal RTTI
    mechanisms, it is the implementor's responsibility to make sure that the property
    editor overrides those methods that would normally access the RTTI for the
    selected objects.

        Once the list of properties has been gathered and before they are sent to the
        Object Inspector, this method is called with the list of properties.  You may
        manupulate this list in any way you see fit, however, remember that another
        selection editor *may* have already modified the list.  You are not guaranteed
        to have the original list.

  ISelectionPropertyFilter = interface
    procedure FilterProperties(const ASelection: IDesignerSelections;
      const ASelectionProperties: IInterfaceList);

So the basic trick is to register a selection editor for TCustomForm that implements this inteface.  When FilterProperties is called, we can now completely manipulate the whole list of IProperty interfaces in the ASelectionProperties argument.  You can add and remove properties.  For instance, if you drop a TFlowPanel on a form, then drop components onto it, you'll notice that the embedded components don't have a Left and a Top property.  Not because the component doesn't have them, but because they are useless at design-time so they get removed from the ASelectionProperties list.  In the TCustomForm case we actually want to add a property.  So we search through the list of properties, find the insertion point, instantiate a class that implements IProperty.  Now your “published” property shows up in the Object Inspector!

While this all seems like a lot of work to go through just to maintain compatibility, the good news is that in a post Delphi 2007 release we can fold these changes back into the TCustomForm class, delete the class helper and the ISelectionPropertyFilter, and the existing DFM files will work just fine.  So will all your source code that accesses these new properties.  One interesting thing is that this proof-of-concept could lead to being able to add some significant functionality into interim releases without breaking existing components/DCUs.  It could even be that every other release is a breaking release.  Just some things for us to consider for the future.

Friday, February 23, 2007

CEOs, conversations, and logos

Ok.. that's three blog posts in a week for our CEO Ben Smith.  As I was leaving work this evening I saw that he was still in his office so I complemented him about having yet another blog post.  He commented that it was actually cathartic and therapeutic.  I have to agree.  Sounds like he's really gotten into the the swing of things.  Could it be that he's got the blogging “bug?“  It is excellent that we have a CEO actively participating in the community and the conversation we're having!

As we were talking Nick stopped by and we started talking about selling CodeGear branded schwag (shirts, mugs, pens, etc...).  The problem is that we needed to get the final artwork for the new CodeGear logo.  Yep, we're working on a lot of changes both visually in terms of the logo, website, and product art work, and in terms of making it easier for our customers to buy from us.  We're streamlining a lot of processes...

Anyway, then Ben asked if we'd seen the latest artwork.  A couple weeks ago, I'd been in a presentation where several mockups had been presented.  After some feedback and some refinements made, the final logo had been selected.  It's interesting all the thought and consideration that goes into selecting a logo.  Anyway, all of a sudden we were on a mission to locate Lisa Romano, our head of marketing (yeah, remember that?  m-a-r-k-e-t-i-n-g).

Went down two floors to her office.  Door open, but no Lisa.  Just then we saw her emerge from Daivd I's office just down the hall carrying a large envelope.  Apparently she'd been showing the latest artwork to David before he leaves on a user-group tour.  So we got sneak peak.  All I can say is that there's some really cool things in the works, not only with the products, but in lots of other areas.

Wednesday, February 21, 2007

Something just froze over...

Yep.  It actually happened.  CodeGear's CEO, Ben Smith, is now officially blogging with his initial post here.  Yes, there was some fear and trepidation in the halls when he started asking around about how to post to the blog server.  I guess we shouldn't have been too concerned since he is a rather tech saavy guy.  He walks around with his little IBM laptop permanently attached.  I regularly get IM's from him since he communicates heavily via that medium.  I know he regularly reads a lot of blogs and other online sources (I get comments regularly about my and other folks blogs from him).  Oh, there he is on IM again... :-).  So if you haven't already, feel free to welcome him into the community and post some comments on his blog.  Be sure to encourage him to continue blogging and communicate regularly.

Tuesday, February 20, 2007

Announcements! Announcements!

It's an exciting day here at CodeGear!  The following two announcements just hit the wire this morning:

Delphi 2007 for Win32 and Delphi for PHP

The first product is a direct result of survey results, direct customer feedback, and input from various other sources.  The second product is our first break-out product for CodeGear.  In the following weeks, you'll begin to hear some more information about what new an interesting things we're putting into these products. One little tidbit is from Steve Shaughnessy with his post about some new developments in the database area.

Something that is pretty exciting about Delphi 2007 is the integration of MSBuild.  There is a whole cottage industry springing up around MSBuild that is providing a myriad of different tools for all kinds of interesting things.  Nick Hodges has a few screenshots over here of some the enhancements related to MSBuild, including named configurations.

Another first for Delphi (and in this release only) is that we've done a lot of work to ensure backward compatibility with Delphi 2006.  This means that nearly all your third-party components for Delphi 2006 will work seemlessly with Delphi 2007.  It is a compatible version of VCL with a lot of enhancements for better supporting Windows Vista including full control over the “glass” frame of top-level forms.  The IDE has also received a lot of performance improvements.  The general feeling from our field-testers is that the IDE is considered as fast as Delphi 7.  We've also done some unprecedented things with our field-test.  For instance, Nick posted to his blog an open invitation to join the field-test.  I don't know if we're still allowing folks in, but it couldn't hurt to ask.

I'm hopeful that these new developments will be begin to prove that we are serious about developers and developer tools here at CodeGear.  We aren't through yet... it should be interesting to hear what we have in store for the rest of this year.

Monday, February 12, 2007

InstallAware... finally an installer tool I can appreciate!

I vaguely hinted at this way back in October 2006 with this post talking about how installers are important.  Well, I can finally come out an actually say something about it.  In case you missed it, InstallAware issued this press release last month.  Over the course of the past year besides being neck deep in all the DevCo/Developer Tools Group/CodeGear happenings, I've also been actively working on completely revamping and re-architecting our whole product delivery, localization and install processes.  Many of these processes were somewhat “dated” and overworked.  Some of these consisted of an indecipherable complex “Rube Goldberg” machines.  Some of which which are still in the process of being overhauled.  All of this is in keeping with one of the CodeGear CEO, Ben Smith's, mantra of “Simplifying the business” at all levels.

At the middle of last last year I personally took on the challenge of completing the initial “proof of concept” research project done by a former Delphi install engineer serving as a temporary contractor.  This research project was to ensure that InstallAware was going to be suitable for our purposes.  On paper and in all the demos given by InstallAware CEO, Sinan Karaca we were very encouraged and excited.  IA offered a flexible scripting mechanism, full MSI support, easy plugin development (although due to the scripting we're doing minimal work here), built-in pre-req integration, auto-update, automatic web download deployment, and a host of other “21st century” features.  So we embarked on a small project to create a proof of concept.  Throughout this process, we received impecible support from Sinan.  As this project was winding down, I was looking at the results and it was clear that some of our upstream processes really needed some work before we'd be able to fully realized the potential of IA.

This is where I got heavily involved.  I needed a new challenge and a little bit of a distraction from the divestiture/spin-off process.  I also wanted to continue to be contributing in a meaningful way to the [B]*DS team and the product.  Since we'd faced some unfortunate attrition within the Integration team, I jumped in to fill in the gap.  Within the next couple of months, I plan on creating a case study for InstallAware outlining in further detail some of the tools and process we've created to work with IA.

I must say that it is has been and continues to be a pleasure working with Sinan, and InstallAware.  He's been very professional and highly responsive.  So, if you haven't taken a look at InstallAware, you should.  Oh!  And did I mention that it is built with Delphi!?

Friday, February 2, 2007

Flicker crusader

Now this post is going to show up in any search on Google for Flickr... Oh well.  I was merely going to suggest you take a look at Steve Trefethen's recent posts about his recent crusade to rid the [B]DS IDE of all flicker.  Some of the things he's recently done have required fixes to the VCL, but a lot of it is merely better management of the existing VCL through correctly setting properties that you can take advantage of now.  He's even posted a small patch you can make to your own copy of the VCL sources.

We're currently working on doing some changes to VCL to better support Windows Vista. With the glass frames, task dialogs, newer file dialogs, etc... At the same time this has also been an opportunity for us to revisit a lot of the core rendering and painting code throughout VCL.  In the coming weeks, some more information about this will begin to emerge... maybe even a screen shot or two ;-).