Monday, April 3, 2006

Getting VCL/.NET ready for .NET 2.0 and 64bit CLR

Changes, my friends, are afoot.  While waiting for the Delphi/.NET compiler (dccil) to embrace all the latest goodies in the CLR world (partial types, generics, nullable types, anonymous methods, etc...) other engineers have been working on a lot of things that are independent of all the new compiler goodness.  The biggest change that is going to happen to VCL/.NET is the change from using Integer for handles (ie. THandle = Integer;), it must now be changes to IntPtr.  One common misconception (and one I admit to having fallen prey to as well) is that somehow IntPtr is some kind of “Pointer” type represented as an Integer.  Well, call me foolish and gullible, but I'm going to go out on a limb here and say that “IntPtr” is not what I'd call a very good name.  OK, OK... it basically sucks as a name.  I can see why at first glance one would be led to the misconception that “IntPtr” is a “Pointer.”  So what is this odd bit of identifier madness?  In actual fact (and to the more intelligent and astute among you, this is all mindless drivel), it really means this “Integer whose physical size and precision matches the physical size of a Pointer on the currently running underlying platform for which the “Just-In-Time” compiler is actually generating code” (pay no attention to the nested quotes in that statement).

What does this mean?  Well... if we've done our job correctly, this should end up being a complete non-sequitor.  However, as I dust off my trusty crystal ball (which can only look into the past, BTW), I would say that in reality, some of you will be affected.  Most noteably, those among you, that find yourself snuggling up a little closer to the Windows API.  This would, generally, be those of you who tend to dabble in the white arts that many call “component development.”  With a few exceptions, as I've outlined, most of the magic will be handled by the judicious and careful implementation of that wonderful bit of class and scope enhancing compiler construct called Helper Classes.  By creating a rich helper for IntPtr that includes overloaded operators, explicit and implicit conversion operators, and other bits of fun, we can now...hopefully...keep the vast majority of existing code out there compiling and correctly functional.  We've already got most of this work done, and now the task at hand is to scour the VCL sources for the rare cases where we have some unintended implicit conversions taking place.  Seppy and I talked about this just this morning and he's working through some interesting cases where the implicit conversions are getting into an infinite loop.  Should be simple to fix.

The biggest thing that may affect folks is if they've declared their own DLL imports and are using Integer for handles.  So start using THandle and things should go much smoother (at least until I can get even more information about our own progress).  Other cases are in the Message crackers where the LPARAM is now an IntPtr since that param will now grow to be an Int64 on 64bit platforms.  Much of this work was scoped for the DeXter release (BDS2006), however since Microsoft never released a version of .NET 1.1 that works in 64bit platforms it seemed to be busy work for little benefit.  We decided to concentrate on more immediately beneficial things... like fixing bugs.  Now that we've turned our resources to bear on the .NET 2.0 platform, we've been dusting off some of this preliminary work (we did get some critial issues fixed in the compiler for DeXter that helps in this effort) and are now in full swing.

On a final note, let's all take a moment of silence for the loss of all that wonderful work late last Saturday.  The good thing is that we've been able to stop David I's machine from calling NORAD. All the black helicopters have left along with all those men in black suits.

9 comments:

  1. huh, I would have thought that THandle would have made an excellent primitive all unto itself, and has the benefit of clear naming.


    I understand the desire to not polute the namespace with a trillion reserved words, but still, clarity has to become an issue at some point, no? I mean, we aren't talking about C here after all.

    ReplyDelete
  2. But Johnson, in C, the HANDLE type _is_ a pointer, so if Delphi's THandle type were a primitive, it would end up doing nothing more than mirroring the IntPtr type anyway.


    IntPtr has more uses than just being an underlying type for THandle, though. Allen mentioned the fourth parameter to SendMessage, LParam. For some messages, it just acts like an integer. For other messages, it's a pointer. I see plenty of code in which people type-cast a pointer value to Integer when they call SendMessage, but if SizeOf(Integer) <> SizeOf(IntPtr), then all that code will break someday. (Personally, I've been type-casting to the LParam type instead.)


    IntPtr is indeed a strange name, but it has the advantage of being similar to the name Microsoft has been using in its API declarations for several years, INT_PTR.


    I don't envy the ones who have to make the changes in the VCL code. This doesn't sound like it's the kind of change where you can just change a declaration, try to recompile, and let the compiler tell you all the places that are wrong.

    ReplyDelete
  3. Thanks for keeping us updated, Allen!


    IMO, it was a pity that you didn't introduce a builtin (defined in the System unit) IntPtr for the Win32 side of Delphi 2006. The .NET side already have IntPtr defined in the .NET System namaepace, of course.


    With IntPtr in place, people could start to convert their code today. And the compiler should be extended to warn against all casts between pointer and integer types (except IntPtr). This will make it easier to spot problematic issues. I think it would also be useful to warn against all *implicit* conversion between integer and IntPtr.


    In the mean time, people can (and probably should) define their own Win32 IntPtr something like this:


    {$IF CompilerVersion <= 18.0}

    type

    IntPtr =

    {$IFDEF Win32}

    Longint;

    {$ENDIF}

    {$IFDEF Win64}

    Int64;

    {$ENDIF}

    [$IFEND}


    I assume that a native Win64 Delphi will eventually introduce a IntPtr too?


    I posted about this earlier:

    http://groups.google.com/groups?rls=GGLD,GGLD:2005-01,GGLD:en&q=hallvard%20vassbotn%20intptr&sa=N&tab=wg

    ReplyDelete
  4. Hi Allen,


    That's a great idea to inform the community about future changes in the compiler. Lots of code has to be updated to be compliant with these new rules.

    Having some informations about integer types in Delphi 64 bit would be highly appreciated.


    Thanks in advance.

    ReplyDelete
  5. I am not so happy about this new IntPtr name, it indeed sound like a pointer to an integer to me, I would rather prefer PtrInt. Nevertheless I see the point in following the naming from MS INT_PTR, if it really have to be.


    When you indeed are working on integer type names, please open QC3628/QC1644 and do a Delphi official definition of int16, int32, int64 etc.


    Sure I can define them myself, but if Delphi set the name style it will be the same whereever we go. This is contrary to the C world where e.g. boolean has ten different namings, and there are several different ways to name 64 bit int - a mess.


    When you work on embedded systems you can work on anything from 8 to 32 bit processors, most often in C. The naming of integers is such a mess.


    Take for example the C short int, how big is that? Half the size of an int? Half the size of a long? Was this module intended for a 16 bit CPU or a 32 bit CPU? Then everybody defines their own set of nameings, some

    variations are: uint16, word16, u16, usint16 ... and add to these all variations for upper/lower case letters (case sensitiveness - yack).


    Before Delphi end up in a similar naming quakemire for Delphi 32/64 bit, I would much prefer that Delphi would take the clear way and define one set for Delphi with bit sizes in the naming for all sizes. Then we can use integer when size does not really matter and int32 when size is important. Even as byte and word is well defined I would prefer that they also make uint8 and uint16 for these.

    ReplyDelete
  6. Rob -> In C, everything is an integer and as such, EVERYTHING is anything else. To me, a handle is a handle, and I could care less how the implementing code handles it. If the platform defines it as 16 bits, 32 bits, 64 bits, I just don't care. It could store text, integers, pointers, I don't care. That is the point. I treat it as an ambiguous blob of data provided by the OS to be returned to the OS, like a cookie in browsers. I call something to get a resource, open a file, I get a handle. I work on it by passing back that handle, and I return it to the OS by closing the handle.


    The fact that it is 32 bits lets me do dirty tricks like cast it into other 32 bit entities, such as the LPARAM of most calls, however. It doesn't ultimately matter if you call it THandle, or ptrint, or fruitcake64 for what it contains, but it doesn't clearly communicate what it does in most cases. THandle clearly communicates that it is a handle. No ambiguity, and no need for me to understand the mechanics. If I try to cast it from a 64 bit entity to a 32 bit holder, I would expect the compiler to tell me that I was being stupid, protecting me from the worst of my dirty tricks as I move forward (such as when we all stuff things into "tag" through casting)


    However, the fact that LPARAM is going to be a different size on win64 has no real bearing on THandle, they don't connect, except where we create artificial connections through casting. LPARAM is lPARAM, not Thandle. It(LPARAM) is the system long, and represents a number. I know its a number, it is how it works. If I choose to stuff an equivalent sized piece of data into it, I should have to cast, because it may not always BE an equivalent size, and it would at least give the compiler a chance to catch it and warn me.


    So I could easily see THandle being a good primitive, LPARAM could stay LongInt, which can be 64 bit as needed, 128 bit eventually even, and it doesn't need a unique primitive as such.


    At least, that is how I see it.

    ReplyDelete
  7. How is the work on the real-world-useful 64bit Delphi compiler coming along (the native one)?


    We're getting quite a bit of pressure here to migrate to Visual Studio, and all this talk of .Net 2.0 and 64bit CLR support in Delphi is being perceived by management as signaling that Delphi is a dead horse, desperately trying to play catch up by forfeiting their own compiler technology and piggybacking MS compilers. In their eyes, Delphi is fast becoming an unnecessary (and risky) proprietary layer over MS's "real" compiler technology.


    Some news from the native 64bit Delphi compiler would certainly help, there has been almost nothing so far...

    ReplyDelete
  8. John,


    Native 64bit is on the roadmap and is coming. There are no *layers* over MS compiler technology. I'm not sure what you mean by a some layer. CLR is a platform that our compiler is targeting just like if we were to target another CPU. No piggybacking, but certainly leveraging the services provided by the environment just like we leverage the native Win32 APIs.


    We've forfeited nothing and are proceeding to execute against the published roadmap. Native 64bit is going to be a relevent platform. We are aware of that and have put it into our plans.


    Allen.

    ReplyDelete
  9. >CLR is a platform that our compiler is targeting

    >just like if we were to target another CPU.


    Well, the main issue is that they don't see the "CLR" as being a "target" comparable to a "CPU", but just as another Microsoft layer, somewhat like VB Runtimes in their time. CPUs for them are solid, stable, you-can-trust-it technology, and I've got to agree with them that CLR is far from being as solid or stable as CPUs right now.

    MS VB runtimes evoke all kind of troubles we had back in those days, when every MS bug (or fix) was a nightmare, with C++ as the only way around (until we found Delphi).


    Right now, the way of thinking goes like "if running Delphi code is going to completely depend on MS runtimes for its execution anyway, we should at least make sure to stick to standard languages"... We're desperately looking for something that'll allow us to say that in the future, Delphi will still not be needing C++ help to get around runtime bugs and quirks.


    Anything that would show Delphi is still being aimed at competing with not just VB.Net or C#, but also C++, is what we're looking for... If you could demonstrate that future Delphi will do almost all things C++ does but C# and VB.Net cannot, that would give us much better arguments.

    ReplyDelete

Please keep your comments related to the post on which you are commenting. No spam, personal attacks, or general nastiness. I will be watching and will delete comments I find irrelevant, offensive and unnecessary.