WWN Issue 296

This is the 296th issue of the Wine Weekly News publication and a special edition to look at the beta release of Wine. It also serves to inform you of what's going on around Wine. Wine is an open source implementation of the Windows API on top of X and Unix. Think of it as a Windows compatibility layer. Wine does not require Microsoft Windows, as it is a completely alternative implementation consisting of 100% Microsoft-free code, but it can optionally use native system DLLs if they are available. You can find more info at www.winehq.org

Beta: How We Got Here
With Wine hitting a beta release today, it's fitting to look back on the past few years of development and pick out some highlights. We have a pretty good history of Wine that provides a pretty good introduction to how Wine came to be. It also covers the ups and downs of Wine's commercial development, various conferences we've had, and Wine's licensing flamewars^H^H^H^H^H^H^H^H^H discussions. This special issue of WWN aims to take more of a technical look at Wine's internals and show you how things have progressed. Eric Pouech began writing the Wine Weekly News over 6 years ago. We have a mostly continuous and mostly weekly account of development since then. While WWN has strived to keep an accurate log, a lot of major projects have been undertaken that barely get mentioned in its pages. That includes things like font handling and the WinMM architecture. On the other hand, many areas show a natural progression over several months that have led to the architecture in Wine 0.9. We'll focus primarily on technical topics covered in WWN, but don't forget that has been brought to you by nearly 800 different contributors over its twelve year history. "Development will not stop with the BETA release. If anything, it will speed up as more programmers become familiar with the project. New BETA releases will be released periodically as Wine progresses." Bob Amstadt, August, 1994 (Linux Journal #4) In the beginning, Wine set out to run early Windows programs. A hefty infrastructure based on a DOS-style architecture emerged. Windowing was done by hooking into Tk. When Windows 95 arrived, a major shift in development began to take place to support the new 32-bit API's and applications. Microsoft seemed to add new API's every day throughout the late 1990's. The pace has certainly slowed down these days, but it changed the direction of Wine development. Most recently we made the switch from using Windows 98 as the default configuration to Windows 2000. That switch signaled a major shift for Wine away from the old DOS heritage to the NT architecture. Going all the way back to WWN #1 , there's two interesting threads. Ulrich Weigand discussed creating separate address spaces for applications. See, back in the beginning Wine used the same memory space for all applications. That model was based on the way Windows 95 worked. It's also the reason Windows 9x was so unstable - any application could stomp on the memory of another. As Ulrich explained, it wasn't exactly a simple task. The topic was discussed further a few weeks later in WWN #4 . Work actually began a few months later in early 2000. It continued through early 2000 and wrapped up in late May. As with many things in Wine's evolution it provided a necessary step forward at the initial expense of growing pains (and loss of functionality.) The second thread in that early issue discussed porting Wine. At the time, Patrik Stridvall outlined four different areas of portability: OS, CPU architectures, compilers, and separating out the graphics drivers. Although Wine's portability is now better than ever, most developers still use Linux on x86 machines along with the gcc compiler. Getting Wine running on other operating systems remains a struggle, although a small group of people continue to help keeping Wine working on FreeBSD and Solaris x86. There has been some success in getting Wine to work on Sparc and PowerPC machines, but since it's not possible to run x86 binaries on them most of the efforts have dissolved. The last item on Patrik's original list, separating out graphics drivers, has been successful. At the moment there's primarily one driver that gets used, winex11.drv , but it's possible to write a custom driver for something like Mac OS X and, in fact, work has been done in that area. It's interesting that two fundamental areas of development were discussed in that first issue. While address space separation is something now taken for granted, it's ironic that portability remains an issue for Wine. Porting is a never ending project that fosters discussion nearly every month on the development mailing list. A huge project that began around the same time as address separation was DLL separation. Wine's original DLLs weren't separated along the same lines as Windows. Wine's DLLs should only call exported functions of other DLLs. In other words, it should be possible to rip out one of Wine's builtin DLLs and drop in a native Windows one without any difference. If you look really hard you'll see a chronicle of how the work began. At the end of 2000 the infrastructure was in place to make the transition smoother. Worked progressed and by early 2003 some of the hardest bits were left. "the 1.0 release might even happen, but probably not before second quarter 2000." Marcus Meissner, November, 1999 Discussion of Wine's registry first appears in WWN #5 with suggestions for improvement. Back then, discussion occurred about whether to move the registry directly into the file system, whether to turn it into a binary format for performance reasons, or whether to leave it as an ASCII file. Well, it's still an ASCII file, though discussions still come up from time to time about moving it to a binary format. Ove Kåven wrote a feature article for WWN #48 describing Wine's registry. Much of that information remains true to this day and bits of that article remain in the Wine User Guide . Although it seems like Wine's registry code has been ultrastable for a long time, issues were still popping up in 2001 that caused problems. It took until June, 2003 for a regedit tool to appear, courtesy of the ReactOS project. Implementing the registry allowed for Wine's own configuration to be handled with it. In fact, one of the key things in the Wine beta release is the new configuration mechanisms. There's two key things that come into play here. First, Wine needed to have a sane set of defaults that allowed it to work out of the box. From there, a tool needed to be created that allowed those settings to get overridden. Back in WWN #52 there began some discussion about creating a graphical configuration tool (though TkWine already existed). Basically the initial idea was to have it serve as a way to edit the text-based config file. In early 2001 an effort to remove commandline options began. It was one of the first patches that made people question the direction Wine's configuration was heading. Looking back, it's clear that Wine is pretty easy to use now without a lot of different commandline options. In 2002 a large to-do list was created that contained the first concrete discussion of building a graphical configuration tool . That work began in early 2003 and continued on and on . For the most part the tool was complete in the middle of 2004. However, a lot of other work remained to be done, such as setting up some sane default parameters, and cutting over to it couldn't be done right away. This past June winecfg went live and the old config file fell by the wayside. Part of the configuration change dovetailed into DLL separation and led to a big development effort by Alexandre and Eric Pouech to rewrite the filesystem code . The project got off the ground around August, 2003 with some ideas and some initial code appeared shortly thereafter. For the most part it was a very smooth transition and by April, 2004 the new code was in place. "Given the amount of progress Wine has made over the past year, it seems (to me anyway) that the time may be appropriate to try for Wine version 1.0." Jeremy White, May 11th, 2000 One of the final big changes to go into Wine's beta has been a change in window management . The work loomed as rather mysterious and vague item on the to-do list for a long time and it was widely agreed Alexandre was the only one who had the background to tackle it. At the core lies the ability for interprocess window communication. Around December of last year the update region management was moved into the server. Optimizations over the next few months solidified this area of code and work mostly wrapped up by the beginning of February. The infrastructure around Wine has seen a lot of changes over the years. in September 2000 Wine replaced an older bug tracking system with Netscape's Bugzilla. Jeremy Newman has been hating it ever since. Around the same time, discussion began about overhauling the Application Database . By May of 2001, a new AppDB had been written by Newman. The code behind that was eventually released publicly and the Wine community stepped up with massive changes to make the AppDB what it is today. Putting together a test suite became a hot topic in early 2002. It's nearly doubled in size each year since then. We now have about 12,000 individual unit tests that run on both Windows and Wine. Besides "poking" Windows to find out how it behaves, the tests also serve to keep key areas of Wine from regressing. "I really feel like 2001 will be the year of Wine 1.0" Eric Pouech, January 1st, 2001 As Wine has matured, more and more higher-level libraries have began appearing. Things like TWAIN scanner support were implemented on top of native Linux libraries (in that case SANE.) The CryptoAPI popped up in 2001 as well as improvements to the Shell Lightweight API . Work on advanced controls, such as Listview and more recently RichEdit 2.0 , have quietly made a lot of things "just work". Huge development efforts, such as Microsoft Installer support, have been financed by CodeWeavers and undertaken by Mike McCormack and Aric Stewart . Libraries like these provide the guts behind Wine that Windows applications require. Of those libraries, DirectX is required by virtually every game on the market. Initial work to support early versions of DirectX (< v7) began in 1997 and predated WWN. A lot of that work was good enough to get games running. In WWN #2 , Lionel Ulmer began asking about implementing hardware accelerated OpenGL. By 2000 , Wine hooked into any ABI compliant OpenGL library allowing for hardware acceleration. Then in 2000 work stagnated when TransGaming formed and began creating a commercial Direct3D implementation. It wasn't until 2002 that Jason Edmeades inquired about supporting DirectX 8. Work really picked up and things really improved over the course of about a year. About a year ago ideas began kicking around about adding support for Direct3D 9. The decision was made to implement it using a new library with the goal of eventually switching Direct3D 8 and earlier versions over to it eventually. Oliver Stieber jumped in earlier this year to mention some of the work he had done on D3D9. Things have really taken off on it and you can expect a lot more DirectX improvements in the very near future. "Why exactly does an installer require such complex parts of OLE anyway? That seems like overkill for a self-extracting exe to me." Mike Hearn, January 21, 2003 DCOM is a four-letter word, namely one that's caused a lot of Wine developers to mutter some other four-letter words. Wine's DCOM implementation has seen quite a few revisions over the years. While the DirectX efforts serve to get a lot of games to run, almost all of the effort for DCOM has gone into supporting a single application: InstallShield. In September 2001 work began on extending DCOM to support InstallShield 6. Initially it took a month to get it working to some degree. Work on it continued, but sort of stalled by 2003. Some of the key DCOM developers started to realize a massive rewrite was required in order to make some of the harder, and crucial, bits work. In January of this year a new roadmap was posted and progressed rapidly with CodeWeavers staff spearheading the development. That DCOM rewrite, 6 months and 135 patches later, is now complete. Earlier we mentioned a lot of topics don't get nearly enough coverage here on the pages of WWN as they deserve. One of those areas is internationalization. A dedicated group has spent a lot of time not only making all sorts of programs from around the world work with Wine, but providing translations and documentation for the project. Back in WWN #25 we didn't have the luxury of a full Unicode implementation. Some of you, namely those living on the North American continent, might be surprised to learn ASCII just doesn't cut it for everyone in the world. In fact, there's a million character sets out there and supporting them is a tough task. Microsoft chose to do this using a special set of "Wide" functions that allowed Unicode character sets to be worked with. Wine, of course, needed to followed this same mechanism to implement the "W" counterpart to all of the ASCII functions. The patch discussed in WWN #25 added support for Cyrillic characters. Ove Kåven suggested looking at the environment variables in the user's environment, and in fact that's what eventually ended up happening. Even as recently as 2000 the discussion was going on about how to best support Unicode. It wasn't exactly clear (WWN #41 ) directly implementing the ASCII functions on top of Unicode was the best direction. Well, that mechanism proved to be the way to go. By 2002 work had progressed to the point where Shachar Shemesh started work on languages requiring bidi support. "We are now working on the last missing piece ... to complete the transition from a Win3.1 architecture to a NT-like architecture. Once this is done we'll be able to release 1.0. I won't give any dates though..." Alexandre Julliard, October 29th, 2001 So where are we headed? As we mentioned, DirectX support is going to continue to improve in the short term. Work on getting games to run will likely drive efforts to improve copy protection support. On the tools side of things, the new Wine Wiki has proven to be quite successful. In the more distant future, Mac OS X on Intel presents an interesting opportunity and could very well lead to very successful port. And, as always, at any time a major patch could arrive from out of the blue that makes Wine do something new. All this work, along with lots of bugfixes, will eventually get us to Wine 1.0. I'm not going to be the one to throw out any predictions of when that'll happen.

Beta: How We Got Here

Wine 0.9 With Wine hitting a beta release today, it's fitting to look back on the past few years of development and pick out some highlights. We have a pretty good history of Wine that provides a pretty good introduction to how Wine came to be. It also covers the ups and downs of Wine's commercial development, various conferences we've had, and Wine's licensing flamewars^H^H^H^H^H^H^H^H^H discussions. This special issue of WWN aims to take more of a technical look at Wine's internals and show you how things have progressed.

Eric Pouech began writing the Wine Weekly News over 6 years ago. We have a mostly continuous and mostly weekly account of development since then. While WWN has strived to keep an accurate log, a lot of major projects have been undertaken that barely get mentioned in its pages. That includes things like font handling and the WinMM architecture. On the other hand, many areas show a natural progression over several months that have led to the architecture in Wine 0.9. We'll focus primarily on technical topics covered in WWN, but don't forget that has been brought to you by nearly 800 different contributors over its twelve year history.

"Development will not stop with the BETA release. If anything, it will speed up as more programmers become familiar with the project. New BETA releases will be released periodically as Wine progresses."

Bob Amstadt, August, 1994 (Linux Journal #4)

In the beginning, Wine set out to run early Windows programs. A hefty infrastructure based on a DOS-style architecture emerged. Windowing was done by hooking into Tk. When Windows 95 arrived, a major shift in development began to take place to support the new 32-bit API's and applications. Microsoft seemed to add new API's every day throughout the late 1990's. The pace has certainly slowed down these days, but it changed the direction of Wine development. Most recently we made the switch from using Windows 98 as the default configuration to Windows 2000. That switch signaled a major shift for Wine away from the old DOS heritage to the NT architecture.

Going all the way back to WWN #1 , there's two interesting threads. Ulrich Weigand discussed creating separate address spaces for applications. See, back in the beginning Wine used the same memory space for all applications. That model was based on the way Windows 95 worked. It's also the reason Windows 9x was so unstable - any application could stomp on the memory of another. As Ulrich explained, it wasn't exactly a simple task. The topic was discussed further a few weeks later in WWN #4 . Work actually began a few months later in early 2000. It continued through early 2000 and wrapped up in late May. As with many things in Wine's evolution it provided a necessary step forward at the initial expense of growing pains (and loss of functionality.)

The second thread in that early issue discussed porting Wine. At the time, Patrik Stridvall outlined four different areas of portability: OS, CPU architectures, compilers, and separating out the graphics drivers. Although Wine's portability is now better than ever, most developers still use Linux on x86 machines along with the gcc compiler. Getting Wine running on other operating systems remains a struggle, although a small group of people continue to help keeping Wine working on FreeBSD and Solaris x86. There has been some success in getting Wine to work on Sparc and PowerPC machines, but since it's not possible to run x86 binaries on them most of the efforts have dissolved. The last item on Patrik's original list, separating out graphics drivers, has been successful. At the moment there's primarily one driver that gets used, winex11.drv , but it's possible to write a custom driver for something like Mac OS X and, in fact, work has been done in that area.

It's interesting that two fundamental areas of development were discussed in that first issue. While address space separation is something now taken for granted, it's ironic that portability remains an issue for Wine. Porting is a never ending project that fosters discussion nearly every month on the development mailing list.

A huge project that began around the same time as address separation was DLL separation. Wine's original DLLs weren't separated along the same lines as Windows. Wine's DLLs should only call exported functions of other DLLs. In other words, it should be possible to rip out one of Wine's builtin DLLs and drop in a native Windows one without any difference. If you look really hard you'll see a chronicle of how the work began. At the end of 2000 the infrastructure was in place to make the transition smoother. Worked progressed and by early 2003 some of the hardest bits were left.

"the 1.0 release might even happen, but probably not before second quarter 2000."

Marcus Meissner, November, 1999

Discussion of Wine's registry first appears in WWN #5 with suggestions for improvement. Back then, discussion occurred about whether to move the registry directly into the file system, whether to turn it into a binary format for performance reasons, or whether to leave it as an ASCII file. Well, it's still an ASCII file, though discussions still come up from time to time about moving it to a binary format. Ove Kåven wrote a feature article for WWN #48 describing Wine's registry. Much of that information remains true to this day and bits of that article remain in the Wine User Guide . Although it seems like Wine's registry code has been ultrastable for a long time, issues were still popping up in 2001 that caused problems. It took until June, 2003 for a regedit tool to appear, courtesy of the ReactOS project. Implementing the registry allowed for Wine's own configuration to be handled with it.

In fact, one of the key things in the Wine beta release is the new configuration mechanisms. There's two key things that come into play here. First, Wine needed to have a sane set of defaults that allowed it to work out of the box. From there, a tool needed to be created that allowed those settings to get overridden. Back in WWN #52 there began some discussion about creating a graphical configuration tool (though TkWine already existed). Basically the initial idea was to have it serve as a way to edit the text-based config file. In early 2001 an effort to remove commandline options began. It was one of the first patches that made people question the direction Wine's configuration was heading. Looking back, it's clear that Wine is pretty easy to use now without a lot of different commandline options. In 2002 a large to-do list was created that contained the first concrete discussion of building a graphical configuration tool . That work began in early 2003 and continued on and on . For the most part the tool was complete in the middle of 2004. However, a lot of other work remained to be done, such as setting up some sane default parameters, and cutting over to it couldn't be done right away. This past June winecfg went live and the old config file fell by the wayside.

Part of the configuration change dovetailed into DLL separation and led to a big development effort by Alexandre and Eric Pouech to rewrite the filesystem code . The project got off the ground around August, 2003 with some ideas and some initial code appeared shortly thereafter. For the most part it was a very smooth transition and by April, 2004 the new code was in place.

"Given the amount of progress Wine has made over the past year, it seems (to me anyway) that the time may be appropriate to try for Wine version 1.0."

Jeremy White, May 11th, 2000

One of the final big changes to go into Wine's beta has been a change in window management . The work loomed as rather mysterious and vague item on the to-do list for a long time and it was widely agreed Alexandre was the only one who had the background to tackle it. At the core lies the ability for interprocess window communication. Around December of last year the update region management was moved into the server. Optimizations over the next few months solidified this area of code and work mostly wrapped up by the beginning of February.

The infrastructure around Wine has seen a lot of changes over the years. in September 2000 Wine replaced an older bug tracking system with Netscape's Bugzilla. Jeremy Newman has been hating it ever since. Around the same time, discussion began about overhauling the Application Database . By May of 2001, a new AppDB had been written by Newman. The code behind that was eventually released publicly and the Wine community stepped up with massive changes to make the AppDB what it is today.

Putting together a test suite became a hot topic in early 2002. It's nearly doubled in size each year since then. We now have about 12,000 individual unit tests that run on both Windows and Wine. Besides "poking" Windows to find out how it behaves, the tests also serve to keep key areas of Wine from regressing.

"I really feel like 2001 will be the year of Wine 1.0"

Eric Pouech, January 1st, 2001

As Wine has matured, more and more higher-level libraries have began appearing. Things like TWAIN scanner support were implemented on top of native Linux libraries (in that case SANE.) The CryptoAPI popped up in 2001 as well as improvements to the Shell Lightweight API . Work on advanced controls, such as Listview and more recently RichEdit 2.0 , have quietly made a lot of things "just work". Huge development efforts, such as Microsoft Installer support, have been financed by CodeWeavers and undertaken by Mike McCormack and Aric Stewart . Libraries like these provide the guts behind Wine that Windows applications require.

Of those libraries, DirectX is required by virtually every game on the market. Initial work to support early versions of DirectX (< v7) began in 1997 and predated WWN. A lot of that work was good enough to get games running. In WWN #2 , Lionel Ulmer began asking about implementing hardware accelerated OpenGL. By 2000 , Wine hooked into any ABI compliant OpenGL library allowing for hardware acceleration. Then in 2000 work stagnated when TransGaming formed and began creating a commercial Direct3D implementation. It wasn't until 2002 that Jason Edmeades inquired about supporting DirectX 8. Work really picked up and things really improved over the course of about a year. About a year ago ideas began kicking around about adding support for Direct3D 9. The decision was made to implement it using a new library with the goal of eventually switching Direct3D 8 and earlier versions over to it eventually. Oliver Stieber jumped in earlier this year to mention some of the work he had done on D3D9. Things have really taken off on it and you can expect a lot more DirectX improvements in the very near future.

"Why exactly does an installer require such complex parts of OLE anyway? That seems like overkill for a self-extracting exe to me."

Mike Hearn, January 21, 2003

DCOM is a four-letter word, namely one that's caused a lot of Wine developers to mutter some other four-letter words. Wine's DCOM implementation has seen quite a few revisions over the years. While the DirectX efforts serve to get a lot of games to run, almost all of the effort for DCOM has gone into supporting a single application: InstallShield. In September 2001 work began on extending DCOM to support InstallShield 6. Initially it took a month to get it working to some degree. Work on it continued, but sort of stalled by 2003. Some of the key DCOM developers started to realize a massive rewrite was required in order to make some of the harder, and crucial, bits work. In January of this year a new roadmap was posted and progressed rapidly with CodeWeavers staff spearheading the development. That DCOM rewrite, 6 months and 135 patches later, is now complete.

Earlier we mentioned a lot of topics don't get nearly enough coverage here on the pages of WWN as they deserve. One of those areas is internationalization. A dedicated group has spent a lot of time not only making all sorts of programs from around the world work with Wine, but providing translations and documentation for the project.

Back in WWN #25 we didn't have the luxury of a full Unicode implementation. Some of you, namely those living on the North American continent, might be surprised to learn ASCII just doesn't cut it for everyone in the world. In fact, there's a million character sets out there and supporting them is a tough task. Microsoft chose to do this using a special set of "Wide" functions that allowed Unicode character sets to be worked with. Wine, of course, needed to followed this same mechanism to implement the "W" counterpart to all of the ASCII functions. The patch discussed in WWN #25 added support for Cyrillic characters. Ove Kåven suggested looking at the environment variables in the user's environment, and in fact that's what eventually ended up happening.

Even as recently as 2000 the discussion was going on about how to best support Unicode. It wasn't exactly clear (WWN #41 ) directly implementing the ASCII functions on top of Unicode was the best direction. Well, that mechanism proved to be the way to go. By 2002 work had progressed to the point where Shachar Shemesh started work on languages requiring bidi support.

"We are now working on the last missing piece ... to complete the transition from a Win3.1 architecture to a NT-like architecture. Once this is done we'll be able to release 1.0. I won't give any dates though..."

Alexandre Julliard, October 29th, 2001

So where are we headed? As we mentioned, DirectX support is going to continue to improve in the short term. Work on getting games to run will likely drive efforts to improve copy protection support. On the tools side of things, the new Wine Wiki has proven to be quite successful. In the more distant future, Mac OS X on Intel presents an interesting opportunity and could very well lead to very successful port. And, as always, at any time a major patch could arrive from out of the blue that makes Wine do something new. All this work, along with lots of bugfixes, will eventually get us to Wine 1.0. I'm not going to be the one to throw out any predictions of when that'll happen.

World Wine News