WWN Issue 258

This is the 258th issue of the Wine Weekly News publication. Its main goal is to buy plane tickets. It also serves to inform you of what's going on around Wine. Wine is an open source implementation of the Windows API on top of X and Unix. Think of it as a Windows compatibility layer. Wine does not require Microsoft Windows, as it is a completely alternative implementation consisting of 100% Microsoft-free code, but it can optionally use native system DLLs if they are available. You can find more info at www.winehq.org

This week, 217 posts consumed 1121 K. There were 59 different contributors. 30 (50%) posted more than once. 39 (66%) posted last week too.

The top 5 posters of the week were:

33 posts in 173K by Luke Kenneth Casson Leighton
24 posts in 75K by Mike Hearn
21 posts in 83K by Robert Shearman
14 posts in 47K by Paul Vriens
8 posts in 23K by Juan Lang

WineConf 2005

01/17/2005

FreeDCE & Wine & DCOM	01/16/2005	Archive
RPC / COM / OLE Last week the Open Group announced the release of DCE 1.2 under the LGPL. DCE, the Distributed Computing Environment, is a massive amount of software previously available under a restrictive license precluding it's use in most free software projects. In fact, if you were to just read the press release above it would be next to impossible to grasp exactly what all this stuff does. What it could potentially mean for Wine is, any application using DCOM or Microsoft's RPC directly could use the low-level mechanisms in DCE to communicate to another machine. If you're thinking, "Oh, that's just a client/server type of thing", well, you're on the right track. That's the nutshell version, but let's take minute to look into COM and the pieces involved. First, Mike Hearn wrote a nice intro to COM a few months ago (see WWN issue #231 ). It's worth taking a look at to get a handle on some of the terminology involved. COM, Component Object Model, is a method for objects within a program to be passed around in way that allows whatever is passing the object to not really care about what's receiving the object. So, say you have a function in a library that needs to send some data over to a different library. If you use COM, the data is sent in a way that allows you to rip out that other library and completely replace it with a different one. COM also allows for fancy things such as Object Linking and Embedding to work. OLE sits on top of COM and lets you do things like stick an Excel spreadsheet in Word by actually embedding an instance of Excel in the other program. That concept in and of itself is quite useful. Microsoft took this a step further and came up with the idea of "Distributed COM". With DCOM, different processes and threads can talk to each other. As Mike described it, Each COM object resides in an apartment, and each apartment resides in a process, and each process resides in a machine, and each machine resides in a network. Allowing those objects to be used from any of these different places is what DCOM is all about. Now, it just so happens that most DCOM communication just needs to happen within a machine. Right now that's where the priority lies with Wine - making things like InstallShield work requires DCOM. Having Wine speak DCOM "over-the-wire" requires hooking it into the RPC subsystem and making sure everything is binary compatible with Microsoft. Wine does have an implementation compatible with Microsoft's RPC, but it needs more work. Right now Wine's DCOM doesn't even use it - it communicates between apartments using named pipes. Luke Kenneth Casson Leighton started off a large thread about DCE going LGPL and pointed out some of the advantages of integrating with it: with FreeDCE using an NTLMSSP authentication plugin and a "Named Pipes" transport in samba or samba tng, it becomes possible for FreeDCE to be utilised to make DCOM applications work under Wine. DCOM applications can be made to WORK under Wine using FreeDCE, Samba or Samba TNG with NamedPipes. unmodified Win32 applications can do DCE/RPC and DCOM utilising FreeDCE and Samba, with a minimum of development effort inside Wine and Samba. also, FreeDCE can be leveraged to provide _proper_ implementations of the underlying Win32 interfaces. for example, REGEDT32.EXE can be made to work over a network running _natively_ under Wine. NetUserGetInfo can return "meaningful" information. There was quite a bit of chatter back and forth between Luke, Mike, and Rob Shearman. Rob and Mike pointed out that FreeDCE didn't contain any DCOM work that could be used by Wine. In fact, no one has a free implementation of the Win32 API's necessary for DCOM. Luke pointed out that it was silly to try to reimplement DCOM on anything other than DCE because it's just a massive amount of code that shouldn't be duplicated. After some lengthy exchanges, Mike summarized things from a Wine perspective: As Rob has pointed out, we're not duplicating anything as the code we need does not exist. There is no NIH (ed. note: Not Invented Here ) syndrome here: if you can point to an LGPL or BSD/X11 licensed implementation of OLE32, OLEAUT32 and things like CreateProxyFromTypeInfo then we'd be very interested. The only one we do know of is in Cedega and we already used quite a lot of code from that. As it is, the DCE-RPC code already in Wine CVS is more than enough for now. If we need to extend it later using code from FreeDCE may be possible, but right now we : a) Don't know of any pure DCE-RPC applications people want to run b) Have a DCOM implementation that does not use RPC (b) will change at some point, however most programs out there today do not use the NDR marshallers or DCE-RPC APIs, instead they use MOP opcode strings or the universal marshaller which generates proxies on the fly from type databases. So there's still a lot to implement outside of the core RPC API, the capabilities of which just aren't used that heavily in Windows. So to recap: FreeDCE implements DCE-RPC better than we do, but we don't care because we don't actually use our current RPC code yet, and when we do it won't be stressing the current capabilities. The capabilities that will be needed from our RPC runtime aren't implemented by any pre-existing project as they are Microsoft-specific extensions to the API Samba4 is focussed only on wire compatibility, but not binary API compatibility. Right now we don't care much about wire compat, and won't for many years. They do not implement much code we care about, that which we do care about (things like the OXID Resolver) are not hard to write. Even so we have a dialog open with them on code sharing. Just to make this point perfectly clear WE ARE WRITING CODE THAT DOES NOT EXIST ELSEWHERE. The closest implementation of DCOM we need can be found in TransGaming's Cedega, and we're using bits of their code already. Then there was a lot more back and forth discussion concerning details of using IDL compilers and such. Wez Furlong jumped into the conversation and approached it from the opposite direction that Mike described: FreeDCE doesn't contain a working DCOM implementation. The following areas need(ed) some work for that: NTLMSSP (which we now have in freedce, thanks to Luke) The rpcd/endpoint mapper needs awareness of ORPC and implement one or two ORPC specific services in order to maintain the lifetime of remotely activated components. The IDL compiler and marshalling stubs need awareness of ORPC On top of that, the local COM library needs to be implemented Filling out these areas is massively less work than re-implementing DCE-RPC; I made a fair start on (2) and (3) 4 years ago, but lack of interest from the world at large (and a need to pay my bills) caused it to be put on hold. Please believe me when I say that there is a large amount of non-trivial code in there; I have huge respect for the people that wrote it and the amount of time that it took to get it there. Don't forget that this is production quality code that has been used by huge players for years. I pity anyone that would think of taking on the task of re-implementing it, not because it's nasty but because it's a huge effort. While I can't commit development time right at this moment (I'm booked up with the PHP project in most of my "free" time), I am happy to help in any other way that I can; I researched the implementation of DCOM quite heavily back then, so I probably have a better idea than most about getting it done. PS: I'd love to have someone sponsor my employer (omniti.com) to have me work on getting this implemented. Discussion also veered into what exactly from DCE could be integrated and how to go about doing so. Rob asked Luke to take a look at the RPC code and see what he thought. Luke did so, and by the end of the week began posting questions about the specifics of it. Discussions like this are interesting because Wine folks tend to have a top-down approach at looking at this and the Samba/TNG folks tend to have a bottom-up view. When they meet in the middle it makes for a lot of people scratching their heads.

Archive

RPC / COM / OLE

Last week the Open Group announced the release of DCE 1.2 under the LGPL. DCE, the Distributed Computing Environment, is a massive amount of software previously available under a restrictive license precluding it's use in most free software projects. In fact, if you were to just read the press release above it would be next to impossible to grasp exactly what all this stuff does. What it could potentially mean for Wine is, any application using DCOM or Microsoft's RPC directly could use the low-level mechanisms in DCE to communicate to another machine. If you're thinking, "Oh, that's just a client/server type of thing", well, you're on the right track. That's the nutshell version, but let's take minute to look into COM and the pieces involved.

First, Mike Hearn wrote a nice intro to COM a few months ago (see WWN issue #231 ). It's worth taking a look at to get a handle on some of the terminology involved. COM, Component Object Model, is a method for objects within a program to be passed around in way that allows whatever is passing the object to not really care about what's receiving the object. So, say you have a function in a library that needs to send some data over to a different library. If you use COM, the data is sent in a way that allows you to rip out that other library and completely replace it with a different one. COM also allows for fancy things such as Object Linking and Embedding to work. OLE sits on top of COM and lets you do things like stick an Excel spreadsheet in Word by actually embedding an instance of Excel in the other program.

That concept in and of itself is quite useful. Microsoft took this a step further and came up with the idea of "Distributed COM". With DCOM, different processes and threads can talk to each other. As Mike described it, Each COM object resides in an apartment, and each apartment resides in a process, and each process resides in a machine, and each machine resides in a network. Allowing those objects to be used from any of these different places is what DCOM is all about.

Now, it just so happens that most DCOM communication just needs to happen within a machine. Right now that's where the priority lies with Wine - making things like InstallShield work requires DCOM. Having Wine speak DCOM "over-the-wire" requires hooking it into the RPC subsystem and making sure everything is binary compatible with Microsoft. Wine does have an implementation compatible with Microsoft's RPC, but it needs more work. Right now Wine's DCOM doesn't even use it - it communicates between apartments using named pipes. Luke Kenneth Casson Leighton started off a large thread about DCE going LGPL and pointed out some of the advantages of integrating with it:

with FreeDCE using an NTLMSSP authentication plugin and a "Named Pipes" transport in samba or samba tng, it becomes possible for FreeDCE to be utilised to make DCOM applications work under Wine.

DCOM applications can be made to WORK under Wine using FreeDCE, Samba or Samba TNG with NamedPipes.

unmodified Win32 applications can do DCE/RPC and DCOM utilising FreeDCE and Samba, with a minimum of development effort inside Wine and Samba.

also, FreeDCE can be leveraged to provide _proper_ implementations of the underlying Win32 interfaces.

for example, REGEDT32.EXE can be made to work over a network running _natively_ under Wine.

NetUserGetInfo can return "meaningful" information.

There was quite a bit of chatter back and forth between Luke, Mike, and Rob Shearman. Rob and Mike pointed out that FreeDCE didn't contain any DCOM work that could be used by Wine. In fact, no one has a free implementation of the Win32 API's necessary for DCOM. Luke pointed out that it was silly to try to reimplement DCOM on anything other than DCE because it's just a massive amount of code that shouldn't be duplicated. After some lengthy exchanges, Mike summarized things from a Wine perspective:

As Rob has pointed out, we're not duplicating anything as the code we need does not exist. There is no NIH (ed. note: Not Invented Here ) syndrome here: if you can point to an LGPL or BSD/X11 licensed implementation of OLE32, OLEAUT32 and things like CreateProxyFromTypeInfo then we'd be very interested. The only one we do know of is in Cedega and we already used quite a lot of code from that.

As it is, the DCE-RPC code already in Wine CVS is more than enough for now. If we need to extend it later using code from FreeDCE may be possible, but right now we :

(b) will change at some point, however most programs out there today do not use the NDR marshallers or DCE-RPC APIs, instead they use MOP opcode strings or the universal marshaller which generates proxies on the fly from type databases. So there's still a lot to implement outside of the core RPC API, the capabilities of which just aren't used that heavily in Windows.

So to recap:

FreeDCE implements DCE-RPC better than we do, but we don't care because we don't actually use our current RPC code yet, and when we do it won't be stressing the current capabilities. The capabilities that *will* be needed from our RPC runtime aren't implemented by any pre-existing project as they are Microsoft-specific extensions to the API
Samba4 is focussed only on wire compatibility, but not binary API compatibility. Right now we don't care much about wire compat, and won't for many years. They do not implement much code we care about, that which we do care about (things like the OXID Resolver) are not hard to write. Even so we have a dialog open with them on code sharing.

Just to make this point perfectly clear *WE ARE WRITING CODE THAT DOES NOT EXIST ELSEWHERE*. The closest implementation of DCOM we need can be found in TransGaming's Cedega, and we're using bits of their code already.

Then there was a lot more back and forth discussion concerning details of using IDL compilers and such. Wez Furlong jumped into the conversation and approached it from the opposite direction that Mike described:

FreeDCE doesn't contain a working DCOM implementation. The following areas need(ed) some work for that:

NTLMSSP (which we now have in freedce, thanks to Luke)
The rpcd/endpoint mapper needs awareness of ORPC and implement one or two ORPC specific services in order to maintain the lifetime of remotely activated components.
The IDL compiler and marshalling stubs need awareness of ORPC
On top of that, the local COM library needs to be implemented

Filling out these areas is *massively less work* than re-implementing DCE-RPC; I made a fair start on (2) and (3) 4 years ago, but lack of interest from the world at large (and a need to pay my bills) caused it to be put on hold.

Please believe me when I say that there is a large amount of non-trivial code in there; I have huge respect for the people that wrote it and the amount of time that it took to get it there. Don't forget that this is production quality code that has been used by huge players for years. I pity anyone that would think of taking on the task of re-implementing it, not because it's nasty but because it's a *huge* effort.

While I can't commit development time right at this moment (I'm booked up with the PHP project in most of my "free" time), I am happy to help in any other way that I can; I researched the implementation of DCOM quite heavily back then, so I probably have a better idea than most about getting it done.

PS: I'd love to have someone sponsor my employer (omniti.com) to have me work on getting this implemented.

Discussion also veered into what exactly from DCE could be integrated and how to go about doing so. Rob asked Luke to take a look at the RPC code and see what he thought. Luke did so, and by the end of the week began posting questions about the specifics of it. Discussions like this are interesting because Wine folks tend to have a top-down approach at looking at this and the Samba/TNG folks tend to have a bottom-up view. When they meet in the middle it makes for a lot of people scratching their heads.

Pirates! Screenshot	01/15/2005	Archive
Integration Screenshots are just cool. Screenshots showing off one of the hottest selling new games running on Wine are kick ass . Pirates!, a remake of that game from the 80's some of us lost our youth to, was coaxed into work thanks to the work of Oliver Stieber. Thus far the code required to run it hasn't appeared, but with the work being done by Jason Edmeades, DirectX 9 is beginning to take shape. Oliver wrote: I've been working on DirectX 9 over the past week and have managed to get Sid Meier's Pirates! working better than it does for me under Cedega along with quite a few demos using DirectX 9. There's still quite a lot of work to do to get a full DirectX 9 implementation but it shouldn't take too long to get DirectX 9 in wine as functional as DirectX under cedega. I'm just doing a bit of housekeeping so the patches will be ready for next week along with a list of what's working and what's not. Mike Hearn wanted to know just how well it ran and how things like installation worked. Oliver explained: I'm getting memory allocation problems after quite a bit of game play, no videos yet, I haven't tried installing and I always use no-cd patches. The speed's a bit choppy, but playable, and there are a few minor graphics gliches (I've got a test case for one so it shouldn't be too hard to fix) I would rather have slow gameplay than buggy gameplay. Out of the box installs require a bit more than DirectX but hopefully the OLE and MSI work that's been going on lately should help that out. I'm also thinking of working on half-life 2 and possibly WoW and The Sims 2 as some point, but I may just try to get the DirectX 9 implementation as complete as possible first. [unless the makers of The Sims 2 give me enough money to pay the rent!] He added later: I'm testing some of the more complex DirectX 9 functionality at the moment and working on an annoying clipping bug to try and improve DirectX with everything, but I'll have a look at Pirates again soon. If anyone's got anything else that they would like me to look at (and there's a demo or you can post me a copy) then send me an email and I'll see what I can do. [Pirates is very simple and will run with stubbed out textures, at the moment it's running fine with simple world detail and any level of model complexity, pixel shaders turned off and simple shadows I can't see it breaking any time soon] Later in the week Oliver sent some patches and noted on wine-devel: I've just sent some patches to patches that are enough to get Pirates running. Installing pirates... Pirates uses a MSI installer, I had to Set winver to win98. When prompted about directX 9 say no. (or yes if you really want to). If you get an OLE error make a note and try again, I managed to install on the fourth attempt. Then to run Pirates you need to... Turn pixel shaders off Turn shadows off. or you get missing textures. I'd also suggest turning down the detail (just in case), but it doesn't seem to matter that much. The problems I've come across so far are: Memory issues: Unknown cause as yet. When visiting the Governor only a section of the screen fades in and out, this is caused by a viewport larger than the window not being handled correctly. Sometimes textures on models like the docks don't render properly. Sometimes the textures are the wrong way around or wrapped incorrectly, this may be the same problem as above. I get 7-12 FPS in sailing, everything else is a lot quicker (up to 70fps). AMD 2000+ 512MB Ram ATI-9600. I've tried a few other DirectX 9 games but I generally can't get passed the installer (using wine or cegeda).

Pirates! Screenshot

01/15/2005

Archive

Integration

Screenshots are just cool. Screenshots showing off one of the hottest selling new games running on Wine are kick ass . Pirates!, a remake of that game from the 80's some of us lost our youth to, was coaxed into work thanks to the work of Oliver Stieber. Thus far the code required to run it hasn't appeared, but with the work being done by Jason Edmeades, DirectX 9 is beginning to take shape. Oliver wrote:

I've been working on DirectX 9 over the past week and have managed to get Sid Meier's Pirates! working better than it does for me under Cedega along with quite a few demos using DirectX 9.

There's still quite a lot of work to do to get a full DirectX 9 implementation but it shouldn't take too long to get DirectX 9 in wine as functional as DirectX under cedega.

I'm just doing a bit of housekeeping so the patches will be ready for next week along with a list of what's working and what's not.

Mike Hearn wanted to know just how well it ran and how things like installation worked. Oliver explained:

I'm getting memory allocation problems after quite a bit of game play, no videos yet, I haven't tried installing and I always use no-cd patches.

The speed's a bit choppy, but playable, and there are a few minor graphics gliches (I've got a test case for one so it shouldn't be too hard to fix) I would rather have slow gameplay than buggy gameplay.

Out of the box installs require a bit more than DirectX but hopefully the OLE and MSI work that's been going on lately should help that out.

I'm also thinking of working on half-life 2 and possibly WoW and The Sims 2 as some point, but I may just try to get the DirectX 9 implementation as complete as possible first. [unless the makers of The Sims 2 give me enough money to pay the rent!]

He added later:

I'm testing some of the more complex DirectX 9 functionality at the moment and working on an annoying clipping bug to try and improve DirectX with everything, but I'll have a look at Pirates again soon.

If anyone's got anything else that they would like me to look at (and there's a demo or you can post me a copy) then send me an email and I'll see what I can do.

[Pirates is very simple and will run with stubbed out textures, at the moment it's running fine with simple world detail and any level of model complexity, pixel shaders turned off and simple shadows I can't see it breaking any time soon]

Later in the week Oliver sent some patches and noted on wine-devel:

I've just sent some patches to patches that are enough to get Pirates running.

Installing pirates... Pirates uses a MSI installer, I had to

Set winver to win98.
When prompted about directX 9 say no. (or yes if you really want to).

If you get an OLE error make a note and try again, I managed to install on the fourth attempt.

Then to run Pirates you need to...

Turn pixel shaders off
Turn shadows off.

or you get missing textures.

I'd also suggest turning down the detail (just in case), but it doesn't seem to matter that much.

The problems I've come across so far are: Memory issues: Unknown cause as yet.

When visiting the Governor only a section of the screen fades in and out, this is caused by a viewport larger than the window not being handled correctly.

Sometimes textures on models like the docks don't render properly.

Sometimes the textures are the wrong way around or wrapped incorrectly, this may be the same problem as above.

I get 7-12 FPS in sailing, everything else is a lot quicker (up to 70fps).

AMD 2000+ 512MB Ram ATI-9600.

I've tried a few other DirectX 9 games but I generally can't get passed the installer (using wine or cegeda).

Wine Wiki	01/19/2005	Archive
Documentation Ivan Leo Puoti forwarded a message about a new Wine Wiki: Can you please add Wine-Wiki.org to the list of Compatibility site. We have been growing greatly, and if it was on the site, it would help everyone.

Nvidia Driver Bug	01/16/2005	Archive
Graphics Mike Hearn gave everyone a heads up on a problem with a graphics driver: Using the so-called 'open source' nv driver that ships with Xorg causes Wine at least on my hardware to misdetect libXrender as being buggy. This disables clientside font rendering and also causes the entire screen to go black whenever a Wine application has focus. Just letting people know in case they see similar symptoms. I normally run with the binary nvidia driver which doesn't have the same problem, so I will not be debugging this anytime soon.

World Wine News