World Wine News

We need ntoskrnl to run some subset of drivers on wine. Notably cd protection drivers. They are not hardware drivers, but a means to access something that is not accessible from user space. So in a sense ntoskrnl is just a process to run those drivers in, and send/receive requests to/from those drivers.

You need to keep in mind that this is a kernel level, so some things will not work, and some things have to work that normally do not in the user space. Also ntoskrnl for windows _is_ what wineserver for wine.

User space programs talk to these drivers in the following way: 1. At the start up, driver creates symbolic link in "\DosDevices\something". 2. Program calls CreateFile("\\.\something",..) to get a handle to the device. 3. Uses ReadFile/WriteFile/DeviceIoControl on that handle. (For now we need only DeviceIoControl. Read/Write could be added later.)

Here is a list of the most pressing problem and the way to solve them.

Any ideas and comments will be greatly appreciated, Vitaliy.

Problems:

1. Open handle to a device - NtCreateFile.
2. Call DeviceIoControl on a device. (This also includes (1) since we need to know that this is not a cdrom, vxd or something else.)
3. Resolve user space handle to the DEVICE_OBJECT. (Assuming that we don't call ntoskrnl through wineserver).

Solutions for (1):

1.1. Use a hack to determine that passed name is a device name. (Device names don't have colon in them).
1.2. Check against all possible names other than device names. (A:..Z:, etc.)
1.3. Try it last if all else fails. (since name spaces don't intersect this should work.)
1.4. Use object manager to lookup the name to find out what is it. (Unless we open all handles using this method it will add one extra wineserver call.)

Solutions for (2):

• ntdll->ntoskrnl
  2.1. Use named pipe to communicate with ntoskrnl process. Pass commands over the pipe in internal format.
  2.2. Use UNIX IPC to talk to ntoskrnl. (pipe, socket, etc.)
• ntdll->wineserver->ntoskrnl
  2.3. Call wineserver which will call ntoskrnl using 2.1. or 2.2.
  2.4. Call wineserver which will call ntoskrnl using proper IoXXX functions from ntoskrnl to create an IRP. ntoskrnl will process them and notify wineserver when call is complete. (This is the most complicated and closer represents what windows does. There is still a problem communicating with ntoskrnl. It is probably the way to go in the future. But that will require number of things implemented which are not trivial.)
  

Solutions for (3):
MSDN quote: "Note that handles are not used for accessing device objects or driver objects."

• 3.1. Use wineserver to resolve handle to some internal identifier and pass it to ntoskrnl.
• 3.2. Pass handle and calling process id (pid, global handle, etc.) to ntoskrnl. ntoskrnl will call wineserver to resolve handle to internal id. (This handle will have to be either cached, or it will have to be created for every call.)
• 3.3. (requires 2.3. or 2.4.) Wineserver will resolve handle and pass internal id to ntoskrnl. (Unless we want wineserver do much more than that.)

I think that the only way to make device drivers work properly is to eliminate wineserver completely, and make it what it really wants to be, which is a user mode implementation of ntoskrnl, something like User Mode Linux.

That would require being able to write to a process's memory from the ntoskrnl, which is what you really need to implement ReadFile and WriteFile so device driver can work properly.

That project is really beyond the scope of Wine... but an interesting one nonetheless :)

First let me thank everyone helping me and Ivan to get ntoskrnl moving forward.

Summary of all the ideas and requirements:

1. wineserver should not be used to run drivers (absence of required functionality, single threaded, stability issues)
2. ntoskrnl should be used to run drivers only and link to ntdll and not the other way around. Unless we implement int 0x2e handling and replace wineserver with ntoskrnl. (this is wine not reactos, ntdll already has all the required functionality)
3. Drivers cannot be run in a process that requires them (drivers we are concerned about keep information that is shared between different processes, drivers require their own environment that is different from a user process environment, ntdll has no means nor should it have them to run a driver)

And that's it.

This validates original design by Ivan Leo Puoti for ntoskrnl. Now, there are still unanswered questions:

1. How to talk to ntoskrnl (directly from ntdll or through wineserver).
2. How to get information required for ntoskrnl from wineserver.
3. How to identify this is a device operation and not a file/named pipe/mail slot, etc.

I know folks you have more ideas, keep them coming.

I should preface this by saying that I don't mean any criticism or offence by this. You asked the question and the following reflects my observations of current limits to the development model and what would have to be changed to improve scalability. That doesn't mean the model has not served well thus far or with the available pool of developers to date.

Having to pipe all the changes through one person limits scalability. Until recently the facilities didn't even exist for supporting the maximum scalability using that model (distributed branching), and even though they exist now few people are using them and they are not integrated into the central commit process.

Even as things are now there is a disincentive to new developers. Some developers don't bother submitting patches because they feel it's too much work to get them accepted, and sometimes the patch doesn't get written at all because there is not enough certainty that it will be accepted. Before you suggest that this is speculative, I have one developer here who has made patches to fix Wine bugs experienced at home and not submitted them for that exact reason. You can try getting guidance on what is likely to be accepted, but again you run into a bandwidth problem with Alexandre - one person cannot effectively give guidance to everybody who needs it, and such guidance as he is able to give doesn't always help that much. The process requires that developers risk their work amounting to nothing because it won't be accepted. How many times have you seen people say that "Alexandre doesn't always know what he wants, but he knows what he doesn't want"?. That perception is partly due to the fact that it is impossible for one person to take the time to really understand what is wanted for every area of Wine that is being worked on at any one time (humans are linear, it limits them).

I suspect the current model is either at or near its limits. It would certainly not cope with a significant number of commercial outfits putting in a serious level of contribution, nor does it encourage them to make the attempt.

To scale better you would need to divide the project into different areas of responsibility and have multiple committers. You could have overall guidelines, but you'd need to be willing to trust the managers of each area to make final decisions in that area - even so far as to let them make mistakes and let them learn from those mistakes. Almost every subdirectory of "dlls" could in principle be a separate area of responsibility with a separate group of committers, although it might be more convenient to have more-or-less related subdirectories under the umbrella of a single group. Each group responsible for an area would need to be prepared to give advice to people working on some patch in that area that if followed would make it virtually certain that the patch would be accepted.

A supporting facility you would probably need would likely include either separate mailing lists for each sub-project (things get lost in wine-devel now with consolidated read-only lists for people who want a convenient way of watching all of the discussions or a shift to an NNTP hierarchy with mailing gateways.

These structures are nothing new - most of the really big projects work this way - look at Gnome, KDE, and for the insane end of the scale, Debian.

Even with the Linux kernel, which is the only project that I am aware of that is comparable in size and follows a similar model to Wine, there appears to me to be some greater division of responsibility.

This is all relevant, of course, only if significant expansion of the pool of developers is a desirable goal.

I must disagree, the LOTM (Lord Of The Manor) governance model may work for an small outfit but wine has already outgrown it. I have two or three withheld patches which are absolute show stoppers for running wine under Solaris. They are withheld despite the fact they work because they were refused, yet every second week I am forced to work around some portability problem introduced by someone else - not exactly ISO9001 quality assurance. This causes problems for both me and the wine project because:

1. wine is NOT as portable as it should be.
2. I am forced to become the LOTM for wine under Solaris since I am currently the only source (I know of) for Solaris wine.

There must have been half a dozen times where I have decided to abandon the wine project due to its governance model, only to be encouraged back to it by my customers. These days I submit my patches to comply with the LGPL, and if they go in all well and good, if not I no longer care... Is this how developers should be thinking about wine ?

It's also important to remember that many developers that contribute (Including myself) are volunteers, volunteers are hard to come by, but really easy to get rid of. You need a governance model that is not only fair and even handed with people, but is SEEN TO BE fair and even handed. This model is not that.

Now I'm not saying here that Alexandre isn't objective, just that his criteria of assessment is not clear to me, and after 2 years doing this I still reckon I have about a 50/50 chance of having anything major accepted. If it's not clear to me, then it must be positively opaque to someone new. You might say that the project is open, but the governance is closed in that the developer and user communities have little influence over governance.

We actually have a todo on Jeremy Newman's list to build a patch management system for wine-devel, for Alexandre.

Our hope was that we could adopt some of the CodeWeavers systems (we have a ticket system that's pretty slick, for example).

However, it became clear that the requirements were fairly substantial (the tight emacs integration became our first clue :-/), and that project got back burnered.

At the time we were discussing that, though, we didn't have many volunteer web programmers; maybe we should revisit that. Alexandre, would you be interested if folks other than Jer volunteered to help build such a system?

With that said, I have to ask - what open source projects are you guys working on that don't suffer from these problems? I'm now a successful contributor to the Linux Kernel (tweaked isofs for Windows CDs) and it took me 3 years and countless dropped emails, despite the personal help of Alan Cox and Andrew Morton before my patch got in (and I have another patch, a minor bug fix, that I despair will ever see the light of day). I had a similar situation with MythTV (and the #mythtv channel is actively hostile to anyone mildly clueless), and the list goes on.

Based on my experiences, I would say that Wine is a cut above, and Alexandre does a very fine job.

On the other hand, I've often thought that the developer section should have a big FAQ to help explain how Alexandre works (notably the fact that he uses the absolute minimum amount of communication required at any time) *grin*

As soon as the last hhctrl patch is committed (embedding in the correct window), HTML Help will be functional. That means that you can open up a chm help file with wine's help viewer, hh. It requires IE6 to be installed for the time being, until our shdocvw and mshtml are up to par. You'll also need a copy of native itss.dll (it has to be registered with wine):

$ regsvr32 itss.dll
$ WINEDLLOVERRIDES=shdocvw,mshtml,urlmon,shlwapi,itss=n wine hh helpfile.chm

A few things need to be completed, but not much. Most noticeably the toolbar buttons need to be drawn and submitted to wine-patches. If you would like to help in that area, send me an email and I'll let you know how it needs to be done. The tabs are blank and need filling in. Attached is a screenshot of hh in action. If anyone has any comments or suggestions, let me know and I'll address those.

I thought it was about time I gave a status update on DirectX9/wined3d, so here goes.

Short of a couple of bug fixes (one to shaders, and another that causes some games to crash) and a couple of patches that need resending (support for non-power2 textures in warhammer 40k) wined3d and d3d9 is functionally as complete as the version on http://directxwine.sourceforge.net/ , that means that as soon as pixel shaders are copied across from d3d8 it's more or less the end of adding features to wined3d and the start of fixing the remaining bugs and getting the performance up to spec.

Anyhow, as soon as I've sent in those patches I'll provide another status update with a bit more information about moving d3d8 over to wined3d and the improvements that have been made.

Finally we've got safedisc1 running on linux. Thanks go to Laurent Pinchart, Vitaliy Margolen, Brad DeMorrow, Marcus Meissner, and Alexandre for contributing time/code/ideas. The code is 100% DMCA compliant and hopefully can be cleaned up to be good enough for cvs over the coming days and weeks. Two games have been tested, they both use the same version of safedisc (don't ask how we know, we do)

http://www.kievinfo.com/images/wine/HoM3_1.jpg
http://www.kievinfo.com/images/wine/safediscworks.jpg

http://www003.portalis.it/115/wine/safediscworks2.png

The webserver is back online. There are still plenty of quirks to fix. Feel free to bug me here on wine-devel about any issues that crop up.

ChangeLog:

• moved OS to Debian Sarge
• now using - Apache 2.0.54 / PHP 4.3.10
• switched from sendmail to Exim4
• upgraded mailing lists from mailman 2.0 to 2.1
• mailing lists now using pipermail for archiving, note: old links will work as the hypermail archives still exist

I'm still tweaking configs and all so expect the site to be a bit erratic over the next few days.

Just because no one else has said it yet:

Nice work, Jer!

(Jer is masking all the behind the scenes wrangling he did with our two ISPs to arrange this, and the mad scramble to replace a raided drive that had fried, and the rebuild to Debian. )

It was darn nice work on his part to get it all essentially done in a day.

I've seen conflicting reports about whether Wine is safe to run on SMP systems.

Have the issues described in http://www.winehq.org/pipermail/wine-devel/2005/01/0060.html been taken care of?

I just pinged the reporter of the only bug with SMP in the summary, http://bugs.winehq.org/show_bug.cgi?id=2708 to see if he can still reproduce it.

If anybody is aware of any cases of Wine's Microsoft Installer code not working, please log them in Wine's bugzilla (http://bugs.winehq.org/), assign them to me, and I will attempt to fix them.

Make sure to set "msi" = "builtin", "*msiexec" = "builtin" when testing your installers, and if possible test with the latest Wine CVS.

If the installer is available as a free download, please include a link to it.