Win16 and DOS implementations

The place to bring up any design issues, or post your own creations

Moderator: Moderator Team

bluefoxicy
Posts: 3
Joined: Tue Oct 07, 2008 12:22 am

Win16 and DOS implementations

Post by bluefoxicy »

I recall many of my compsci teachers explaining how it's just not possible to implement "legacy" support, hence why Win9x is so unstable. I was recently running some 16 bit apps (Wine can do it, ROS can't) and thinking on what would allow stable 16-bit and 8-bit support. Here's my thoughts; use them, abuse them, ignore them, comment on them, I don't much care really, this is one hell of a fun mental exercise for me anyway. Comments are appreciated though, since it'd probably help my understanding of the problem, which is nice...

Full native API implementation

First off, this whole discussion assumes all APIs get implemented as full native code. This means 32-bit or 64-bit, even for 16-bit DOS and Windows 3.1 calls. All 16-bit calls should get thunked (I seriously hope that's the right term) into 32-bit libraries.

Emulator backend

The backend for native Win16 and DOS should be a virtualizer or emulator. For 64-bit hosts you will need to use a real emulator like Qemu or a custom implementation; in 32-bit protected mode you can probably use VM86 mode for real mode applications. Some magic can happen with the GDT and LDT to do 286 protected mode, but again only in 32-bit protected mode; the LDT is no longer valid in 64-bit long mode, and you must fully emulate 16-bit code.

For 286 protected mode, I believe you must have a GDT; you can probably change the GDT when changing to the emulator task. Since the OS has to manage the LDT, the program loader should set that up upon loading the program. Any 16-bit code goes into a segment with the D-bit cleared of course.

In cases where virtualization fails-- for example, a loader needing a memory segment taken by the OS--dirtier tricks should be used. For example, clear the interrupt table and let interrupts cause a fault, such that the emulator itself can pick up and handle the interrupt. This will require tight cooperation with the OS. In the most extreme case, a process may need to fall back completely to emulation.

Any and all activity that can be done outside Ring-0 should occur in Ring-3. This means if you go into full emulator mode, you better be running that outside the kernel; that stuff can crash so easy it's not funny.

DOS emulation

DOS emulation should fall to VM86 mode or a fully emulated 8086 mode. DOS emulation is, from my point of view, needlessly complex; however I would like to be able to run Doom (not Doom95) and Duke Nukem; Duke Nukem and especially Shadow Warrior (using the same engine) work on Windows 9x but seriously dislike Windows NT.

When running DOS emulation in VM86 mode, you can simply catch VM86-mode interrupts and pass them to the application itself. When a processor in VM86 mode calls an interrupt, EFLAGS will show 1 for the VM86 bit ([1], pg. 242), and you can thus pass control down to the application (emulator) to handle this. More recent processors uses a more complex method if VM86 mode extensions are enabled, whereby a 32 byte map tells which interrupts to handle in the old way, and which to handle by using a real-mode IDT to call real-mode interrupt handlers; this is great for running DOS, but not for getting an interrupt to pass control into a protected mode application, and thus isn't useful here (it's useful for DOSEMU).

If emulating rather than using VM86 mode, you can simply handle calling an interrupt by calling the appropriate code directly.

In either case, a full implementation of the DOS ABI needs to exist. This means something needs to handle the interrupts used to make OS calls. Something should optionally emulate EMM386, HIMEM.SYS, SVGA, etc. Something will have to emulate a SB16. It should be enough that a graphical game like Duke Nukem can run inside a DOS box; even more interesting would be running Windows 3.1 or 3.11 inside a DOS box.

What you don't need to emulate is a hard drive. Anything attempting to defrag or partition or whatever should fail; sorry, but there's a sanity limit, and you just hit it. OS calls to the file system will specify concrete goals like "Change to D:" or "cd \games\quake" and should not need to care that this is a compressed folder on NTFS. In fact, you could theoretically allow a "sandbox" mode to move a DOS application's C:\ root into any folder; deny access to other drives; and in general just muck around with how you want the app to interact with your system.

Win16

Win16 is not as touchy a discussion as DOS. Win16 wants to run in 386ENH mode and use protected mode to run Win16 applications; seeing as Win16 isn't running, things are easier than that.

Again, this code runs in 16-bit mode, and you need to use either VM86 mode, segmentation with 16-bit segments, or emulation to pull it off. Beyond that, Wine implements the Win16 API, so actually running stuff shouldn't be too difficult. The API calls can directly thunk, rather than play with interrupts; the Win16 API exposes sound, CD-ROM, video, etc.

There's not much I really want to say on Win16 that I didn't say on DOS. You don't need to worry about real mode artifacts like HIMEM.SYS or EMM386; you can probably avoid running into interrupt handlers (Wine somehow runs as a user mode program under Linux and does just fine with Win16); in general it's just a simpler beast. Make of it what you will.

References

[1] Pentium Processor System Architecture By Don Anderson, Tom Shanley, MindShare, Inc. Read on books.google.com
[2] Wikipedia on Windows 3.0.
[3] Wikipedia on Windows 3.1.
[4] Wikipedia on Virtual 8086 Mode.
[5] Brian's Kernel Development Tutorial: Global Descriptor Table.
[6] How to use the vm86 system call in Linux.
[7] Microsoft Help and Support on Real Mode Not Supported By Windows 3.1.
[8] Giese, Chris. Protected Mode. Bona Fide OS Development.
Z98
Release Engineer
Posts: 3379
Joined: Tue May 02, 2006 8:16 pm
Contact:

Re: Win16 and DOS implementations

Post by Z98 »

It's not that these things haven't been done, IE, DOSbox and co, it's just that the amount of effort needed to do it safely outside of running it in some kind of virtual machine just isn't worth it for the sake of backwards compatibility. And such an effort is outside the scope of this project, which is why so few people are willing to put the effort needed into it. Also, let's not forget that Wine was originally started targeting the 9x platform.
Carlo Bramix
Posts: 282
Joined: Thu Jan 04, 2007 12:43 am
Location: Italy

Re: Win16 and DOS implementations

Post by Carlo Bramix »

While dosbox is a nice way for getting dos application running, not as good as V86 emulation but still quite efficient, I must admit the Win16 limitation is real...
I found that there are lot of applications and (especially) drivers that are 32 bit programmes, but they have a 16-bit based installer.
Result: unknown file format error when you try to launch the executable of the installer. And it's very annoying...

Sincerely,

Carlo Bramini.
Z98
Release Engineer
Posts: 3379
Joined: Tue May 02, 2006 8:16 pm
Contact:

Re: Win16 and DOS implementations

Post by Z98 »

Well, that 16bit installer thing is still about 16bit compatibility, which isn't happening anytime soon, if at all.
gyROS
Posts: 199
Joined: Sun Nov 09, 2008 8:10 pm
Location: Deutschland

Re: Win16 and DOS implementations

Post by gyROS »

Doesn't the VESA-BIOS use 16bit code? If so, some possibility of executing 16bit code on the real machine has to be there anyway, otherwise it wouldn't be possible to change the screen resolution.
For sure it's a great deal of work, and if the original drivers are working, it's completely unnecessary.

The problem is obviously, that in 64 bit mode it's entirely impossible to run 16 bit code directly, so once ReactOS is 64 bit, one would have to drop native 16bit-support anyway! (as win64 did)

One possibility would be, to run a VM (like DOSBox) and redirect the API-calls to ReactOS, as MacOS is doing it for years, due to all the architecture switches. Most Win16 API functions should be available in the Win32 API anyway.
Actually this solution wouldn't be a real part of ReactOS, it would rather be an extra tool, to run 16bit windows apps in windows versions, which don't support those apps anymore.
Blackcrack
Posts: 1808
Joined: Tue Dec 20, 2005 12:55 pm
Contact:

Re: Win16 and DOS implementations

Post by Blackcrack »

Hy,

i had think twice like Carlo , but shold be an lightwave/open-bios&hardware-emulator and the dos shold have fs-support to let the command.com run as interpreter and a virtual -memory and hardwareemu on directx like n P1 or maby an P2 with an own used Memory wo have an realdos-memory and a svga+sb16 and maby an opengl-support to Direktlayer/Directx .. i have make me thinking's or like it can say : http://www.reactos.org/forum/viewtopic. ... 456#p53905 and http://www.reactos.org/forum/viewtopic. ... =30#p54506

Dosbox it's an intresting thing, but to "closed", it shold be an dossystem/subsystem/or how ever, who run from directory like an VirtualDrive and posibility to use an drive&realhardware and iso als CD-rom and folder,img and realfloppy to use... (an near like realdos Dos who can installing in the User account later if need...) with tty's support to enabling telnet&ssh any maby sftp on dos too. Dos can be like bash, but it mus be existand to develope more apps in modern times, linux get an exemple with bash... who be more integrated if it's installing as only the dosbox.. more like the "Net-package", only as exemple... in linux it's the kernel,bash and then the xserver, and why not on NT the "X"(NT) and on it the dos as service and with an own tsr hardware-emu who take an part of cpu and an part of Ram .. 1-32mbyte ( and it take/need realy not manny).. to have a "secure dos" with an tty-support who cant make anything agains the NTSystem. To have twice and get all who like this or this.. I see in this Dosproject an possibility to have easy enhanced possibilitys to working and maby on Terminal or ssh with terminal-programms/ascii-gui's with tunneling. , use old dos-soft too and to have a original platform to use maby the old Win in win*s* and old games like Doom, Tirian, some Jump and Run games who need only these and so furter to use some old software again and dont get it in trash..(oky, for it it's can take an big fad full emulator too, but need mutch more resources !!!) it is old knowing what used there by 2nd look and why not a 2.nd project to work by side of Reactos for Dos-Geeks or maby Freaks an Reactdos ;) to get more "Color's" inside like a summer meadow that now flourish :D The Programmers can make it now, they are show the Source Code it is now an absolute advantage to have the possibility it on the body of reactos to tailor ... On the old Dostimes was an which to have some possibilitys, now they are here.... imho..

same art of work can make an c64-sub and other one ... to have come possibilyt too ;) but, as own project too like Reactos and this speak about Dos project here right now..

and it is an own installable Project, so can use it in other NT-Architectures too.. isnt it ?
Also an o installable OpenSource-Dos-System in WinNT-architectures.. with tcpip and
tty/telnet/ssh/sftp-support :D

best Regards
Blacky
silverblade
Developer
Posts: 58
Joined: Sat Dec 18, 2004 9:41 pm
Location: Newbury, UK
Contact:

Re: Win16 and DOS implementations

Post by silverblade »

I've been looking into using DosBox to provide 16-bit support.

This relies upon SDL for video/sound/keyboard/network/CDROM support, and whilst I managed to get the CPU core and some very basic parts to build successfully, it's not very useful because you can't see or use it. :lol:

Several developers have disagreed with importing SDL into the main tree, and have suggested it should instead go into ros-apps, if anything. This would also mean DosBox needs to go into ros-apps. Both of which I'm fine with and if and when I get further with this, I'll commit the changes there.

DosBox already provides support for most DOS apps, can emulate various popular types of hardware and seems an ideal base on which to build. 16-bit apps that want to use the hardware directly typically won't work (properly) under the 16-bit subsystem in NT.

A few changes I'd want to make:

:arrow: Whilst DosBox could be used on ReactOS, I'd like to go a little further and make it properly integrated, eg have a configuration front-end in the style of a "properties" dialog window.

:arrow: DosBox supports "virtual drives", and starts with the Z: drive as the main one. Then you can mount other drives. For the sake of keeping things seamless I'd be looking at auto-mounting all drives that are present on the system, and instead create virtual directories/files inside the virtualised environment (ie, to provide command.com). This means every drive in DosBox would match 1:1 with the Windows drives.

:arrow: Running a 16-bit application from cmd.exe should invoke DosBox and re-use the existing console window. If the app wants to switch graphics mode then an SDL drawing surface can be used. This change basically means using Windows consoles for text-mode (though I am not sure how this would handle unusual characters and such.)

:arrow: Win16 support can be achieved by including an 'NE' format EXE loader. The way I'd then proceed would be to write 32-bit libraries that provides the same exported routines as in 16-bit versions of Windows, but which simply translate the parameters, handles etc. appropriately before calling Win32 APIs. When running Win16 applications we can most likely ignore the peripheral emulation as the apps themselves should go through the Windows API rather than directly access hardware.

That's pretty much all I have thought up about it so far. I may do some further work on this in the near future.
Andrew / Silver Blade

[ external image ]
Blackcrack
Posts: 1808
Joined: Tue Dec 20, 2005 12:55 pm
Contact:

Re: Win16 and DOS implementations

Post by Blackcrack »

Hy SilverBlade,

very nice to hear :) letz see and letz try, i be very impatient to try it out :D
You think to biosflash or other hardware too ? direckt connect to hardware ? maby making with an cpl in control ?
were can be maby the "control"-Center of Dosbox, like more in the System integrated ?
Or, to enabling at own Dos-version to installing in dos-box to use the own buyed,
old MS 5.00 or 6.00/6.2 or other diffrents Systems? as an "dosmashin/VM" maby and controling with the control.exe, an dos.cpl maby and in more as one folder copyed and use as Z: maby used the diffrend folders(or used as folders) by an own "xxxDosxxx.lnk/pif" ? only so as idea :D ? :mrgreen: To have more as only one dosversions installed ?
You know, the one who had learned on it to have it's realy crewl, you know what i mean *vbs*

"This means every drive in DosBox would match 1:1 with the Windows drives."
usable as Folder ?? in the FS ? to administration some
files with any "Commander" or Explorer² like an Zip or compressed direcktory/drive (DTools) ? ;)
who say with file to taken in this drive to boot and or like boot with maked as "bootsector"..
i think on io.sys and linuxkernel? *question-look* hihi :] if it's to direckt, delete it easy ;) *bg*

bests regards
Blacky

p.s.: sorry if i babylike, but i see the hudge swweeett Cake :mrgreen:
silverblade
Developer
Posts: 58
Joined: Sat Dec 18, 2004 9:41 pm
Location: Newbury, UK
Contact:

Re: Win16 and DOS implementations

Post by silverblade »

Well, to be honest, I'm trying to keep it simple.

Allowing a DOS app direct access to the hardware under NT pretty much undermines the stability offered by the NT architecture. So no, you wouldn't be able to flash the BIOS of your machine, nor would you be able to directly access the hardware of the PC.

Effectively it is a semi-virtual machine. It is isolated from the hardware but is capable of accessing the files of the PC.

Being able to install an actual DOS (MS-DOS, FreeDOS etc.) is not the aim, though you should be able to use programs that ship with those systems (eg: EDIT.COM) There are obviously possible incompatibilities between DOS versions, which could be catered for within DosBox if necessary - if a DOS API changed between versions, just allow the user to choose what version they wish for DosBox to act like.

I'd like to avoid the use of virtual drives in DosBox, for the sake of clarity. Seeing Z:\ in DosBox and then not seeing it in Windows could potentially be confusing. Initially I'll stick with the virtual drive Z:\ but in future I'd probably suggest having a dummy 16-bit C:\COMMAND.COM which causes DosBox to load and run its own internal shell. (Basically any 16-bit app being run would invoke DosBox, which would then run the app inside itself - so a dummy COMMAND.COM would end up running the DosBox shell.) Kind of like a placeholder file. Might not work out though, we'll see.

I don't think DosBox lets you touch the bootsector of a disk directly? Again it isolates you from the actual hardware. If a file management program doesn't touch the disk directly it should be fine.

For settings I'd probably go for either a Control Panel applet, and/or per-application settings.

It's still a lot of work to do.
Andrew / Silver Blade

[ external image ]
Blackcrack
Posts: 1808
Joined: Tue Dec 20, 2005 12:55 pm
Contact:

Re: Win16 and DOS implementations

Post by Blackcrack »

Hy SilverBlade :)

humm... have you seen the gimp picture who i had make
on an other tread ? only as a try to get an exemple
play around with gimp , it have layers .. i dont now if you know it.. :

http://blackysgate.dyndns.org/~blackcra ... OS-VSM.xcf

i think, a little semi virtual mashin, who it is possibility to use diffrent bios
to support other "subsystems" like c=64 or with a P1/P2 Bios to use it for an dos..
only so as idea..

But until now, many thanks that you've explained to me,
let's see how this will look and funtion :D I'm very anxious/ strained on it :D

thank you very mutch :D

bests regards
Blacky
silverblade
Developer
Posts: 58
Joined: Sat Dec 18, 2004 9:41 pm
Location: Newbury, UK
Contact:

Re: Win16 and DOS implementations

Post by silverblade »

I would suggest using a virtual machine or emulator dedicated to each specific system.

There's no benefit in having a single VM that can be PC, C64, Amiga, Genesis or anything like that - each one is entirely different so they would ultimately need to be developed individually.

Systems are made up of more than just the BIOS ;)

That said, I have considered in the past the possibility of being able to load programs from old "retro" platforms seamlessly, in the same way as being able to run a 16-bit app on 32-bit Windows.

Emulators let you run programs from older platforms already, so it'd just be a case of making them integrate more into the platform they are running on. I'm not sure how that'd be best done.
Andrew / Silver Blade

[ external image ]
Lone_Rifle
Test Team
Posts: 802
Joined: Thu Apr 03, 2008 2:17 pm
Contact:

Re: Win16 and DOS implementations

Post by Lone_Rifle »

Taking a step back from the technical details, is there a real need to have such compatibility?
hto
Developer
Posts: 2193
Joined: Sun Oct 01, 2006 3:43 pm

Re: Win16 and DOS implementations

Post by hto »

silverblade wrote: This change basically means using Windows consoles for text-mode (though I am not sure how this would handle unusual characters and such.)
An option which allows to use a separate window instead of console?
silverblade wrote: I don't think DosBox lets you touch the bootsector of a disk directly? Again it isolates you from the actual hardware.
Maybe you can redirect access to the bootsector to a special file, to support programs which need it.
Lone_Rifle wrote: […] is there a real need to have such compatibility?
Bug 4041?
silverblade
Developer
Posts: 58
Joined: Sat Dec 18, 2004 9:41 pm
Location: Newbury, UK
Contact:

Re: Win16 and DOS implementations

Post by silverblade »

Lone_Rifle wrote:Taking a step back from the technical details, is there a real need to have such compatibility?
Aside from the bug mentioned previously, some people may want to run old 16-bit apps generally.

And it seems like an interesting thing to do / a good challenge.

Besides, since when was anything open-source 'needed'? ;)
Andrew / Silver Blade

[ external image ]
User avatar
Black_Fox
Posts: 1584
Joined: Fri Feb 15, 2008 9:44 pm
Location: Czechia

Re: Win16 and DOS implementations

Post by Black_Fox »

hto wrote:Bug 4041?
I don't want to put guesses into BugZilla - this taken from there shows that the driver is for XP, does it use Win 3.x functions or why do you assume it's "old"?
Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests