64 bit

[Z98]

The reason to bother with a 32bit version is mainly drivers. Older stuff will not have 64bit drivers. Applications can care less if you have a compatibility layer like what Windows has. Any argument for dropping a particular platform would be for in favor of dropping the 32bit version of the OS, since there is little reason beyond old hardware drivers to not move to the 64bit version.

A 128bit processor would be useful for reasons beyond the memory addressable. Its primary advantage would be its ability to operate on larger values, which is especially important for any major number crunching operation like in cryptology or scientific computing. And there are applications out there that will bring a very high end 8 core, 64GB RAM system to its knees. These programs also tend to cost something like thousands if not tens of thousands per license seat, but they exist. Programmers will find a way to take advantage of any bit of processing power that is made available.

Keep in mind that for all intents and purposes, the differences between x64 and x86 are a bit more fundamental than just one being 64bit and the other 32bit. The size of the native values matters a lot less than the differences in the instructions and the interface presented by the individual platforms. So porting an x86 OS to x64 is not that much different conceptually from porting an x86 OS to ARM. Don't get caught up on the value size differences.

[platatomi]

64-bit will be very important but 32-bit support should also be maintained because Windows 8's success will not support 32-bit. Therefore, it would be very advantageous for ReactOS to support 32-bit machines as well.

[PurpleGurl]

Well, if we abandon 32-bit, then we may have to do something like a WOW64 wrapper to help 32-bit stuff to run seamlessly if we want full compatibility. But it makes more sense if we want to run primarily 32-bit stuff to have a 32-bit version and not emulate it. That is sufficient for most things out there. We should work on 64-bit since that is where things are going, but I don't see it as time to give up on 32-bit. I would love to see some 16-bit emulation, but that is a minor thing.

As for memory, you can address up to 64 GB with PAE extensions even on a 32-bit system. You would need that mode anyway if you want to run the no-execute instructions to prevent buffer overruns, even if you decide to block memory over 4 GB for compatibility reasons. I think XP RTM or maybe SP 1 actually allowed more than 4 GB, but there were problems.

A summary of this thread probably should be added as a point in one of the sticky messages. 64-bit questions are common, and people confuse the 4 GB limit (which is voluntarily imposed for compatibility) with 64-bit instructions and register usage.

Theoretically, it might be possible to emulate 64-bit on 32-bit, but things would be so horrible and slow that such an endeavor would be pointless. For instance, I know a bit about 8087 emulation and processing long integers on 286 and 8088 processors. Prior to the 486 DX, FPU packages were optional, and most people didn't have them. 32-bit long division was possible on a 16-bit CPU without a FPU, but guess how much code it took to process everything as a string and to hold the result in memory because there were no registers that would hold it? About 6 pages of assembly. So that was one problem with software on 16-bit processors without math chips. Another was the lack of instructions. So most programs that needed FPU-type functions added a 10-20K library which emulated them. Most used an interrupt system to implement them. The code was written to where exceptions were assumed and the illegal instructions were redirected to software interrupts which contained the emulator code. Now, in case the instructions were valid, there was a routine to test for an 80x87 chip. If it existed, then there was self-modifying code to patch out the redirection to the software interrupts the library installed in memory, allowing the hardware to be used rather than the emulator. One instruction was never emulated, and that was FWAIT. In single-threaded real mode, there was no possibility of a CPU vs. FPU race condition if the FPU was merely an emulator (the CPU would inherently have to wait on itself to finish). Wow, those were the old days!

Another confusion I'd like to clear up is that memory address bit size is independent of processing bit size, and independent of data bus size. For instance, the 8088 was a 16-bit processor (internally), but had an 8-bit data bus (had to take 2 passes to transfer data - which was quite a bottle neck) and 20 address bits. Address bits are what determine the memory addressing. In 286 to Pentium 1, there was the whole A20 dilemma. To keep real mode compatibility, only the first 20 address bits could be used. So to go in and out of protected mode, switching the A20 line was part of it. At the time, most used an unused channel in the onboard keyboard controller to toggle the line. Other manufacturers moved the A20 switching to the chipset since it could be done faster there, and a CMOS option added in case you had to do it the old way for compatibility. If I remember right, most 32-bit processors now (and 64-bits running in 32-bit mode) have 32-bit instructions and registers, 32-bit data bus, but a 36-bit address bus.

[Ohnemichel]

Why can't you just keep 32-bit drivers run under an 64-bit environment? As the 64 bit drivers are just PE executables and 64 bit drivers won't be programmed for some of older hardware, ReactOS would be able of bridging a gap here. From a processors point of view, it won't be a problem because of the processors' compatibility mode. As far as I know, the drivers don't receive memory addresses, but just data streams.
So the interface that transfers commands from 32 bit applications to 64 bit drivers could be easily used as a template for an interface that does the reverse.
Even if adresses would have to be accessed from the entire memory, you could just page the corresponding positions down to below 4 GB. (or better down to 2 GB, then you would be sure that the memory area's length would surely fit inside the 32 bit space).
Of course, the drivers would be needed to be placed below the 4 GB limit.

As far as I understand, Microsoft only doesn't support 32 bit drivers in order to force the driver programmers to digitally verify their driver. Or is there another, technical reason why a driver can't be run in compatibility mode? Am I wrong with that >data is only transfered by datastreams< oppinion?


greetings

- Ohnemichel

[PurpleGurl]

Why can't you just keep 32-bit drivers run under an 64-bit environment? As the 64 bit drivers are just PE executables and 64 bit drivers won't be programmed for some of older hardware, ReactOS would be able of bridging a gap here. From a processors point of view, it won't be a problem because of the processors' compatibility mode. As far as I know, the drivers don't receive memory addresses, but just data streams.
So the interface that transfers commands from 32 bit applications to 64 bit drivers could be easily used as a template for an interface that does the reverse.
Even if adresses would have to be accessed from the entire memory, you could just page the corresponding positions down to below 4 GB. (or better down to 2 GB, then you would be sure that the memory area's length would surely fit inside the 32 bit space).
Of course, the drivers would be needed to be placed below the 4 GB limit.

As far as I understand, Microsoft only doesn't support 32 bit drivers in order to force the driver programmers to digitally verify their driver. Or is there another, technical reason why a driver can't be run in compatibility mode? Am I wrong with that >data is only transfered by datastreams< oppinion?


greetings

- Ohnemichel

I think a good part of it has to do with some of the older drivers using the areas above 3GB for their own addressing, that it could make everything unstable in the higher memory. I imagine there is more to it than that, and maybe a dev will come in and explain that. I really don't see why there couldn't be a wrapper of sorts, but wouldn't that harm performance?

[milon]

I really don't see why there couldn't be a wrapper of sorts, but wouldn't that harm performance?

Even if performance is harmed, that's still a step up from no performance - which is what Microsoft has right now. No performance of old hardware/drivers with new software. It would be pretty awesome if ROS worked where MS doesn't.

[Z98]

32bit drivers are expecting memory addresses to be given to them in 32bits. They have no provision for when the OS hands them an address in 64bits. The most obvious thing that goes wrong is the driver starts writing to the wrong memory address, which can end up corrupting data and crash the OS. Then there are the differing page sizes between 32bit and 64bit Intel/AMD processors. In kernel-mode, a "wrapper" is pretty much useless. Drivers basically have direct access to hardware or are expecting that direct access, which means 32bit drivers expect to see things a certain way. In addition to differing address sizes, interrupts are also different between 32 and 64bit modes. For a "wrapper," you would literally need to run a virtual machine of sorts to map the differences, and this goes well beyond "minimal performance degradation." The industry as a whole pretty much decided this kind of compatibility was not worth the effort and so have not bothered. Considering this isn't just Windows, but also Linux and OS X that didn't bother, at this point I'm pretty sure no one would care to bother reallocating resources that could be better spent on other features to this.

[PurpleGurl]

I agree with Z98, we are on the right track. Those who don't have 64-bit drivers can use the 32-bit Reactos. At least they can run something, and there is plenty of older software. I imagine, we would be able to keep the 32-bit software alive for longer. If there is a market for something, people will make and sell it.

Too bad the chip makers didn't make the switch to 64-bit more seamless. Going from 16-bit to 32-bit was seamless. There were no separate bit-based processing modes per se (sure, there was protected mode and virtual mode, but those didn't prevent 32-bit instructions and registers from being used in 16-bit code). A 32-bit CPU can run pure 16-bit code natively. That code simply doesn't use the 386+ features nor instructions. Software simply queried the CPU and FPU to see what they could do and executed the portions for them, or they would bail if the requirements were not met. So I was surprised that the 32-bit and 64-bit were made separate modes, and not as easy as testing for 64-bit capabilities and then branching to the appropriate code. 32-bit mode doesn't allow for you to use 64-bit specific instructions and registers. Too bad it isn't that simple. If it was 16-bit code, I could test for a 386+ CPU and if there, and use the extended (32-bit) registers.

Anyway, I agree that our efforts should be put into the current goals of getting Reactos functional and ready for daily use.

[Z98]

I'll point out that the problems you're describing are not rooted in 32bit vs 64bit. They're rooted in the fact that AMD64 is a superset of IA32. That AMD64 brought us 64bit processors may be the primary point of AMD64, the 64bit aspect is again not the biggest cause of the incompatibilities. Also, the transition between 16bit to 32bit faced the same driver issues going from 32bit to 64bit is seeing/settling now. The 16bit transition was probably even rougher considering some developers were too used to direct hardware/memory access and had trouble dealing with getting code to work in protected mode.

[PurpleGurl]

I'll point out that the problems you're describing are not rooted in 32bit vs 64bit. They're rooted in the fact that AMD64 is a superset of IA32. That AMD64 brought us 64bit processors may be the primary point of AMD64, the 64bit aspect is again not the biggest cause of the incompatibilities. Also, the transition between 16bit to 32bit faced the same driver issues going from 32bit to 64bit is seeing/settling now. The 16bit transition was probably even rougher considering some developers were too used to direct hardware/memory access and had trouble dealing with getting code to work in protected mode.

I understand what you are saying, and you are correct. I appreciate your wealth of information. I quit coding when MS-DOS finally went completely out of favor, so it is always nice to chat with someone who knows about more recent architecture. Maybe I could take up coding again, but I have a lot to catch up on.

It seems AMD, even while trying to keep things compatible with the legacy way of doing things, they still came out with a new standard. Sure, it can emulate the legacy way, but not while adding 64-bit registers, 64-bit addressing commands, etc. Comparing to the V20 might be more fair. The V20 was NEC's clone of the 8088 (and V30 was an 8086 clone). But since their CMOS technology enabled them to cram more on the die and use less power, they decided to also add 8080 instructions. While Intel tried to keep some semblance to the 8080 in the 8086/8088, NEC added a true compatibility mode. That is relevant to today since Intel did something similar with the Itanium. It has a true 32-bit core in with the 64-bit cores.

Why couldn't they have instead with the AMD 64 to make it where there was one mode, 64-bit, but have it default to 32-bit addressing that could be changed on the fly?

[SomeGuy]

Actually, we are quite lucky to still have any kind of x86 compatiblity at all in modern CPUs. A way while back Intel was suggesting they might compleatly ditch x86 in favor of the IA-64 (Itanium) instruction set. Shortly after that Microsoft started pushing ".NET" with one of the main ideas being that apps written in that could be CPU independent.

Fortunately AMD came out with the AMD64 which at least lets OSes run 32 and 16 bit application code from 64 bit mode and Intel adopted it. Although I strongly suspect Microsoft played a big roll in that.

Additional compatibility means more transistors, more power used, and more heat. They had to draw a line somewhere.

> "Going from 16-bit to 32-bit was seamless."

Not as seamless as you might think. There was never a "16 bit" version of Windows NT, so nobody had to deal with driver migration issues there.

What everyone was using at the time was Windows 3.x and 9x. And despite how Microsoft sold it, Windows 95 was actually somewhat a 16-bit OS at it's core. You can think of Windows 95 as Windows 3.1 with a more complete Win32s application layer built on top of it. (Yes, Win3.1 could run some Win32 apps using an addon) Look at the original Chicago releases and everything was 100% 16-bit. Mostly only a UI change over Windows 3.1. By the final 95 release most of the high level applications were converted to "32-bit" but it still ran on top of dos, and used 16-bit drivers for things like printers. The upshot was most existing Windows 3.1 drivers and applications ran perfectly under Windows 95.

If ReactOS were to take an approach like that, you would mostly be talking about a 32-bit OS that could somehow run 64-bit applications.

[Z98]

AMD64's ability to run 32bit code was never in doubt and works perfectly fine. Assuming I'm interpreting your point correctly, you're dissatisfied with x86 processors' inability to run x64 code. Forward compatibility is generally much more difficult to design for and since Intel did not make any provisions for it, at least as far as I know, there isn't a mechanism to allow for easy "translation" of x64 instructions to the x86 subset without outside help like a virtual machine. You suggested basically probing a processor to see what it supports and then switching between two versions at runtime. Most of the industry seems to think this is not worth the performance hit, since this would result in an inflated in-memory binary. Even worse would be the effect something like this might have on cache thrashing since the instructions would not be nearly as sequential compared to a purely x64 binary.

AMD changed the format of the page table/entries when they put together AMD64, into a format that baseline x86 would not understand, though this is tangential to the above point. I'm not quite sure I understand what you think would be achieved by having x64 processors do addressing in 32bits, since x64 running in 64bit mode has no problem running 32bit processes. I suppose this might mean you don't need 64bit drivers, but I'd take the expanded memory address space over the ability to run 32bit drivers. If something only has 32bit drivers, it is either highly specialized (like way too many industrial and scientific instruments) or very old, at which point the driver itself is of increasingly questionable utility due to the changes in kernel interfaces.

[PurpleGurl]

AMD64's ability to run 32bit code was never in doubt and works perfectly fine. Assuming I'm interpreting your point correctly, you're dissatisfied with x86 processors' inability to run x64 code. Forward compatibility is generally much more difficult to design for and since Intel did not make any provisions for it, at least as far as I know, there isn't a mechanism to allow for easy "translation" of x64 instructions to the x86 subset without outside help like a virtual machine. You suggested basically probing a processor to see what it supports and then switching between two versions at runtime. Most of the industry seems to think this is not worth the performance hit, since this would result in an inflated in-memory binary. Even worse would be the effect something like this might have on cache thrashing since the instructions would not be nearly as sequential compared to a purely x64 binary.

AMD changed the format of the page table/entries when they put together AMD64, into a format that baseline x86 would not understand, though this is tangential to the above point. I'm not quite sure I understand what you think would be achieved by having x64 processors do addressing in 32bits, since x64 running in 64bit mode has no problem running 32bit processes. I suppose this might mean you don't need 64bit drivers, but I'd take the expanded memory address space over the ability to run 32bit drivers. If something only has 32bit drivers, it is either highly specialized (like way too many industrial and scientific instruments) or very old, at which point the driver itself is of increasingly questionable utility due to the changes in kernel interfaces.

Fascinating, but you still don't get me. I am not discussing what x86 can do, but the 64-bit chip's capabilities while in 32-bit mode. You can write code that will work on a 286, and test for a 386+, and use those instructions and registers without changing to a mode that the 286 cannot run. With AMD 64, they didn't give us that option. It works like a regular Athlon (think Barton or Palomino core) in 32-bit mode and cannot access extra memory addresses, extra instructions, nor use 64-bit registers.

Of course, one thing I didn't make clear was that I was thinking more of real mode. Running 16-bit code, the 286 and 386 are mostly the same, except the software could ask if it is a 386, and if so, use the extended registers and 386-specific real mode instructions, otherwise, fall into 286 code. Of course, there were huge differences in protected modes between them. The 386 was made to switch in and out of protected mode at will (in addition to acquiring virtual mode and so on), where the 286 was not. It was intended for programmers to go to the flat plane addressing mode and access "all that ram" (up to whatever MBs), and not return to the segmented mode. But programmers had other ideas, hence the whole A20 dilemma and the reason for a way to transfer back.

So my dismay was due to the AMD not letting you use a hybrid sort of mode where you can still use special registers and commands while in 32-bit mode. If they had done that, then regular Reactos and other 32-bit software could take advantage of wider registers and more instructions. Now do you follow?

[Z98]

I'd say that decision by AMD was part market part technical decision. Adding in the new registers and instructions would mean that code compiled against the new ISA would not run on vanilla x86. Creating another ISA on top of existing ones is a difficult thing to sell to the market, unless you have really, really compelling features. AMD64's main selling point was extending the x86 ISA to 64bit. AMD took the opportunity to try and fix a lot of what was ugly about x86, but "fixing" the ugliness of x86 was probably not the primary objective. AMD64 succeeded in the marketplace because it added a lot of value while also maintaining backwards compatibility to x86. The market sought a 64bit solution that was backwards compatible with x86. I'm not sure there was much demand for an x86-like ISA that did not overcome the 32bit limitation.

[PurpleGurl]

I'd say that decision by AMD was part market part technical decision. Adding in the new registers and instructions would mean that code compiled against the new ISA would not run on vanilla x86. Creating another ISA on top of existing ones is a difficult thing to sell to the market, unless you have really, really compelling features. AMD64's main selling point was extending the x86 ISA to 64bit. AMD took the opportunity to try and fix a lot of what was ugly about x86, but "fixing" the ugliness of x86 was probably not the primary objective. AMD64 succeeded in the marketplace because it added a lot of value while also maintaining backwards compatibility to x86. The market sought a 64bit solution that was backwards compatible with x86. I'm not sure there was much demand for an x86-like ISA that did not overcome the 32bit limitation.

I understand that much. But I don't see why they couldn't have added the new features to the 32-bit portion too or allow spillover between the modes. The think is just having the new registers and options there in the 32-bit mode, and good coders would have the option to add multiple code paths for crucial loops. So it would not be a new ISA, but an extended one. Whether that would break compatibility would be up to each programmer. Using the 286 to 386 example again, it would be possible to add a long-division library *and* test for a CPU with native long division support, mixing compatibility with the old and the performance of the new.

But I do see another possible issue. For all we know, they could have been running low on opcodes. Being variable length complicates things, since shorter ones couldn't be incorporated into the start of others (well, not easily). Variable length instructions are one of the curses of the architecture.