[ros-kernel] ReactOS Roadmap

[KJK::Hyperion]

At 06.41 15/10/2003, you wrote:
>>12-18 months: getting rid of the CSRSS monstrosity for good.
>[...]
>Well put, as usual. :-)  It's probably way too early to worry about this 
>sort of thing, but what *do* you propose instead of the csrss-style 
>architecture?

The consoles will become true devices. The console driver will still be 
just a bridge between client applications and a server, but this shouldn't 
be understimated. On the slave side there *has* to be a server process, 
because it's just better, and if said server process can be user-defined 
(plain impossible with CSRSS - if not for anything else, no way you could 
signal a console client from a console server running under a different 
account) it's even better. Using I/O for inter-process communication *may* 
be even less efficient than LPC, but there's the big advantage that I/O is 
exponentially easier to use in kernel-mode, and supporting a kernel-mode 
console server is vital (imagine the headless server scenario. A simple 
kernel-mode console server is efficient - cutting on all the fancy 
user-friendliness that a headless server rarely needs - and removes the 
need for win32k.sys, while still exposing a decently usable console). 
Console handles, as a side effect, will be true, full-fledged handles, with 
correct handle semantics. Ever wondered why the runas command always opens 
a new console? because console handles aren't true handles, that's why (see 
"half-assed attempts at a microkernel"). True handles mean that, when 
switching from text-only mode to GUI, the text-mode console server could 
spawn a child process (the GUI console server) and hand over the ownership 
of the slave side to it by simply duplicating its handle - thus preserving 
all the consoles you opened in text mode. True handles mean you could 
attach to any console and write to it - message broadcasts in text mode 
anyone? Does this sound a bit like UNIX? good, because that's the point

Preserving the old console control semantics (creating a dispatcher thread 
that goes through the list of registered handlers) won't be hard, once you 
remove the requirement for threads to be registered with CSRSS: you just 
create the thread. End of the story. Threads created from outside 
kernel32.dll will lack the default exception handler, so what? if you're 
really interested in it, you write it by yourself - the default exception 
filter does all the magic, and it's a public function. The real point is 
that you can happily call Win32 functions (and - especially - write to 
consoles) with no special initialization steps. On top of that, since I 
haven't abandoned my old dream of a POSIX subsystem, this will make things 
much easier for my successor - let's not go into what unholy steps are to 
be taken on Windows to trick CSRSS into accepting a Win32 process that 
wasn't started, in turn, from a Win32 process (let's just say it involves a 
disassembler - in your code, not as a development tool - and a lot of 
testing - two things I still haven't been successful at forcing myself to). 
Really, ReactOS will be a much better place without CSRSS. The birds will 
sing, the sun will shine and the processes will roam free, regardless of 
age, gender, religion or subsystem

No CSRSS means nothing that must be started before you can start anything 
else. No CSRSS, nor SMSS: if I want the kernel to run a batch file at 
startup instead of SMSS, I should be able to - in fact, a boot-time script 
should be the *default*. Boot time activities, such as loading the well 
known DLLs, creating the swap files, saving the crash dump to a file, etc. 
will be assigned to startup scripts and applications, to allow a better 
granularity of configuration. The whole concept of a "boot verification 
program" will be obsolete. You will be able to decrypt the SAM with a 
custom procedure, instead of being limited to a measly three options (see 
SysKey), none of which sounds all that exciting (what if you want to use 
SHA-1 + AES? what if you want to use a smart card? etc.). Winlogon and the 
SCM will be two normal user-mode applications, with nothing really special 
about them: you'll be able to start and stop them at will, or kill them if 
they misbehave (all too common for such a complex beast as Winlogon - and I 
speak from experience) - or, heck, not run them at all. Hey, what I was 
thinking? the setup program could be a Win32 app! To whomever is currently 
developing it: imagine being able to debug USetup in Visual Studio (or your 
favorite debugger/IDE), instead of... whatever you're using now (with, I 
think, much pain). You could use format and fdisk (maybe implementing a 
console server in USetup to better control them), instead of duplicating 
their features inside USetup, and the future transition to a GUI installer 
would be seamless. Also, phasing out CSRSS alone could be the single 
biggest step towards the mythical one-phase setup. Does this sound more and 
more like UNIX? no, make that "like any decent operating system"

Hard errors will be handled per-thread, with callback semantics, like they 
are meant to be (except those with an action of ShutdownSystem, that are 
meant to cause a controlled BSOD). The PEB has room for several callback 
pointers: it won't hurt to reserve one for hard errors, right? and an extra 
one called by win32k.sys to convert a thread into a GUI thread, that could 
be handled in user32.dll by, among other things, replacing the hard error 
handler with one that shows a message box (possibly thread-modal, certainly 
not broadcasted). For those unsure: yes, it has to be a callback - it's 
vital for out ntdll!Ki* symbols to strictly match those of Windows, because 
the awareness of the hidden, locked potential of the Native layer is 
growing, and it would be gratuitous to change well-known stuff "just 
because". Having few, predictable entry points from which execution can 
resume is good design, too: just look at all the pains the Valgrind team is 
going through to catch all signals, lacking Linux well-defined points at 
which execution resumes (not counting the NtContinue and NtSetContextThread 
anomalies, Windows has exactly four - namely, user-mode APC, callback, 
exception and system exception. Five, if we count LdrInitializeThunk in)

CSRSS as a host for win32k.sys kernel-mode worker threads (no, really, I'm 
not making this up: it's documented in "Inside Microsoft Windows 2000". Try 
to terminate CSRSS, just for fun: observe how the GUI freezes) is useless. 
win32k.sys can either exploit the under-used System process (it typically 
has just less than 500 KBs of virtual memory), or fork it with 
PsCreateSystemProcess

CSRSS as a debugging proxy server is an outdated concept: Windows XP and 
later have abandoned the old LPC-based debugging API in favor of a more 
traditional object-oriented native API, and I mean to do exactly the same

>Goal #1 (and #2 and #2) has to be windows app compatibility, in my book.

removing CSRSS will give you backward *and* forward compatibility. 
"Forward" because you'll be able to use Windows applications in contexts 
where it used to be impossible. I really don't care how hard it will be to 
move the consoles to a driver: it just begs to be done, and it's the key to 
unlock a better Windows

>Also, a tangential point:  you had mentioned to me several months ago that 
>you had looked into kd support;

sorry to disappoint you, but that doesn't sound like something I would say. 
kd is black magic to me. I have no idea about the remote debugging 
protocol, nor the kernel-mode voodoo required to place breakpoints, alter 
thread contexts, etc. Furthermore, most features of kd extensions depend on 
non-public symbols and data structures, so we'd have to reimplement those, 
like ReactOS was a new release of Windows. A possible, alternate debugging 
strategy? somehow making the ReactOS kernel run in Windows, possibly as an 
user-mode virtual machine, UM-Linux style. Nah, overkill. I'm sure our 
kernel debugging genius is out there, somewhere, and has a better idea (who 
ported Private Ice, again?)