ReactOS Remote Debugger
ReactOS Remote Debugger is a debugging shell that can take protocol modules (currently only for talking to KDBG), .nostrip files from the ReactOS build, and aggregate information in a way that is a bit impractical in KDBG. It is a Windows forms application (C#) with a number of window types, a simple poor man's dockability, and tabs.
It has a local variables window, backtrace, debug interaction, threads and processes. It can display a lot of info, and will become more useful as features are fleshed out.
Unofficial download link: Rev-42955.
Setting it up
You need a full build with
ROS_BUILDNOSTRIP=yes. Clean before building it the first time as already built (and up to date) modules will not be rebuilt. Do it like this:
make ROS_BUILDNOSTRIP=yes bootcd
Now that you've got symbols, select your source directory and output directory in the 'Extras|Settings...' menu of ReactOS Remote Debugger. Set the source directory to the 'reactos' directory that contains your sources. Set the output directory to the output-i386 directory you build into.
Make sure your freeldr.ini boots ReactOS like this:
Options=/DEBUG /DEBUGPORT=COM1 /KDSERIAL /BAUDRATE=115200
Actually, you can use any baud rate you want as long as your cabling allows a relatively low error rate. Optionally add /BREAK to break in early.
Newer builds of ReactOS already have a ReactOS (RosDbg) option in FreeLoader (/KDSERIAL only).
Using the debugger
Connect your null modem cable between com ports on your two computers and connect using the serial interface.
Virtual machine specific debugging details can be found on the VM pages:
TCP/IP (QEMU only)
to your QEMU command line.
Serial connection (QEMU, VMware, VirtualBox)
Use the serial port output together with the com0com driver (more here). The ReactOS Remote Debugger is your terminal application in this case.
Named pipe (QEMU, VMware, VirtualBox)
Serial port configuration in VMWARE:
If all goes well, connect to a running ReactOS instance and crash it (or hit Tab+K) and you'll break in in the debugger. The ReactOS Remote Debugger will request a number of updates from KDBG to see where things are, examine the stack, etc. If you don't see this traffic, the most likely reason is that you've booted without /KDSERIAL, and KDBG input is coming from the keyboard. Boot ReactOS with the "ReactOS (RosDbg)" option in this case.
If you see garbage and you're on a serial port, the most likely reason is that you've got ReactOS Remote Debugger and ReactOS set to different baud rates. We don't support auto baud setting yet, although somebody might be able to implement it fairly easily. It could also be dangerous at early /BREAK.
If all goes well, ReactOS will break in, ReactOSDbg will poke around, and display a number of line in backtrace (addresses followed by source file and line number), and will populate the locals and threads+processes data grids.
You can check that your locals appear by putting an __asm__("int3"); in your code. You should see all locally scoped variables and their values in the locals window when ReactOS breaks in.
Unlike KDBG, ReactOS Remote Debugger uses dbghelp.dll (the wine version, that reads stabs) to read the local copy of the .nostrip files that are the byproduct of the ROS_BUILDNOSTRIP=yes build. Unlike the abbreviated rossym sections in the canoncal binaries (suitable only for line number information), these contain all the information that's normally used by GDB to examine values in a running process. Since it doesn't rely on KDBG to lookup debug info, we could run a fully stripped build and still get debug info on the host (that is; debug a release build in a sensible way), and eventually, it'll be switched to kd protocol when ReactOS supports it. After that, it could be used to debug real windows, when given appropriate pdb files and the redistributable dbghelp.dll.
Notes and troubles
The most common complaint is that the channel back to ReactOS doesn't respond. You have to start ReactOS with the "ReactOS (RosDbg)" FreeLDR option. If there is no such option, you have to add /KDSERIAL to your freeldr.ini boot options manually.
KDBG, being a human interaction protocol, doesn't support retransmission. Luckily, ReactOSDbg is mostly stateless and responds to reports from KDBG as they become available. When in doubt, typing 'regs' and hitting return into the command prompt should get the essential updates flowing. It'll have a better automatic mechanism for that at some point in the future.