Debugging

This page describes different methods of debugging ReactOS and the steps necessary to debug ReactOS.

Introduction

To be able to help ReactOS development, whether this be participating in the development of the source code or taking part in crucial testing, you are going to need knowledge of how to generate useful debug logs.

Useful debug logs are essential pieces of information which the developer needs to quickly pinpoint and identify exactly what the operating system is doing. Many people know how to get default debug output from the operating system, but this is generally not particularly useful for locating problems, esspecially bugs.

This article aims to give users knowledge not only on how to generate a debug log, but on how to generate a useful debug log which can be used directly to asses what the operating system is doing.

Available debugging methods

There are various methods to debug ReactOS, some require more knowledge than others. These are listed below.

COM1

<add detail>

Physical serial cable

If you don't have a ready cable, you can build one for yourself like this one:

DTE1_______________________________________________DTE 2

9pol 25pol (female)_________________________________25pol 9pol (female)
5    7  ---GND---------------------GND-------  7   5

2    3  ---RxD--------. ,----------RxD-------  3   2
                       X
3    2  ---TxD--------' `----------TxD-------  2   3

7    4  ---RTS--------. ,----------RTS-------  4   7
                       X
8    5  ---CTS--------' `----------CTS-------  5   8

4   20  ---DTR--------. ,----------DTR------- 20   4
                       X
6    6  ---DSR--o-----' `-------o--DSR-------  6   6
                |               |
1    8  ---DCD--'               `--DCD-------  8   1

This is the most used variant of it and is the best for output debugging.

Debug Output to file

<add detail>

Debug Output to screen

<add detail>

Virtual machines

<add detail>

Named pipe

<add detail>

Riderect to file

<add detail>

kdbg

<add detail>

gdb

<add detail>

Knowing what debug output to generate

<add detail>

Turning on debugging in key modules

<add detail>

Turning on at compile time

<add detail>

Turning on at runtime

<add detail>

Forcing a bugcheck

Bug checks (or system crashes as they are also known) occur when the operating system can no longer operate safely and to avoid corrupting data, it halts operation. This will normally throw up a Blue Screen Of Death, but if you have the kernel debugger activated, it will drop you into a gdb prompt giving you access to explore the system.

There are two reasons for forcing a bugchecks, each reason employing a different method:

Dynamic

If you want to halt the system for any given reason and break into the the kernel debugger, you can force a bugcheck from the keyboard.

If you are not using a DBG build, you first need to add the following value in the registry:

Path:    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\i8042prt
Keyname: CrashOnCtrlScroll
Type:    REG_DWORD
Value:   1

Then, when ReactOS is running, hold down the right CTRL key, and press SCROLL LOCK key twice.

Static

This is useful when you want to halt the operating system when it hits a particular area of code. This is esspecially useful as you can get an immediate back trace to see where the code flow came before the bugcheck was forced.

For people who are seeing a particular bug, and who have the ability to follow the source code leading up to this, a forced bugcheck can be interted into the code.

This is done in kernel mode by calling the KeBugCheck or KeBugCheckEx, but it is more easily forced by adding: ASSERT(FALSE); into the code.

Again, if you have the kernel debugger activated, you will be dropped into a gdb session whereby you can issue the 'bt' command to get a back trace of the code flow previous to this bugcheck. This is esspecially useful to developers as it helps to pinpoint the buildup to the bugcheck and this can be followed in the source to gain a better understanding of what is happening.

Generating a backtrace

Before a backtrace can be initiated, you will first need to have broken into the kernel debugger. See the section forcing a bugcheck for directions on how to do this.

Once you are in the kernel debugger, the 'kdb:>' symbol will appear in your debug output. This is a prompt to recieve commands for the debugger.

If you are using a virtual machine, ensure that the VM still has keyboard focus as this is the machine in which kdb is running.

Enter 'bt' and you should see the letters appear in your debug output. If so, hit <enter> and you should see something similar to the following:

(drivers\filesystems\vfat\rw.c:809) <\ReactOS\system32\kernel32.dll>
Entered debugger on embedded INT3 at 0x0008:0x800935f2.
kdb:> bt
Eip:
<ntoskrnl.exe:935f3 (lib\rtl\i386\debug_asm.S:31 (DbgBreakPoint@0))>
Frames:
<vfatfs.sys:97de (drivers/filesystems/vfat/misc.c:111 (VfatDispatchRequest))>
<vfatfs.sys:9b25 (drivers/filesystems/vfat/misc.c:167 (VfatBuildRequest))>
<ntoskrnl.exe:3ab23 (ntoskrnl/io/iomgr/irp.c:1088 (IofCallDriver))>
<ntoskrnl.exe:36206 (ntoskrnl/io/iomgr/iofunc.c:686 (IoSynchronousPageWrite))>
<ntoskrnl.exe:59daa (ntoskrnl/mm/section.c:6330 (MmspWriteDataSectionPages))>
<ntoskrnl.exe:244c6 (ntoskrnl/ex/work.c:162 (ExpWorkerThreadEntryPoint))>
<ntoskrnl.exe:70e90 (ntoskrnl/ps/thread.c:134 (PspSystemThreadStartup))>
<ntoskrnl.exe:7b142 (ntoskrnl\ke\i386\ctxswitch.S:258 (KiThreadStartup@156))>
kdb:> 

Examining this in more detail, we can see that the bugcheck occured via the INT3 operation which then dropped us into kdb. The next line shows us Eip which is the instruction pointer, and this points to the last address before the system halted.

Following on from that are the frames. This is the important part of generating our backtrace and it contains all the function addresses in the buildup to the bugcheck. This is the crutial information developers need to understand the codeflow before the bugcheck.

Translating Addresses

Occasionally, there will come a time when you will need to manually translate addresses. When kdbg is not enabled and a bugcheck occurs, you will be presented with a stack trace similar to the following:

(subsystems\win32\csrss\win32csr\conio.c:1101) Console_Api Ctrl-C 

*** Fatal System Error: 0x00000001
                       (0x80079279,0x00000000,0x0000FFFF,0x00000000) 

<\SystemRoot\System32\NTOSKRNL.EXE: 29bb>
<\SystemRoot\System32\HAL.DLL: 4749>
<\SystemRoot\System32\NTOSKRNL.EXE: 54cb4>
<\SystemRoot\System32\NTOSKRNL.EXE: 582bf>
<\SystemRoot\System32\NTOSKRNL.EXE: 583fd>
<\SystemRoot\System32\NTOSKRNL.EXE: 89956>
<\SystemRoot\system32\drivers\videoprt.sys: 2417>
<\SystemRoot\system32\drivers\vbemp.sys: 17f5>
<\SystemRoot\system32\drivers\vbemp.sys: 19cf>
<\SystemRoot\system32\drivers\videoprt.sys: 1c48>
<\SystemRoot\System32\NTOSKRNL.EXE: 34c17>
<\SystemRoot\System32\NTOSKRNL.EXE: 21e0>
<\SystemRoot\System32\NTOSKRNL.EXE: 2908>
<\SystemRoot\System32\NTOSKRNL.EXE: 29bb>
<\SystemRoot\System32\NTOSKRNL.EXE: 85fa8>

As you can see, this is largely the same as the information presented when issuing a 'bt' command in kdbg. The problem here however, is that only the addresses are given. As these addresses are different for everyones builds, this information is useless for anyone trying to follow what events occured in the lead up towards to bugcheck.

The solution here is to translate the addresses into human readable function names. This is done via a tool named 'raddr2line' which is a modified version of the unix tool 'addr2line'. This tool will translate the addresses given to it into file names and line numbers. It does this by using debug information in the executable files to accociate the address with this human friendly info and outputs it into the console. This information can be pasted into the above debug log alongside the addresses providing the developers with a detailed stack trace.

radr2line is included in the Reactos Build Environment. It is invoked in the following way :

raddr2line <file> <address>

So taking the bottom address in the above stack trace :

C:\Users\Ged\MyFiles\ReactOS\clean_source>raddr2line ntoskrnl.exe 85fa8

C:\Users\Ged\MyFiles\ReactOS\clean_source\output-i386\ntoskrnl\ntoskrnl.exe
obj-i386\ntoskrnl\ex\zw.S:253 (ZwClearEvent)

we can see here that the address translation for 0x85fa8 is line 253 in file ntoskrnl\ex\zw.S (this will differ if you try it on your build)

This information can now be added into the above stack trace as follows :

<\SystemRoot\System32\NTOSKRNL.EXE: 29bb>    <enter next one here>
<\SystemRoot\System32\NTOSKRNL.EXE: 85fa8>   obj-i386\ntoskrnl\ex\zw.S:253 (ZwClearEvent)

Debugging preparation

One of the first things you need to decide is how to get the debugging output:

• Debug Output to serial port
  • Physical serial cable
  • VMware named pipe
  • VMware output to file
• Debug Output to file
• Debug Output to screen
• Bochs
• QEMU

Options for debugging reactos:

• gdb -- The GNU Debugger
• Wine Debugging Using Wine Debug Channels
• kdbg -- The built-in kernel debugger