Debug Logs on Windows - The Easy Guide

[milon]

How to obtain a VirtualBox debug log on Windows
The Easy Guide

It's really hard figuring out how to make those mythical debug logs the devs keep asking for, so I decided to write a (hopefully!) easy to follow guide. (Sticky it if you think it's worth it.) Note that I'm using 64-bit Windows 7 and haven't tested with other Windows versions. The steps should be very similar for Vista and later. Earlier versions of Windows likely won't have to fight with getting unsigned drivers to install. This tutorial assumes you have some experience with using VirtualBox. While this guide is written specifically for Windows, the overall idea should be the same for Linux etc (just make sure to name your ports correctly).

Please also note that I'm typing this up after I have tried several things to make it all work. I may have inadvertently included an extra step or omitted a necessary step because of that. If you encounter problems, let me know and I'll update this post.


Overview
VirtualBox can send debug output to a COM port. PuTTY can listen to a COM port and write it to file. All we need is a "bridge" between the two and you've got your debug logs!


Software You'll Need
VirtualBox - you can use another VM but I have never used anything else and this tutorial is written specifically for VirtualBox.

A ReactOS Debug build. It'll be named something like "bootcd-#####-dbg.7z" or "bootcd-#####-dbgwin.7z". You MUST use a Debug build. You will also need 7-Zip to get the ReactOS ISO out of the .7z file.

PuTTY - This is the program that creates the debug logs. If you don't know what you're looking for, get the "For Windows on Intel x86" putty.exe file. It doesn't need to be installed; you just have to know where you downloaded it to.

com0com - This is our VirtualBox/PuTTY bridge. Very important. Install it. It gives you the option to have 2 COM port pairs, but you really only need 1 COM port pair. It's your choice. We're going to use Virtual Port Pair 0. You can rename them if you like by running com0com's setupg.exe (I use COM4 and COM5) but it really doesn't matter. You may have trouble installing its unsigned drivers, which brings us to...


Installing Unsigned Drivers on Windows Vista / 7 / etc
Again, earlier versions of Windows may not have this problem at all or may need to take different steps to make this work. I'll update this notice if anyone can provide me good feedback.
Windows Vista, 7, etc users will need to grab Driver Signature Enforcement Overrider. Run it (give it Admin privileges) then click the following buttons:
- Yes
- Next
- Enable Test Mode & Next
- OK
- Sign a System File & Next
Now Enter com0com's driver file. It'll be something like C:\Windows\system32\DRIVERS\com0com.sys
- OK
Now reboot your computer. You'll know it's working when your Device Manager shows 3 working drivers under "com0com - serial port emulators". You'll have 2 of "com0com - serial port emulator" and a third driver which for me is called "com0com - bus for serial port pair emulator 0 (COM4 <-> COM5)". Whatever name you gave the ports is what shows up in the driver's name. If you installed 2 port pairs instead of 1 port pair, you may see more com0com driver entries.


Configuring VirtualBox
Open up VirtualBox and select the virtual machine you want to test (again it MUST be a Debug build of ReactOS). Go to Settings -> Serial Ports -> Port 1. Check Enable Port 1. Make sure Port Number is set to COM1. No matter what you called your com0com ports, use COM1 here. For Port Mode choose Host Device. For Port File/Path type in the name of one of your com0com (virtual pair 0) ports. I use COM5. That's all there is to it for VirtualBox.


Configuring PuTTY
Open PuTTY. Make sure Session is selected on the TreeView on the left. In the main part of the window, under Connection Type, choose Serial. Under Serial Line type the name of your other com0com (virtual pair 0) port. I use COM4. Under speed, type 115200. Go to the TreeView on the left and click on Logging under Session. Under Session Logging, choose "All session output". Log file name indicates where the debug log will be and what it will be called. Use the Browse button to change the location if you like, but make sure you know where it's being saved! Go back to the TreeView on the left and click on Serial (under Connection). Set Flow Control to None. PuTTY is now configured. Note that on the main page of PuTTY you can Save/Load configurations which makes it even easier!


Getting that Debug Log
Finally! In PuTTY click on Open at the bottom of the window. This activates PuTTY's magical debug-log-creating abilities, which is a black window with a flashing green cursor. Open up VirtualBox and start your Debug ReactOS build. Make sure you choose the "ReactOS (debug)" option at boot! If you can see your PuTTY window in the background, you'll see lots of text flying past as PuTTY does its thing. When you've finished making ReactOS (or whatever it was you were aiming for), you'll have your debug log that is so coveted by the devs! Stick it in a bug report, upload it to the forum, mail it to your aunt, whatever. You've got your debug log!

One Last Thing
If you're going to run VirtualBox without PuTTY, make sure you disable Serial Port 1 again. VirtualBox (or ROS) does not like to have an open port if there's nothing to receive the data. It frequently results in BSOD or outright failure to boot.

[FlyingIsFun1217]

You may want to enter something in the title to indicate this is only for Windows.

FlyingIsFun1217

[milon]

Good idea, and done.

[SomeGuy]

Don't forget that for simple crash logs, having VirtualBox's com port just output to a file will often suffice. The only thing to remember is that one must blindly type "bt" and hit enter a few times when the crash occurs (long backtraces may pause at each "page" of output).

For problems that require more interactivity with the debugger, use of com0com and putty makes life much easier. Good tutorial!

[Webunny]

Another suggestion:

when you say "users will need to grab Driver Signature Enforcement Overrider." this will...well, it will work, but the problem is, that it gives watermarks with 'test mode' on your Vista, and no-one wants to have that forever on his screen. The 'remove watermark' option on that prog *does not* work on Vista SP2 anymore. So you either try it with starting Vista up in the mode that lets you test unsigned drivers (through F8 at startup), or you do use the prog, install the driver, and than bcdedit /set nointegritychecks off again (after you rebooted and Vista installed the drivers properly, of course).

Also, you have a spelling-error there; it should be: "The steps should be very similar for Vista and later."

Edit: while I think it's very good, I still think there could be some things been made easier/more simple. I would just ommit all talk about the portnumbers where they aren't really necessary, for instance. Only causes confusion. And the use of a few printscreens, maybe. But I guess this would be a good startingpoint for a revamp of the wiki.

[milon]

Don't forget that for simple crash logs, having VirtualBox's com port just output to a file will often suffice. The only thing to remember is that one must blindly type "bt" and hit enter a few times when the crash occurs (long backtraces may pause at each "page" of output).

For problems that require more interactivity with the debugger, use of com0com and putty makes life much easier. Good tutorial!

I read something about that somewhere, but I didn't pursue it when I read that VBox's output is kinda buggy. At least, I assume that a buggy debug log wouldn't help a whole lot.
Anyway, it sounds like it does have a use afterall. Instead of setting VBox's output to a COM port you would direct it to a file, yes? And how do you know when you're done pressing Enter? Does VBox alert you that you've created the full log?

Another suggestion:

when you say "users will need to grab Driver Signature Enforcement Overrider." this will...well, it will work, but the problem is, that it gives watermarks with 'test mode' on your Vista, and no-one wants to have that forever on his screen. The 'remove watermark' option on that prog *does not* work on Vista SP2 anymore. So you either try it with starting Vista up in the mode that lets you test unsigned drivers (through F8 at startup), or you do use the prog, install the driver, and than bcdedit /set nointegritychecks off again (after you rebooted and Vista installed the drivers properly, of course).

Also, you have a spelling-error there; it should be: "The steps should be very similar for Vista and later."

Edit: while I think it's very good, I still think there could be some things been made easier/more simple. I would just ommit all talk about the portnumbers where they aren't really necessary, for instance. Only causes confusion. And the use of a few printscreens, maybe. But I guess this would be a good startingpoint for a revamp of the wiki.

Fixed the typo, thanks. And I just noticed the watermark today - I've normally got a black background with icons around the edges, so I never saw it before. I'll look into that and update accordingly. At some point I want to "start over" so I can walk through the steps and simplify/streamline a bit. Screenshots are a good idea too. I'll keep it in mind, but it probably won't happen this week. If someone posts screenshots before I do, I'll incorporate them into my post. I'm hoping that ultimately, the 1st post will be good enough for a nearly direct copy & past into the wiki.

EDIT - Whoops! Just realized I didn't put this in the Tutorial forum. Thanks to whoever fixed that for me!

[Webunny]

Don't forget that for simple crash logs, having VirtualBox's com port just output to a file will often suffice. The only thing to remember is that one must blindly type "bt" and hit enter a few times when the crash occurs (long backtraces may pause at each "page" of output).

For problems that require more interactivity with the debugger, use of com0com and putty makes life much easier. Good tutorial!

I read something about that somewhere, but I didn't pursue it when I read that VBox's output is kinda buggy. At least, I assume that a buggy debug log wouldn't help a whole lot.
Anyway, it sounds like it does have a use afterall. Instead of setting VBox's output to a COM port you would direct it to a file, yes? And how do you know when you're done pressing Enter? Does VBox alert you that you've created the full log?

Another suggestion:

when you say "users will need to grab Driver Signature Enforcement Overrider." this will...well, it will work, but the problem is, that it gives watermarks with 'test mode' on your Vista, and no-one wants to have that forever on his screen. The 'remove watermark' option on that prog *does not* work on Vista SP2 anymore. So you either try it with starting Vista up in the mode that lets you test unsigned drivers (through F8 at startup), or you do use the prog, install the driver, and than bcdedit /set nointegritychecks off again (after you rebooted and Vista installed the drivers properly, of course).

Also, you have a spelling-error there; it should be: "The steps should be very similar for Vista and later."

Edit: while I think it's very good, I still think there could be some things been made easier/more simple. I would just ommit all talk about the portnumbers where they aren't really necessary, for instance. Only causes confusion. And the use of a few printscreens, maybe. But I guess this would be a good startingpoint for a revamp of the wiki.

Fixed the typo, thanks. And I just noticed the watermark today - I've normally got a black background with icons around the edges, so I never saw it before. I'll look into that and update accordingly. At some point I want to "start over" so I can walk through the steps and simplify/streamline a bit. Screenshots are a good idea too. I'll keep it in mind, but it probably won't happen this week. If someone posts screenshots before I do, I'll incorporate them into my post. I'm hoping that ultimately, the 1st post will be good enough for a nearly direct copy & past into the wiki.

EDIT - Whoops! Just realized I didn't put this in the Tutorial forum. Thanks to whoever fixed that for me!

Ermm... I already started with revamping the wiki based on your tutorial. I first made a new wikipage heavily based on your tutorial, but then I went back to have a look at the old one, and really, it ain't bad BAD, it just needs some tweaking. So I decided I had been a bit hasty (Do we really need yet another debug-logging page on the wiki, after all?) and now I'm in the middle of merging your tutorial with the one that was already there, so that the best parts of both remain, and the rest is discarded. I plan to place some printscreens there too, to give a visual aid to newbies, so that at least the virtualbox one will be very easy to follow.

Hope you don't mind I've already started with it!

[milon]

Even better! No, I don't mind at all. Less is more! Less work for me and less clutter for the ROS Wiki.
I noticed just shortly after finding this Tutorials forum that Black Fox has already written a tutorial on the same topic. He's got screenshots too! If you get his permission you could use his pictures in the wiki.

EDIT - Hey, could you post a link to it? I took a quick peek around and I can't find it just now...

[Webunny]

Even better! No, I don't mind at all. Less is more! Less work for me and less clutter for the ROS Wiki.
I noticed just shortly after finding this Tutorials forum that Black Fox has already written a tutorial on the same topic. He's got screenshots too! If you get his permission you could use his pictures in the wiki.

EDIT - Hey, could you post a link to it? I took a quick peek around and I can't find it just now...

Here you go: http://www.reactos.org/wiki/VirtualBox

I mostly finished it (text-wise, at least). Just need to put in some printscreens, and even an idiot...I mean, an IT-challenged person, can debug with Virtualbox.

They should redirect to it with the word 'debug' too, though.

Edit: yeah...that's the problems with the tutorials, I think. I'm sure many are made, but they are not well known. The first reference anyone gives, and look to, is the wiki. That's the obvious first area where you look (at least, it was with me), and it's also the first thing that is being linked too (like you guys did). So the impact there is the greatest.

[Black_Fox]

I noticed just shortly after finding this Tutorials forum that Black Fox has already written a tutorial on the same topic. He's got screenshots too! If you get his permission you could use his pictures in the wiki.

Hey there, my tutorial is to be used by the people. As long as:
- it's used for good cause
- someone reads it
- it actually helps someone

I don't object to using of any of its content, be it in other tutorial or wiki Thanks for letting me know.

[Webunny]

I noticed just shortly after finding this Tutorials forum that Black Fox has already written a tutorial on the same topic. He's got screenshots too! If you get his permission you could use his pictures in the wiki.

Hey there, my tutorial is to be used by the people. As long as:
- it's used for good cause
- someone reads it
- it actually helps someone

I don't object to using of any of its content, be it in other tutorial or wiki Thanks for letting me know.

Haha, luckily I chose the "\\.\pipe\ros_pipe" too.

Ok, those screenshots seem very useful. The only thing missing is one of the com0com, or did I overlook it?



Thanks, I'll use some of those, then.

[Black_Fox]

Good choice!
Regarding com0com, that is a different (and completely separate) way to achieve the same goal, but I chose the named pipes instead. From the tutorial itself: "This tutorial uses named pipes, for com0com serial tutorial look here: viewtopic.php?t=7867"

[milon]

I actually got a little lost with the named pipes. Isn't that a Linux thing? Or is that an actual file path? If you use com0com (Windows only) you can name whatever COM port it's simulating instead of a named pipe. More useful for intense outputs and VBox's can be buggy.

I also wonder if it would be useful to split the "Getting debug output" to a separate page, and leave the Setup part on the VirtualBox page. It might even be good to have one page for Windows debug output and another for Linux (and a third for Mac?). I'm just thinking about newbies getting scared off by the giant word-wall, and a little organizing might help there.

EDIT - Totally forgot to say that it is looking good, and it's already much better than the old one!

[Black_Fox]

I actually got a little lost with the named pipes. Isn't that a Linux thing?

No, named pipes are normal in Windows:

The Windows NT family of operating systems support named pipe clients and servers.

The wiki article is currently wrong IMO, the step 2 is totally redundant (and the whole block is called "Redirect to virtual serial port (com0com, Windows host)" instead of "Redirect to named pipe (Windows host)". If I set up a named pipe in VBox and then connect to it with Putty, why would I need to emulate COM ports?

[milon]

It sounds like you either use COM port emulating (com0com) or named pipes. That's another point that could be cleared up on the wiki.