AHK Test Suite

From ReactOS Wiki
Revision as of 04:57, 10 April 2013 by EDIJS (talk | contribs) (How to write new tests)
Jump to: navigation, search

The page AHK Test Suite is a short article on something that should have a lot more information written on it.
If you know anything else about it, you are STRONGLY encouraged to add the information in. If you are unsure of proper formatting or style, add it to the talk page or this page itself as you think best and others will help.


What is AHK Test Suite

AHK Test Suite is testing framework written in AutoHotKey scripting language of goal to test applications automatically in ReactOS operating system.

How to download AHK Test Suite

...

How to see test results

Online tests

If you want to see results of online tests you need:
1) Enter http://reactos.org/testman/ in your browser address;
2) Specify ReactOS revision number you want to check results of;
3) Enter 'CMake_x86_GCCWin' in 'Source' field;
4) Select up to five revisions using checkboxes provided;
5) Hit 'Compare selected Results' button.
Picture below demonstrates all the steps.
https://sites.google.com/site/thricesoftas/Home/stuff/reactos/testman.jpg
Scroll down the newly opened page until you see tests that start with 'zz_' prefix.
https://sites.google.com/site/thricesoftas/Home/stuff/reactos/Test_results.jpg

Compiling and running test manually

In order to run test you need to compile it. First, download the test (how to do so check titles above). When download is done, you need to compile using Ahk2Exe.exe (AutoHotKey v1.0.48.05 is used when compiling ReactOS tests). If you are planing to compile using GUI tool, there is the steps:
1) Run Ahk2Exe.exe;
2) When Ahk2Exe window appears, hit "Browse" button in in "Source (script file)" and open file that matches 'zz_*_test.ahk' file pattern (e.g. 'zz_firefox_2.0.0.20_test.ahk');
3) Hit "> Convert <" button and dialog box saying 'Conversion complete' will appear.
Picture below demonstrates all the steps.
https://sites.google.com/site/thricesoftas/Home/stuff/reactos/Ahk2Exe.jpg
If compilation went successfully a 'zz_*_test.exe' (e.g. 'zz_firefox_2.0.0.20_test.exe') file will be created. There are dependencies. You gonna need exact the same test application version as described in test (e.g. 'Firefox 2.0.0.20'). Sadly, we do not provide any host links to test media we have for AHK tests. In case you managed to get everything you need (asked Caemyr, Edijus?) you need to create two folders in target machine 'Apps' and 'Media'. You put test program *.exe (e.g. 'Firefox 2.0.0.20 Setup.exe') in 'Apps' and rest of stuff (e.g. 'FF_2_Search_Arrow.jpg') in 'Media' folder. The compiled script executable goes into root folder.
Picture below demonstrates folder hierarchy.
https://sites.google.com/site/thricesoftas/Home/stuff/reactos/Folders.jpg
Now, all you need is to start test executable by passing test name via command line (e.g. 'D:\AHK_Tests\zz_firefox_2.0.0.20_test.exe 1.install').
https://sites.google.com/site/thricesoftas/Home/stuff/reactos/Start_Test_CMD.jpg
Results will be displayed in debug output. Debug output can be monitored with DebugView application.
Picture below demonstrates how results are displayed in DebugView application.
https://sites.google.com/site/thricesoftas/Home/stuff/reactos/DebugView.jpg

How to write new tests

If you want to write test for completely new application, use those as templates:

Create two folders: 1 - Application name, 2 - Application version. E.g.: Firefox\2.0.0.20
This is so we could have same application tested, but different version e.g. Firefox\2.0.0.20, Firefox\3.0.11, Firefox\12.0, etc.
There are different ways of making application to do things. You can use mouse to click controls and you can use keyboard. Sending keystrokes is not most reliable thing to do, but sometimes applications haves no control names. In such case (e.g. Firefox 12.0 case) send keystrokes (e.g. Alt+N to click 'Next' button, etc.).
Tests for installers: first of all you have to remove all previous versions of applications. Use uninstaller with silent switches. If not possible, just delete hard-coded default path and settings if any.
Lets say some window takes long to close (some installers takes some time to extract files). Do not use something like WinWaitClose, Window Caption, Window Text, 60, because window may never close because of bugs in ReactOS code and some window just pop-out. Instead, write a loop. Something like this:

TestsInfo("'Window Caption (Window Tex)' window appeared, waiting for it to close.")
iTimeOut := 60
while iTimeOut > 0
{
    IfWinActive, Window Caption, Window Text    {
        WinWaitClose, Window Caption, Window Tex, 1
        iTimeOut--
    }
    else
    {
        WinGetActiveTitle, ActiveWndTitle
        TestsInfo("'" ActiveWndTitle "' window poped-up.")
        break ; exit the loop if something poped-up
    }
}

WinWaitClose, Window Caption, Window Tex, 1
if ErrorLevel
    TestsFailed("'Window Caption (Window Tex)' window failed to close (iTimeOut=" iTimeOut ").")
else
    TestsOK("'Window Caption (Window Tex)' window closed (iTimeOut=" iTimeOut ").")

We want tests to consume as less time as possible. sysreg will time out ([SYSREG] timeout) if there is no debug output in 60 seconds.