Testing Central

From ReactOS Wiki
Revision as of 21:46, 14 March 2013 by Black Fox (talk | contribs) (GOLDENAPPS: Notepad Lite 3.3.1)
Jump to: navigation, search

Welcome to the Testing Central! Here we can coordinate our efforts, this is useful in case you don't have access to or aren't in IRC. To fill the tables just follow the easy understandable guidelines. If you don't understand them please ask in the IRC channel #reactos-testers. Thanks!

Just getting started with testing? Head over to the Testing Introduction page

DEVELOPERS REQUEST

Here any developer can ask testers for a test: i.e. a testbox, an application, or a patch. The request can be fulfilled just by a developer. Testers can fill in the result report. While you are about to test a revision, change the tester field and set the LRT to T+number(i.e. T40020) so other testers know you are testing this revision. After testing and knowing the result change the T to R (i.e. R40020) and update the result field.

  • DEV: Name of the developer who asked for a test.
  • Request: A short description ( i.e. Test for FF mouse bug).
  • Application/TestBox: A link to the application (if needed), please use Wiki syntax.
  • Way of Testing: A link to a Bugzilla report where the testing steps are described or a short overview of how to test. Please try to keep it short.
  • Tester: The name of the tester who did the testing.
  • LRT (Last Revision Tested): Add a letter before the number, T means currently testing, R means tested
  • RES: Results of the test: failed or no failure.


Dev Request App/TestBox Way of Testing Tester LRT Res:
Devname Short Description Download it:HERE Open the menu and check if font spacing still wrong, etc. TestrNam r40xxx FAIL
rafalh Test newest UniATA 0.41b2 (without ReactOS specific hacks) http://www.megaupload.com/?d=5ZZF478R Check for regressions in real hardware and all virtual machines. - - -

REGRESSION TESTING

Here any developer or tester who finds a regression can add a line, so that testers can easily keep track of running regression tests and to allow the coordination of the efforts. If you discovered the regression then add your name, the regression description and in which Virtual Machines did you test it. Also add a revision where the bug was not present and the failing one (if you have that information). If you are regression testing, then add (don't delete others testing efforts) the testing revision that you are going to test in the testing field (add your 4 first nick letters). If there are more testers regtesting you will see the revisions they are going to test. It will help you to choose the most interesting revision. We recommend the binary search. When you finished testing a revision, update the Working/Failing and the VMW/QEM/VBX columns. Also delete your testing revision number from the testing.

  • Name: The name of the developer or tester who found the regression.
  • Regression: Description of the regression, or a link to a Bugzilla report.
  • VMWA : "Yes" if the regression happens in VMware, "NO" if doesn't. "NTD" if Not Tested.
  • QEMU : "Yes" if the regression happens in QEMU, "NO" if doesn't. "NTD" if Not Tested.
  • VBOX : "Yes" if the regression happens in VirtualBox, "NO" if doesn't. "NTD" if Not Tested.
  • RHDW : "Yes" if the regression happens in Real Hardware, "NO" if doesn't. "NTD" if Not Tested.
  • WRKs: Number of the revision where the bug isn't present.
  • Fails: Number of the revision where the bug is failing.
  • Testing: Before going to test (before downloading the revision indeed) add your number and your three letters here (i.e. 40100VIC).
Name Regression VMWA QEMU VBOX RHDW Works Fails Testing
DEV/TES Regression description YES YES NTD NTD 40102 40235 40180CAE,40210VIC,40150AMI,
Sh4ring4n gonzoMD mshtml freezes NTD NTD YES NTD r50311 r50312 50481SH4, 50458SH4, 50411SH4, 50381SH4, 50311SH4, 50365SH4, 50319SH4, 50312SH4, 53276GON
Ey3 Acrobat Reader 6 IT WORKS - menu issues. NTD NTD NTD YES r39653 r39654 Still present in r43812. Guilty commit: r39654 - "Fixes #4114"

TESTING APPS

Testing APPS is being currently linked to the greatest, impresive, astonishing and new Compatibility Database. So it will keep the Database updated and at the same time we can track easily GoldenApps and CandidateApps. All the Apps has their own links also you can find here a list of all of them.


This section is divided in 2 tables.

  • GoldenAPPS: The "GoldenApps Table" tracks the most important working apps which nowadays are working correctly on ReactOS.
  • CandidateAPPs: The "CandidateAPPs Table" tracks non working apps which must be tracked because of their importance. They will be moved to "GoldenApps" when they start working on ReactOS.


Description:

  • App : Name of the application and the link to download it.
  • Latest Working Revision: Latest revision where the App was working.
  • Status: It shows if the App has been tested with the "Golden Revision" (NTD: Not tested) and if it Worked(YES) or Not(NO).
  • Link to Compatibility Database(CD): Link to the App's Form in the CD to fill.

GOLDENAPPS

Current Test Target: bootcd-58487-dbg aka rev 58487


App Latest working revision Status Link to Compatibility Database
Firefox 2.0.0.20 54527 Works well - minor menu issues. Submit Test View Results
Firefox 3.0.11 54527 Works with rendering issues and occasional BSODs. Submit Test View Results
K-Meleon 1.5.2 54527 Works - Switching tabs does not repaint. Loading pages sometimes hangs. Ocasional URL bar entry issue. Submit Test View Results
Adobe Reader 7.1 54527 Works - menu issues. PDF-file for Test Submit Test View Results
Foxit Reader 2.1 54527 Works - menu issues. Needs MFC42.DLL. Submit Test View Results
Adobe Flash Player 10 Tested in r54527, the installer now succeeds however the plugin crashes. Submit Test View Results
Thunderbird 2.0.0.18 54527 Works - menu issues. Submit Test View Results
Sunbird 0.9 54527 Installs and seems to work well now. Submit Test View Results
Abiword 2.6.4 54527 Works - Text rendering problems. Scroll bar doesn't work. Toolbars have red background in 16-bit color. Submit Test View Results
FAR Manager 1.7 52433 Tested in r54527: installer BSODs at exit. The application itself, however appears to work fine. Submit Test View Results
Winrar 3.80 54527 Installs and mostly works. Comments have wrong font, can not see what is typed in extraction dialog, and can not export settings. WARNING: Encountered filesystem corruption extracting some files, but this is not limited to WinRar! Submit Test View Results
7-Zip 4.65 58487 Works - minor redrawing issues. Gone through built-in benchmark 40 times, after that I have compressed and then tested a 558MB installation of GOG Heroes of Might and Magic 2, seems stable. The integrated help displays only index, but not the page contents. Submit Test View Results
DosBox 0.72 54527 Works very well now. Submit Test View Results
Mirc 6.35 54527 Works - clicks don't always register in the application. Minor fonts issues. Submit Test View Results
Samba-TNG 54527 A couple of warnings but seems to work stable. Submit Test View Results
IrfanView 4.23 54527 Needs MFC42.DLL. Be sure to deselect the Google Search Bar in setup. - Application Works excellently. Redrawing issues using zoom. Submit Test View Results
Notepad Lite 3.3.1 52433 Tested in 58487: Installs. Application fails to start with the error "The application failed to initialize properly (0xC0000005)" Submit Test View Results
SeaMonkey 1.1.17 Tested on 54527. Still refuses to start normally on a fresh install, but now starting from the profile manager no longer works either. Submit Test View Results
Filzip 3.06 54527 Over all works, there are some crashes after install and opening the preferences. Some visual glitches. Submit Test View Results
AIDA32 3.94 54527 Application seems to function perfectly. Icons are colored wrong in 16-bit color mode. Submit Test View Results
UniversalVW 54527 Works except for some minor visual glitches. Submit Test View Results
SciTE 1.77 54527 Works - seems to run OK. Submit Test View Results
VLC Media Player 0.8.6i 54527 Works very well. Submit Test View Results

CandidateAPPS

Name App Rev VMWA QEMU VBOX ViPC RHDW BUG INSTALLING BUG WORKING
Vic,Jedi,(Bit) Opera 9.64 44317 NTD NTD YES YES YES VMWA/VBOX/RHDW:None VMWA/VBOX/ViPC7/RHDW:None - @Jedi and Vic: doesn't it crash for you if you type something in Opera?- basisbit

It hangs - Jedi

Opera 11.11 NTD NTD NTD NTD NTD
Vic/Bit/Ey3 Chrome 10.0.648.204 44317 NO NTD NO NTD NTD VMWA: None. HW: It keeps saying "Initialising" in a loop.
Ey3/Bit Word Viewer 2003 44317 NO NTD NO NTD NO All: none VMWA: It doesn't launch. #4587
Ey3/Bit Excel Viewer 2003 44317 NO NTD NO NTD NO HW: None HW: It doesn't launch. Then it tries to run in safe mode, but fails again.#4466
Ey3 Powerpoint Viewer 2003 42470 YES NTD YES NTD YES VBOX/VMWA/HW: None VBOX/VMWA/HW: None - not online at microsoft.com any more...
Ey3/Bit OpenOffice 3.3.0 44317 NO NTD NO NTD NO HW: Installation fails because MSVCR90.dll was not found.
Vic/Bit, Jedi-to-be Total Commander 7.56a 42470

42934

YES NTD YES YES YES
Vic/Bit Net Framework 3.0 42470 NO NTD NO NTD NO VMWA: Error:"Insuficient privileges." #4470
Vicmarcal, Jedi-to-be Java 6.25 41233

42934

NO NTD NTD NO NTD VMWA,ViPC7:None VMWA,ViPC7: Error the memory could not be read #4176
Ey3 Microsoft Office 2003 43812 NO NTD NO NTD NO HW: It does no longer freeze. It stops telling "Installation interrupted because of an error"

Application related bug reports

Main article: File Bugs

Make sure the bug happens only in ReactOS and is not a bug that also happens in Windows. If it happens in both then you should consult with the applications support or Dev team. You should, if possible, try different versions of Windows, making sure to use versions the application is meant to work with.

If the bug happens on some versions of Windows and in ReactOS, it is probably not ReactOS.

  • Note all details of your testing on different versions of windows when filing your bug report
  • Try a recent nightly build
  • With your nightly build gather a debug log from ReactOS
  • Make your bug report.
    • Attach your debug log detailed notes from Windows tests
    • Record full versions of all software involved (usually something like "2.34.4567.562 rev 19382 RC2 ..."): VMware, ReactOS SVN number, failing application.
    • Take screen shots if applicable of how the app failed, how it worked or failed differently in Windows than ReactOS and attach them.
  • If it is a commercial application that is closed source and cost money then you may be the one that has to do all Debugging for ReactOS and possibly the one whom submits the patches.
  • Give a appropriately formated copy of the bug report to the applications support or Dev team that made the app.

What makes an application important

The applications being tracked are not being tracked because of only their popularity or because someone likes the application a lot, there are more important reasons.

  • The application makes use of some critical part of the operating system, and it may be able to show progression or regression that will affect many other applications.
  • The application meets common needs that are needed on most systems.
  • The application is needed for development of the operating system.

Because tracking too many applications at this level of detail is difficult you should not just add another application to the list. New additions to this list should be discussed with Fireball, Caemyr, vicmarcal on IRC or their talk page. But before that you should ask yourself these questions:

  • Will this application test many different parts of the operating system?
  • Does this application do something that will help with testing or development by tracking its usage?
  • Is there a tracked application that already does this?
  • Is there a good reason for wanting this particular application to be tracked?

If you can answer yes to two of the first three and the fourth then you might consider asking in IRC about adding it.

Tracking other applications

You can track the app yourself but note that it will be your responsibility to test your applications and report any bugs. You must also make sure the bugs are not reproducible on the current version/versions of Windows, in case the bug is in the application and not in the operating system. If you are still interested your best option is to track the bug on your user page. You can copy the above apps table and edit it for this.