A PHP test harness for Komodo

Update 17 April 2012: The test harness now supports XML configuration files. It is compatible with Komodo IDE 6 and 7 (maybe also with older versions) and is tested with PHPUnit 3.4 through 3.6.

Why use it? What's the deal?

The new harness fixes a whole bunch of issues:

• It works with current versions of PHPUnit. Without it, Komodo just quits with an error message.
• It supports advanced PHPUnit features like @dataProvider. By contrast, the Komodo harness fails in the worst possible way when encoutering this construct, namely by resetting the count of failed tests to zero (!) and continuing as if nothing has happened.
• The new harness reports fatal errors during test execution in a way that will get your attention. Previously, Komodo just added an info entry at the very end of the list of tests, without signalling an error or alerting you to the premature exit of the test suite.
• The new harness is way faster than the built-in version, provided you suppress the enumeration of all "green" tests (which are uninteresting anyway and perhaps even distracting). You can actually run large test suites without the extended coffee break you got used to ;-)
• The output generated by the new harness is as detailed as that provided by PHPUnit in a terminal, but more structured and much easier to navigate if a lot of tests fail.
• It will present you with a more readable list of tests than either Komodo or PHPUnit, by transforming those VeryLongConcatenated_AndUnderscored_WormlikeTestNameConstructs into statements formatted for actual humans. (You can turn that feature off if it doesn't match your naming conventions.)
• It takes arguments using the same syntax as PHPUnit.
• It gives you a host of options for customizing the output.

Setup

The new test harness is a patch for Komodo, not an extension, ie it will replace some Komodo code. Installing it is easy. Download it and drop the extracted files into

{Komodo program dir}/lib/mozilla/python/komodo/harnesses/php/

on Linux and Windows and into

/Applications/Komodo\IDE.app/Contents/MacOS/python/komodo/harnesses/php

on Mac OSX. Overwrite files when prompted. And that's it.

Using it is equally easy. Just create test plans as you did before.

By default, the improved version is only active if PHPUnit >= 3.5 is installed. In all other cases, the old Komodo test harness is run from a 'legacy' directory, so installing the patch won't break anyone's existing test plans. You can use the new test harness with PHPUnit 3.4, though, and considering some of the flaws of the built-in functionality, that might actually be a very good idea. Just add --force-new-harness to the arguments of your test plan if your PHPUnit version is older than 3.5.

Customizing the output

The new harness will pass your command line arguments on to PHPUnit, which means that you can now use options in standard PHPUnit syntax. In addition, the harness supports these switches:

--in-groups       Display test results grouped by status, problems first. Default.
--in-sequence     Display test results in order of execution.
 
--as-text         Convert test names to descriptive text. Default.
--as-name         Display test names as in the source.
 
--show-class      Show the name of the testcase class along with the method name. Default.
--hide-class      Show the name of the test method only.
 
--show-passed     Display the names of all tests. Default.
--hide-passed     Display the names of failing tests only (including skipped and incomplete
                  tests). [SHOULD be the default because tests run faster by an order of
                  magnitude, but a Komodo bug needs to be fixed first - see below.]
 
--show-summary    Show the test summary generated by PHPUnit. Default.
--no-summary      Don't show the PHPUnit test summary.
                  [Komodo currently doesn't report skipped and incomplete tests in its own
                  summary, so they might go unnoticed with this option.]

You can change the defaults themselves in /Ko/PrinterOptions.php.

Caveats?

Some bugs remain unfixed and can only be handled by the Komodo team. One is worth mentioning. If you use the --hide-passed switch to restrict reporting to failing tests only (and dramatically speed up test execution at the same time), a Komodo bug makes the status bar lose count of passed tests. It will always display "0 passed". That's no big deal, though, because you can see the true number of tests, along with a full test summary, in an information entry on top of the test list.

You'll find more information on the state of the Komodo bugs in the Komodo support forum.

Download

Here it is:

Download the files | View the source

If, and only if, you want to modify the harness itself for your own purposes, use these links which include unit and integration tests for the harness:

Download the files, with tests | View the source, with tests