PHPUnit test harness for Komodo
integration-test-readme.markdown
Integration test readme
=======================
Essentially, just run KoHarnessTest with PHPUnit. But here is the fine print.
## How the test operates
The integration test
1. runs the test harness against a dummy test class
2. captures the output
3. compares it to a reference file.
This is repeated for all combinations of test harness switches, and for a set
of XML configurations.
To make the comparison work consistently, output is normalized. Information
about memory usage, execution time and system specifics like the PHP version
is represented by placeholders.
## Limitations
- The integration test uses the @dataProvider annotation. Ie, it requires at
least PHPUnit 3.2 to run.
- Older versions of PHPUnit can be tested as well, though, by
1. creating a new set of reference files on the target system (see below)
2. verifing that these files are identical to the original ones.
## When to regenerate the reference files
If the test harness is modified in a way that alters the output, the integration
test will fail. For intentional changes, the reference files must be updated:
1. Move the original reference-output folder to a different location.
2. Regenerate the reference files. Do it by running
/tools/Ko/tasks/ko\_create\_reference\_files.php.
3. Verify that both sets of files differ only where they should.
## Tools
Tools for creating test output and for debugging are in the /tools/Ko/tasks
directory. Their names should be self-explanatory.
The /special directory contains a setup for triggering a fatal error during test
execution. This is not an automated test - the output must be inspected manually.
