What is Open Test?

Open Test provides a mean to easily integrate a testing framework with Open Test test drivers.

The open test concept emerged from the complete separation of the way tests of running test from driving and monitoring the test run in CppUnit 2.

Those two activities are managed by two components: the TestRunner and the TestDriver.

While implementing a TestRunner is cheap, implementing TestDriver cost a lot, especially if the test driver must provide a graphical user interface.

With Open Test, it is possible to reuse the existing TestDriver and implement a TestRunner for a specific test framework. This makes implementing a specific functional test framework cheap since you only need to implement the testing part. All the monitoring, user interface parts are already implemented and reusable.

What is a TestRunner?

A test runner is a lightweight component that provides the following services:

• declare all the available tests
• run a specific list of tests and track the run of each test.

[Notes: need to be enhanced, resource acquisition service...]

What is a TestDriver?

A test driver as many responsibility:

• monitoring the test run and detecting crash and infinite-loop to resume the test run.
• selecting the test(s) to run
• providing feedback during the test run (what test is being run, did the test pass...)
• generating report of the test run (output result in XML, compiler and human friendly text format...)

A test driver can usually be configured from both the command-line and a configuration file for automated testing.

Graphical test drivers are provided for interactive browsing of the test hierarchy, and visual feedback...

It should be noted that some test drivers could be connected to multiple test runner at the same time.

How does the TestDriver communicate with the TestRunner?

While it is possible for a TestDriver to directly call methods of a TestRunner, it is not the way it is usually done. Implementing reliable test run monitoring requires running the test in a separate process.

Instead of going the easy way and using shared memory to share test data structure, communication between the TestDriver and a TestRunner rely on a duplex stream communication. While a little more complex to implement, this have the great advantage that a TestDriver can be connected to any TestRunner that implements the same protocol.

A TestDriver can communicate with a TestRunner that:

• is not using the same run-time version of the C library (a common issue on the Windows platform)
• is not compiled using the same compiler
• is running on a different architecture (is a duplex network connection is used)
• is possibly implemented using another language (would require porting the communication layer to the alternate language)