Author:	Baptiste Lepilleur
Contact:	blep@users.sourceforge.net

General status

Open Test is a fairly new concept and hence has a low level of maturity. Everything is to be done.

The TestRunner interface has been defined and implemented for CppUnit 2.

Everything related to test driver need to be defined and implemented. This involves:

defining property tree for suite and test declaration, test runner configuration and test result.
writing a simple text based TestDriver
test monitoring and duplex communication through stream
test driver xml output
extend the text based TestDriver to accept multiple TestRunner
GUI based TestDriver
allow TestDriver selection from command-line

Property tree structure

Suite and test declarations

TestDeclarator::beginSuite() and TestDeclarator::addTest().

time-out delay
description,
input/expected values (for functional testing based on input/output)

Test runner configuration

TestRunner::runTests().

is logging enabled
number of threads

Test configuration

TestPlanEntry::configuration().

Two categories of data are associated to a test at this stage of the testing:

test specific runner configuration
test descriptive data

Test specific runner configuration:

enable logging

Test descriptive data:

input/expected values override

Test execution result

TestRunTracker::addTestInfo().

Notes: this method probably needs to be modified since it can be called multiple times (for example, every time a line of log is added).

test status: success ?
test status type: success/failure/fault...
test result message
test result source code location
test result (input/) actual/expected
test result log

Simple text based TestDriver

Implementing a simple text based TestDriver is required to validate the interface of the TestRunner. It should use direct method call for communication with the TestRunner (which means no monitoring).

Features:

text progress (like CppUnit BriefProgressListener)
dynamic log output as the test runner
compiler output (assertion location in a compiler compatible format with failure detail below).
optionally select test to run from the command-line

Status: partially implemented in opentest/texttestdriver.h.

Test monitoring

The test monitoring feature needs to be implemented. It requires that the TestDriver spawn a child process that run the TestRunner and communicate with it.

Communication between the TestDriver and the TestRunner should be stream based as it allows the usage of various implementation, such as shared memory, pipe, and network... Ideally, all implementations should provide the same interface, only requiring the TestDriver to bother about the implementation type at connection time.

A spike solution is being worked on to explore shared memory on Windows (Baptiste: a working implementation is in place but need a lot of clean-up).

GUI based TestDriver

should provide all the functionalities of a text based TestDriver (it simply does not close the result window when the test are done).
should support for multiple TestRunner
allow loading of multiple test plug-in (spawn test plug-in wrapper application for each DLL)

QT will likely be the first GUI toolkit to be ported as it is multi-platform and is easier to implement than MFC or wxWidget.

TestDriver selection from command-line

It should be possible to select the TestDriver to use from the command-line. This implies that all test drivers provide a similar interface.

The user should be able to choose between text driver and GUI driver from the command-line (use plug-in for TestDriver ?).

Open Test todo