| Author: | Baptiste Lepilleur |
|---|---|
| Contact: | blep@users.sourceforge.net |
The framework offers the traditional features found in unit testing framework (assertions, grouping tests in suite...). A significant move has been made to provide more support for functional test:
The framework help having a quick develop/test cycle with the following features:
In a well-tuned environment the develop/test cycle can be as short as a few seconds, as you usually only recompile only two translation units (the test and the tested implementation), and run the test automatically after a successful build. This makes test driven design possible in C++.
Tests can be embedded into the dynamic library to test. An application provided with the framework is used to run the test contained in a dynamic library. The application can run the test of a specific dynamic library or of multiple dynamic libraries at once.
Test-plug in also help separating the tests from the way they are run, thus allowing to centralize the way tests are run and their result reported.
The framework provides easy mechanism to monitor the test run. It can automatically spawn a child process that run the test and monitor it for crash or infinite loop. In the case of infinite loop it terminates the child process. If either a crash or an infinite loop occurred, a new child process is spawned and testing resume after the problematic test case.
Test monitoring is provided by most test drivers of Open Test.
A text test driver is provided by Open Test to. It is commonly used in a post-build step to run unit test with failure location in a compiler like format for integration with the development environment.
Graphical test driver are provided by Open Test to:
The test drivers (the components that run and monitor the tests) can work with any test runner that implements the Open Test interface. This means that the framework text and graphical test drivers can be used to run and monitor tests implemented using another test framework as long as the Open Test interface can be implemented for that other framework.
More detail can be found in the Open Test document.
The result of a test run is thoroughly inspectable. Outputter are provided to output the result in multiple format, such as XML for integration in a tool chain, or a human readable format with compiler like assertion location for integration with IDE.
Test drivers can be configured either using the command-line or persistent configuration files.
The configuration management is designed in such a way that new configuration options can easily be added.
The assertion design separate the assertion location tracking from the function call used to implement the assertion.
This means that assertion functions can have a variable number of parameters, and that default parameters are allowed.
One of the advantages of writing custom assertions is that in addition to detecting that the assertion failed, a detailed diagnostic can be provided. For example, when comparing two structured trees, knowing that they are different let you know that something is wrong, but not what. With custom assertions, the assertion can be tested, and a detailed diagnostic based on the knowledge of what the structure represent and what would be useful to diagnostic the failure can be reported.
An application of advanced assertions can be found in enumeration and string assertions.
(Notes: should move the detail below somewhere else.)
Here is an example of how the assertion CPPUT_FAIL( "bad parameters." ) is expanded.
The assertion is defined as follow:
# define CPPUT_FAIL \ CPPUT_BEGIN_ASSERTION_MACRO() \ ::CppUT::fail
Which is expanded into:
::CppUT::setCheckPointLocation(
::CppUT::Location( __FILE__, __LINE__ ) ), // notes the comma
::CppUT::fail
Hence, the code CPPUT_FAIL( "bad parameters." ) first call SetCheckPointLocation to memorize where the assertion occurred in the source code, and then call the function fail with the parameter(s) specified by the user. As the parameters are not part of the macro, their number can vary.
The framework provides both assertions that abort the current test (throwing an exception), and assertions that do not abort the current test but keep track of the failures. Those assertions are named respectively "aborting assertions " and "checking assertions".
The implementation of this feature relies on a mechanism similar to setCheckPointLocation() described in Custom assertion support, meaning that all aborting assertions can easily be made available as checking assertions and vice versa.
The result of each test case run is always a property tree. All data related to the execution of a test case are stored as property in this tree. For instance, the status of the test (success, failure...) is just a specific property in this tree.
Using a property tree provide a lot of flexibility as it allows association of complex data to each test run such as:
Being able to report actual and expected values in a test result is a very powerful features. For example, if you are generating input values from a spreadsheet application, after running the test, you can put the actual values back into the spreadsheet.
Properties can be added to result result during the test case run. This makes it easy to provide rich structure output for functional testing.
Each test has a property tree. The property tree can be used to store descriptive data for each test. Property can be marked as "inheritable", meaning all tests of a test suite inherit this property (but they can override it).
Here is some example of properties that can be associated with a test:
The framework provides a specific service to convert an object into a string. This service is used by assertions to generate diagnostic.
Two operations are provided by the service:
While the default implementation relies on std::ostream for conversion of object to string, an alternative implementation can easily be provided (for platform such eVC++ 4 which do not provide iostream).
If the target compiler supports function template partial specialization and the feature is enabled in the configuration, pointer will be automatically deferenced.
The framework provides a specific service to check if two objects are equal.
If the target compiler supports function template partial specialization and the feature is enabled in the configuration, pointer will be automatically deferenced.
Two advanced assertions facility are provided:
The enumeration based assertions work with the concept of enumerator. An enumerator implements a specific interface allowing enumeration of the elements of a collection. The CppTL library provides a complete implementation of enumerator that supports STL containers and iterators and can be very easily extended to support other container interfaces.
The following assertions are provided for enumeration:
insert example here
The following assertions are provided for string:
String assertions works with heterogeneous string types and use the stringize mechanism provided by the framework to convert object to string.
String assertions provide detailed diagnostics on failure, escaping the string over multiple lines when it contains \n or \r character.
insert example here
Test cases delegate the actual task of setting up the test, running the test and cleaning up after the test (setUp/run/tearDown) to generic functors.
A generic functor can be created from a C function, an object instance and a member function or from a functor. When a function with parameters used for testing, its parameters can be bound to specific value, hence creating a generic functor that requires no parameter.
insert example here
The framework uses the fundamental RAII C++ idiom (Resource Acquisition Is Initialization idiom) for clarity.
Smart-pointer are used to pass dynamically allocated pointer around.