[[TableOfContents]] 

= Objectives =

The CppUnit code base has grown into a monster and there are still many features that I would like to see added.

CppUnit2 is an attempt to propose a much cleaner implementation of the current features, and to follow good C++ practices.

The main objectives are:
 * use of smart-pointer for memory management
 * better support for custom test cases (repeat...) and parametrized test methods
 * flexible enough to allow sharing of a fixture across test cases (bad practice as it breaks test isolation, but may save a lot of time at execution if the setup is heavy).
 * better assertion support (equality testing, object conversion to string,...)
 * association of user data with test cases
 * thread support (passing assertion exceptions across threads, assertion...)
 * input-based test case support (read input data from external source, such as XML files...)
 * python binding

= Getting the sources =

CppUnit2 sources are finally available on CVS. You can obtain them as follows:
{{{
cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/cppunit login
cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/cppunit co cppunit2
}}}

= Features =

 * /Assertions
 * /DescriptiveData
 * /ConvertToString
 * /EqualityTesting
 * /ParametrizedTestCase

= Compiler Requirements =

Four major requirements:
 * C++ exception
 * namespace support
 * enough template support to be able to implement /GenericFunctor
 * C++ style cast (easy to remove)
 * explicit keyword (easy to remove)

Compiler features that are '''not''' required:
 * function template overloading
 * template member support
 * run-time type information (rtti)
 * template partial specialization

Also, CppUnit2 should compile correctly on MSVC6/7.x and gcc 2.95, as those are probably the two most widely used compilers.

= Compatibility with CppUnit 1.x =

 CppUnit2 is implemented in a different namespace (Cpp``UT). This allows a compatibility layer to be created, as well as having both frameworks cohabiting. 

 It should be possible to build a compatibility layer for test fixtures, assertions, auto registration, hence preserving most of the written test code. Most of the remaining code written against other would like need to be ported (test listener, test registry, test runner, test context...).

= Build system =

 Experience with the autotools has shown that it is far from being perfect (AIX still doesn't support shared libraries), and that very few people actually know how to maintain and fix such build scripts. Also, the autotools don't work on Windows (managing builds on Windows with a total of about 8 configurations per project would be too much work).

 [Scons http://scons.sf.net] seems to be a good candidate for multiplatform builds. It should also be fairly easy for a user to 'hack' into the build script to adapt it to their compiler when not supported.

= Memory management =

 In CppUnit 1, it was difficult to know when passing a parameter what its expected life was and who was responsible to delete it.

 CppUnit2 solves this by using smart-pointer for dynamically allocated objects, and the following conventions:
  * pass by smart-pointer when the ownership of the parameter is transfered/shared.
  * pass by const reference indicates a 'pass by value', the parameter will be copied if needed.
  * pass by non const reference indicates that the ownership will not be assumed. The life-cycle requirement depends on the semantic of the function. In most case, if it is a constructor, then the reference is expected to be valid as long as the object being constructed. 

 The use of smart-pointer has greatly simplified the overall implementation (especially the TestFactoryRegistry).

 CppUnit2 comes with its own smart-pointer, which implements a subset of the generic smart-pointer that has been adopted by the standard C++ committee (neither weak pointer, nor checked_deleter are supported).

 A standard conformant implementation ([http://www.boost.org/libs/smart_ptr/smart_ptr.htm boost:shared_ptr]) can be used if CPPUT_USE_BOOST_SHARED_PTR is defined.

= Test fixture sharing for test case methods / shared resources =
 When writing heavy functional test cases which have a time-consuming setUp()/tearDown() (often database related), you often want the setUp()/tearDown() step to be executed only once to run all the test cases to decrease the time required to run the test.

 The obvious drawback of this is that tests are not isolated from each other, which means that a test may pass because a test before it was run, or a test may fail because a previous test failed.

 This can easily be implemented by dynamically controlling the behavior of Fixture``Factory (should the last created fixture be returned, or should a new one be returned).

 * Open issue: how should the user specify that a shared fixture policy needs to be used?
 * Possible solutions: 
  * make this a 'property' of the fixture creation and have the Fixture``Factory look up those properties
  * provide specific methods on the Fixture``Factory and wrap them into macros for helper macro integration

 Notes: currently trying to approach this from a different angle. Shared test fixtures with single setUp/tearDown are complex to implement with some of the new features of the framework (such as multiple execution of a test case or execution of a few test cases of a test fixture). The rough idea is to dissociate the setUp/tearDown from the test fixture (let's call them ''resource'') and have the test framework properly handle those resources.

 After some thinking, the shared test fixture will be taken out. It's basically trying to achieve what the resource concept would do without materializing it. The idea of the resource concept is to have test cases depend on some resources. A resource is a simple C++ object with a setUp/tearDown, which can be retrieved using a key (likely a string identifier). A resource is shared among multiple objects and is initialized only once per test run.

 What's interesting about resources is that you have explicit resource dependency tracking and that, for example, in a concurrent test runner, you could prevent two tests using the same resource from running concurrently. This puts more complexity on the test driver side, but it provides much more flexiblity and make things easier to understand.

= Threaded test cases =

 Having a test case that spans multiple threads to check for thread-safety or interaction is a fairly common task. 

 This puts the following requirements over the framework:
 * thread-safety
 * test failure propagation from a thread spawned by a test case to the test framework

 == Thread-safety ==

 All components used in multiple threads must be thread-safe. (This mainly concerns assertions.)

 The checkpoint mecanism used by the assertions used a static variable. It was made thread-safe by the use of thread local storage. To achieve this, a minimal portable thread api is embedded in CppUnit2.

 The smart-pointer reference counter needs to be thread-safe.

 Care must be taken when manipulating std::strings across thread as they often use non-thread-safe reference counting.

 Notes: WIN32 implementations for this are done. pthread & MacOS implementations need to be done.  (Mac OS X uses pthread for its threading model)

 == Test failure propagation ==

 Assertions that occur in threads spawned by a test case need to be transfered to the main test framework thread and associated with the test case.

 This is achieved by reusing the Test``Failure``Guard``Chain components which capture test case uncaught exceptions and assertions. These components are reused to protect spawned thread main loop and 'memorize' failures. Those failures can then be dispatched in the main test framework thread.

== Test plug-in ==

 There are two main uses for plug-ins in cppunit:
 * writing unit tests in plug-ins
 * customizing/extending cppunit at run-time

 Unit test plug-ins are already handled by linking against a dll build of cppunit 2. The static registry is shared among users of the dll. A simple runner needed to be written though.

 Customization/extension should provide more flexibility than in cppunit 1. A very interesting goal to achieve would be to allow graphic test runner to be a simple 'extension', though this might be complex to achieve.

= Input-based test cases =
  
= Python binding =

Expose execution of unit tests, and other high-level functionnalities in python. If possible provide integration with python unit testing module.

 * Ease the development of new test runners (test plug-in runner...) and increase their reusability
 * Integration with python environment for unit testing

= Integration with test-driving environments =

 Study possiblity to integrate the test framework with:
 * [Cruise``Control http://cruisecontrol.sourceforge.net/]
 * [Cruise``Control.NET http://ccnet.thoughtworks.com/]
 * [QMTest http://www.codesourcery.com/qm/qmtest]
 * ?

= Architecture =

The test hierarchy:

attachment:uml_test.gif

Main changes compared to CppUnit 1.x include the use of the visitor pattern, a reference to the parent test in each test, and the use of generic functors in test case implementation.

 == Visitor pattern to distinguish between test case and test suite ==
   It simplifies implementation, and gives much more flexibility to explore the test hierarchy...

 == Each test references its parent ==
   * Helps provide useful reports on failure (the full test suite path can be easily reported). 
   * Does not depend on execution order to track which test suite the test belongs to. 
   * Avoid the ugly hack in the test fixture to prepend the method name with the fixture class name to report the suite the test belongs to.

 == Generic Functor in test case implementation ==
   * Test setUp/tearDown and run can be implemented in any form (functor, C function, member function)
   * Parameters can be bound to the generic functor implementing setUp/tearDown and run, thereby allowing simple test parametrization.
`
= To do =
 * build system
 * thread api for pthread (need to implement Thread``Id``Type)
 * thread api for Mac OS
 * thread guard (failure propagator for spawned thread in test case)
 * test plug-in
 * test listener plug-in
 * plug-in loader & manager
 * progress listener
 * result outputter (text, compiler, xml)
 * test plan (find test by pattern...)
 * test runner (mfc, qt, command line based)
 * input-based test case (likely to have an impact on the core design)
 * shared test fixture with single setUp/tearDown (likely to have an impact on the core design)

= Progress =
 * 2004/06/16:
  * enhanced assertenum.h to support convertion to string customization through functor.
  * idea: enhance class Message to carry a bag of properties, allowing structured result. Find a way to get this into success too (could be used to store elapsed time...). At the current time in cppunit 1, we must use a test listener and can only store results with the xml outputter.
 * 2004/02/08:
  * incorporated the portable dll loading api of cppunit 1
  * added header content descriptions to README.
