• FrequentlyAskedQuestions

Frequently Asked Questions about CppUnit

Feel free to add questions or answer ones that already exist

Concepts

What are Test Plug-ins for

Test plug-ins can provide two services:

• test cases to run for the DllPlugInTester

• specific test listeners to use when running the tests with the DllPlugInTester.

  This provides a very convenient mean to run unit tests contained in a dynamic library as a post-build step. Also the DllPlugInTester contains multiple command-line options to help getting the progress/output you want.

  Additional test listeners can be provided as plug-in to the DllPlugInTester. See examples/ClockerPlugIn for an example that measures how long each test case takes to run and integrate those measurement in the xml output.

Items 2 and 3 of CppUnit2 features provide more details: http://cppunit.sourceforge.net/cppunit2/features.html.

Workings

How is flow of control when CppUnit runs tests? What can run in parallel, what is forced to be sequential. (Answer needed)

CppUnit seems to create one instance of the test class for each test function in it. Also, it seems to run setUp and tearDown in these different instances in parallell. Though it can't run tests in parallell, can it? I have trouble using global resource in test, which seems to be caused by setUp/tearDown running in parallel. Would be nice to know exactly what CppUnit did here. Please replace this with a specification.

How do I?

How do I run something like setUp() and tearDown(), but only once in total, not once per test?

Sometimes, the code for setUp() is really expensive in time or resources, and every test has to do the same setup. In these cases, it would be nice to be able to run the setup just once, and tear it down at the end of all the tests.

First answer is, Don't do it! If you are writing tests "correctly", then each test should take less than 1/100 of a second or less. A reasonable sized program may have thousands of tests, and if they run in much more than a few seconds, then developers won't run them every time they compile, then forget to run them before they check in, etc. etc.

Usually, long setups are due to a class depending on external libraries with setup sequences (such as a random number generator that collects some entropy), or loads an external data source (such as querying a web page, loading a file, connecting to a database), or prepares some class that requires a lot of setup (like a configuration class that loads an XML file). A common solution is to use an abstract interface for this functionality. Your production code uses an implementation class that hooks up to the real resource, and your testing class uses a mock implementation (which just tells you what functions were called), a stub implementation (which just implements what you need), or some other abstraction. See the Wikipedia entry for mock objects for a starting place, and MichaelFeathers' Working Effectively With Legacy Code for in-depth discussions. A word of warning - mock objects can be created on the fly for languages with reflection, but C++ doesn't support reflection. This means a lot of web information about mock objects for C#, Python, Ruby, etc. etc. won't apply to CppUnit.

Using two different implementations implies that you need some sort of acceptance testing for the "real" class, which is expected to be much slower. You can use CppUnit for these acceptance tests as well - just keep them out of your once-per-compile unit tests. You can create an abstract test suite that tests your abstract class (using CPPUNIT_TEST_SUITE_END_ABSTRACT), using a pure virtual "get_impl" that returns a pointer of the interface class type. You can then create a derived class (using CPPUNIT_TEST_SUB_SUITE) that returns your mock interface for get_impl, and another that uses the "real" interface for get_impl, and run one or the other depending on if you are running unit or acceptance tests. See the section on Writing Test Fixtures for help on the non-standard suite macros.

If you've decided that you still really want to just initialize something once, then you have to resort to some sort of global. A fairly clean way is to define a few class static functions in your test:

   1 #include <cppunit/extensions/HelperMacros.h>
   2 #include "BigNastyClass.h"
   3 
   4 class MyTest : public CppUnit::TestFixture {
   5    CPPUNIT_TEST_SUITE( MyTest );
   6    CPPUNIT_TEST( testMyTest );
   7    CPPUNIT_TEST_SUITE_END();
   8  public:
   9    static BigNastyClass *my_big_nasty_class = 0;
  10 
  11    static void acquire_BigNastyClass()
  12    {
  13      if (!my_big_nasty_class) {
  14         my_big_nasty_class = new BigNastyClass("plz do lots of things");
  15      }
  16    }
  17 
  18    static void release_BigNastyClass()
  19    {
  20      delete my_big_nasty_class;
  21      my_big_nasty_class = 0;
  22    }
  23 
  24    void setUp()
  25    {
  26      acquire_BigNastyClass();
  27      my_big_nasty_class->reset();
  28    }
  29 
  30    void testMyTest()
  31    {
  32      CPPUNIT_ASSERT(my_big_nasty_class->phone_boss_for_approval_to_run_unit_tests());
  33    }
  34  };

NOTE: tearDown does not release the static object (that would defeat the purpose), so there will be a memory leak. Luckily, unit tests are not (or should not be) a long-running process, so it may be OK in this instance. A autoptr may be a good idea, or explicitly release the object in the unit testing main().

How do I run a subset of tests

Non interactive test runners accept one or many test cases to run. You need to figure out which one need to be run and only add those. The base Test class contains some 'find' methods which can be used to locate a specific test/suite.

How would I use CppUnit for automated testing (answer needed)

(more precision needed: what is exactly the meaning of automated testing ?)

Well, CppUnit is all about automatic tests, just avoid user interactions, visual checking and blocking (system) calls.

How do I stop a test run if a test fails? (answer needed)

How do I print debug info (answer needed)

Like what test is it that makes an segmentation fault?

I see mostly two options:

• run the tests using BriefTestProgressListener which print the name of each test before running it. In case of crash, the last name is the one of the test that crashed.

• Implements a Protector used to capture system specific exception that occurs during crash (hard to implement this portably).

Here is a code fragment/diff showing how to add this to some of the cookbook recipes:

   1 +#include <cppunit/TestResult.h>
   2 +#include <cppunit/BriefTestProgressListener.h>
   3 
   4 ... other setup code here ...
   5 
   6   // Adds the test to the list of test to run
   7   CppUnit::TextTestRunner runner;
   8   runner.addTest( suite );
   9 
  10 +  // Shows a message as each test starts
  11 +  CppUnit::BriefTestProgressListener listener;
  12 +  runner.eventManager().addListener( &listener );

How do I stop abort() being called when a test fails?

Enable exceptions. In GCC, compile with -fexceptions. In VC++, use /EHsc (Enable C++ exceptions under Code Generation).

How do I get the name of the test from within the setUp function?

Neither the setUp() nor tearDown() functions know the name of the test being called, so you can't customize these for different tests. If you need to setup the environment for a single test, include that setup with the test. If several tests share the environment, then consider private data and functions, with each test calling the shared setup function. If all but a few need the same environment, then consider putting it in setUp() and tearDown(), and adding extra code to the non-cooperative tests.

How can I create test hierarchy (suite of tests that contains other suites)?

Use the CPPUNIT_TEST_SUB_SUITE() macro. For instance:

   1  #include <cppunit/extensions/HelperMacros.h>
   2 
   3  class MyTest {
   4    CPPUNIT_TEST_SUITE( MyTest );
   5    CPPUNIT_TEST( testBasic );
   6    CPPUNIT_TEST_SUITE_END();
   7  public:
   8    void testBasic();
   9  };
  10 
  11  class MySubTest : public MyTest {
  12    CPPUNIT_TEST_SUB_SUITE( MySubTest, MyTest );
  13    CPPUNIT_TEST( testAdd );
  14    CPPUNIT_TEST( testSub );
  15    CPPUNIT_TEST_SUITE_END();
  16  public:
  17    void testAdd();
  18    void testSub();
  19  };

MyTest will run testBasic, while MySubTest will run testAdd, testSub, and testBasic. See the Writing Test Fixture section of the documentation. If you are testing a class heirarchy with abstract classes, CPPUNIT_TEST_SUITE_END_ABSTRACT() may also be useful, to create a test fixture that exercises pure virtual functions.

How do I get a minimal set of the cppunit?

What sources are necessary if I only want to have the basic functionality of unittest? eg. i only need a runner, an assert(boolean), a register. Is there any definition for the minimal set sources to make it runnable?

How do I test private/protected member functions?

There is a lot of debate in the unit testing world surrounding this topic. Some argue that if the logic of the private functions needs unit testing, one should try to refactor it into a different class altogether. After all, private functions should be an implementation detail, which may change in the future. However, protected members should still be considered as APIs. One of the most common way to accomplish this is by declaring the test suite class as the friend of the class to be tested. To be clean, it is better to scope this declaration in a ifdef section.

   1 class Foo {
   2   public:
   3 #ifdef UNITTEST
   4     friend class FooTest;
   5 #endif
   6     ...
   7 
   8   protected:
   9     ...
  10 
  11   private:
  12     ...
  13 };

It would even be better to come up with a helper macro like the following.

   1 #ifdef UNITTEST
   2 #define ASSIST_UNIT_TEST( class__ ) friend class class__##Test
   3 #else
   4 #define ASSIST_UNIT_TEST( class__ )
   5 #endif

Use the above macro in the class to be tested.

   1 class Foo {
   2   public:
   3     ASSIST_UNIT_TEST( Foo );
   4     ...
   5 
   6   protected:
   7     ...
   8 
   9   private:
  10     ...
  11 };

Alternatively, if you just need access to the protected members:

   1 // Original Class
   2 class Foo {
   3   protected:
   4     void bar();
   5 };
   6 
   7 // In testing code, possibly in another file
   8 class TestFoo : public Foo {
   9   public:
  10     void test_bar() { bar(); }
  11 };

Do your testing against TestFoo, or at least those tests that require access to the protected members. The advantage to this approach is that testing code can be cleanly separated from production code without using macros.

PS: If someone knows a better approach, please document it here.

Working With Other Programs

When unit testing a static library created with "ar", why does the registry not show any tests?

The ar utility strips out some of the static variable information used by CppUnit. So if you create a static library (.a), the CppUnit::TestFactoryRegistry won't return any tests for classes in that static library. The solution is to either compile your test classes in to a dynamic library (.so) or to link your test runner directly with the individual object (.o) files.

Note that you could just explicitly add the test fixtures from a static library to test runner class, bypassing the factory registry entirely.

Is there a program, like JUnitDoclet, that will generate test skeletons automatically?

yes, this was recently developed and is licensed under GPL. It is a component of the turaya-distribution. Short Instructions to get this working:

1. Download there: http://www.emscb.de/content/pages/turaya.downloads (the file turaya-snapshot-060619.tar.gz)

2. extraxt the files: tar xvfz turaya-snapshot-060619.tar.gz
3. cd turaya/scripts/code2test
4. read the readme: less README

Also, see the new code2test Sourceforge project.

Can I integrate CppUnit with CruiseControl?

Yes, with a bit of leg work.

1. You must be using a build process capable of being launched by CruiseControl (nant works well)

2. From within that build you should build cppunit, and build and run the unit tests.

3. By setting the unit tests to output in junit format to a log directory (for example, using the xslt from the latest development release, or by modifying cppunit) cruise control will display nicely formatted test results. (Don't forget to merge the log results)

Using cruisecontrol as a continous integration process is very nice, even when compared to running nightly builds. Seeing the build results appear very quickly after checkins on integration branches is _very_ nice.