Unit Testing & CxxTest

C/C++ Users Journal, December

2005
By Eric Gufford
Presentation by Phil Ulrich
What is unit testing?
♦ A unit test is a procedure used to verify that a
particular module of code is working properly.
♦ The idea about unit tests is to write test cases for
all functions and methods so that whenever a
change causes a problem, it can be identified and
fixed quickly. Ideally each test is separate from
the others.
♦ Unit testing in C++ is still not widely understood
or used, however, despite its usefulness.
Why isn’t unit testing widely
used?
♦ There are many explanations, all of which boil
down to one of three reasons.
– It is hard. Many classes aren’t easy to adapt to unit
testing. It’s easy to say “my code is too complicated to
unit test” and leave it at that.
– It is time-consuming. Unit test frameworks can be
difficult to learn and set up, thus using valuable time
that could be used for other tasks.
– It is complicated. There are many different frameworks
on the market (some open source, some commercial),
and all of them are large, unwieldy, and cannot easily
be used in an IDE.
What is CxxTest?
♦ While the first excuse to not use unit testing
can be countered, the second and third are
generally valid for most C++ unit testing
frameworks.
♦ CxxTest aims to counter those two excuses,
too.
♦ CxxTest is a framework that uses Perl to
construct robust testing classes.
How does it work?
♦ You include a header file (TestSuite.h) and declare
a class that inherits from CxxTest::TestSuite, the
base class of all unit tests.
– In the child class, you insert all the various tests the
class is meant to perform.
– From there, a Perl (or Python) script is run that actually
writes the class that will perform the tests.
– Thus, the script does all the heavy lifting, keeping the
developer from wasting hours of time trying to
understand the arcane mechanics of the framework.
How does it work? (cont.)
♦ Once the testing class is declared, each test
method can be defined. The methods must start
with a lower case ‘test’ in order for the Perl script
to see it and have no return type or any
parameters.
– i.e. void testMain(void) to test main().
♦ There are no real restrictions on the testing
functions; they can perform any actions that make
sense – instantiating classes, interacting with the
file system, etc.
Does CxxTest support logging?
♦ CxxTest supports a number of macros to record
test status, called TS_FAIL, TS_WARNING, and
TS_TRACE.
♦ Passing a test does not have a status – if the test
function ends with no error, the test was passed.
♦ These macros print to stdout as well as providing
an unambiguous statement of test status.
– TS_FAIL fails the test and stops execution of the
testing function.
– TS_WARNING does not end execution of the testing
function, but it does mark the test as not succeeding.
– TS_TRACE simply writes to stdout without affecting
the test status.
Does CxxTest support logging?
(cont.)
♦ There are also a number of assertions that can be used for
quality assurance purposes.
♦ These include equality operators (TS_ASSERT_EQUALS,
TS_ASSERT_LESS_THAN,
TS_ASSERT_GREATER_THAN), a variation for data
equality (TS_ASSERT_SAME_DATA), and even a
variation that tests for exceptions
(TS_ASSERT_THROWS_ANYTHING,
TS_ASSERT_THROWS_NOTHING, or
TS_ASSERT_THROWS_EQUALS).
♦ All macros have a variant beginning with TSM_ that will
print a user-defined message, and ETS_/ETSM_ variants
that throw exceptions.
What are test fixtures?
♦ Test fixtures are objects that are constructed
at the start of a test and are destroyed at the
end of the test.
♦ CxxTest supports test fixtures.
♦ By defining fixtures in one place, one
insures that each test will be using objects
constructed in the same way – thus, the
code to set up the test only has to be tested
once itself.
What are text fixtures? (cont.)
♦ CxxTest supports the concept of suite-level
fixtures, which are objects created at the beginning
of a suite of tests and destroyed at the end of them.
♦ This is useful when an object is complicated or
time-consuming to create, or support classes are
required that can be shared across a test suite.
♦ Static functions called createSuite() and
destroySuite() are called when the suite is created
and disposed of to create/destroy these suite-level
fixtures.
How does CxxTest output data?
♦ The method of output is specified on the command line of the Perl
script that generates the class.
♦ The default mechanism is designed for use in an IDE and is controlled
via the --runner option.
– This can use regular formatting like stdout, print line numbers like
Visual Studio expects, or simply provide true/false answers.
– This can be combined with a switch to describe the type of user
interface you would like in the testing application – options are
Win32, X11, or Qt. The behavior is similar in all three windowing
environments.
♦ Once the testing app is built, the executable can be run on demand. On
Windows, the executable shows its progress as a progress-bar-like
mechanism that shows failures as red and successes as green. At the
end, the overall result is shown on the bar – red for failures, yellow for
warnings, green for all successes.
How does CxxTest tie into a
normal build?
♦ Ideally, unit tests should be incorporated into a
makefile or a build process so they are ran when
the program is compiled.
♦ CxxTest comes with makefiles for a variety of
platforms and compilers, such as Linux/gcc,
UNIX/gcc, and Windows/Visual Studio.
♦ Under the Windows configuration, the testing
application is incorporated into the build process
such that if any tests fail, the build fails; if there
are any warnings, VS reports a warning. Setting
the system to fail on a warning will perform as
expected.
Conclusion
♦ In the past, unit testing was difficult, but
CxxTest helps easily create a test harness to
enable unit testing without spending hours
setting up and learning to use a complex
unit testing framework.
♦ More info about (and downloads of)
CxxTest can be found online at:
– http://cxxtest.sourceforge.net/