| Author: | Leandro Lucarella |
|---|---|
| Contact: | [email protected] |
| Version: | 1.0 |
| Date: | 2024-12-12 |
| Copyright: | Leandro Lucarella (2008), released under the BOLA license |
Abstract
mutest is a micro unit testing framework for C (with some C++ support). It's mostly an idea (it even comes with 2 implementations of the idea!) with the goal of being easy to use (just write your test cases grouped in test suites and you're set) and so small and simple that you don't mind to copy the files to your project and just use it (i.e., no dependencies).
The idea is simple: a source file is a test suite, a function is a test case (special functions can be used for test suite initialization and termination), which can can have several checks. Checks comes in 2 flavors, one that only prints an error, and one that terminates the current test case too. A (normally) automated test program run all the test suites and print some stats. It fails (returns non-zero) if any test suite fails.
Contents
Download the latest distribution tarball and uncompress it.
You can also download any release from the releases directory or get it using Git directly from the Git repository.
You can get this manual too, or a PDF version of it.
To actually install mutest run:
$ make install
Default installation path is /usr/local (because of that, you'll probably
need superuser privileges to install to the default location). You can override
that by passing the prefix make variable, for example:
$ make prefix=/opt/mutest install
If you want to install just the docs, you can do:
$ make install-doc
Or even install-readme, install-html or install-pdf if you are too
picky.
If you want to install just one particular implementation, to can use the
install-c and install-py targets.
You can find some samples in the sample directory.
This is an example taken from there. A simple module called factorial.c with its corresponding test suite (factorial_test.c).
You can see some C++ support in the exception_test.cpp test suite.
mutest is about 4 simple concepts: test program, test suite, test case and checks. Well, to be honest you probably will need test suite initialization and termination too =)
A test program is the higher level unit of mutest. The test program is the one in charge of running all your tests. Probably one of the more important features of mutest is that you are not supposed to bother about the test program. So, different implementations have different ways to tackle this. Some need more or less interactions from your part, and each have their pros and cons.
But this is all you need to know for now, for more details see how the test program is implemented by your implementation of choice.
A test suite is the higher level unit of mutest that you should care about =). Is not much more than a way to group test cases. Code-wise, a test suite is a C (or C++) module (or compilation unit). Not clear enough? A unit test is an object file (could be a shared object depending on the implementation). This module should have one or more test cases and it could have any number (including zero) of initialization and termination functions.
A test suite, is inspected by the test program for test cases and initialization and termination functions, and run them.
A test suite fail if one or more test cases fail, and it's skipped if one or more initialization functions fail (or, depending on the implementation, if the test suite can't be loaded at all).
A test case is just a plain function with a special signature and name.
A test case function name must start with mu_test, and take no arguments
and return nothing. For example:
void mu_test_something(void);
A test case (probably) only make sense if it has checks. A test case succeed only if all its checks succeed too.
Test are executed in an implementation-dependant order, but usually the default order is alphabetical.
Checks are assertions that a test case must pass (a boolean expression that must evaluate to true). There are 2 big flavors of checks: check and ensure. check just print an error (and mark the test case as failed) and ensure halt the test case execution, jumping to the next one.
For better C++ support there are check macros that assert that a specified exception is thrown (instead of check for a boolean expression to evaluate to true).
You can take a look at the reference to see the different flavors of check macros in more detail.
Sometimes you need to setup some environment shared between all the test cases in a test suite. You can use initialization functions for this.
An initialization function, like a test case, is a plain C function with
a special name and signature. The name must start with mu_init and it must
take no arguments, and return an error code (0 being success). For
example:
int mu_init_something(void);
All initialization functions are executed before any test case, in an implementation-dependant order, and if one of them fail (returns non-zero), the whole test suite is skipped immediately.
Termination functions are just like initialization functions, but they're
executed after the test cases, their names start with mu_term and
they return nothing. For example:
void mu_term_something(void);
You can use mutest with C++, the only care you must take is that, because of
C++ name mangling (and mutest relaying on function names), you must
declare your test cases and initialization and termination functions as
extern "C" (see exception_test.cpp for an example).
Checks become exception-safe when using mutest with a C++ compiler, and 2 extra checks designed for C++ get defined (mu_echeck() and mu_eensure()). They assert that an expression throws a particular exception.
There are 2 big groups of possible implementations that I can think of: static and dynamic.
mutest comes with one implementation of each group.
Static implementations can be only written in C/C++ (or other language that is link-compatible with C, like the D Programming Language, but since one of the main goals of mutest is avoid unnecessary dependencies, you probably don't want to depend on an extra language/compiler to run your tests =).
The main advantage is better debugging support, because you can run the test program in a standard debugger and see what happens with test cases very naturally.
The main disadvantage is, the test suites must be figured out in compile-time, usually using some kind of code generation (if you want to avoid writing repetitive code yourself). There's also a limitation in the test case, initialization and termination functions names: they should be unique for all the test program.
mutest comes with a C static implementation. Only 3 files are needed:
mutest.c (the user-independent part of the test program), mkmutest
(a bash script for generating the user-dependent part of the test program)
and mutest.h (the header file that test suites should include).
You can copy this 3 files to your project or install them at system-level and use them globally.
The procedure is simple, You should compile you test suites, mutest.c
and the generated output of mkmutest as object files and link them
together.
For example:
$ cc -c -o mutest.o mutest.c $ cc -c -o test1.o test1.c $ cc -c -o test2.o test2.c $ mkmutest mutest.h test1.o test2.o | cc -xc -c -o runmutest.o - $ cc -o testprg mutest.o test1.o test2.o runmutest.o
Then you can run the test program invoking it with no arguments:
$ ./testprg
This small script take 1 mandatory positional argument: the path to the
mutest.h file. All remaining positional arguments should be object files
representing test suites.
The test program can be invoked without arguments, but can take some extra options:
-vBe verbose. This is accumulative, when you add extra
-vyou will get extra verbosity.By default, you just get failed checks printed. If you use a single
-v, a summary of failed/passed test suites, test cases and checks will be printed. If an extra-vis used, you'll see the current test suite being executed. Another-vand you'll get the current test case, and another one, and you'll get each check.
Even when dependencies are kept minimal, there always be a few ;)
To use this implementation you just need:
- A C compiler (you already needed that, so...)
- The
nmprogram (from GNU Binutils, included in virtually any *NIX) - The GNU Bash shell interpreter (also included in virtually any *NIX)
Dynamic implementations, on the other hand, can be written in any language that can access to shared objects. The idea is to inspect a shared object for test suites and run them, without requiring any information about test suites at compile time.
There are several advantages in this kind of implementations. The dynamic nature let you completely separate the test program from the user-written test suites and you can choose at run-time what test suites to execute by just selecting the correct shared objects. Also, test case, initialization and termination functions names only have to be unique in the scope of the test suites, because test suites are completely isolated in separate shared objects.
But everything comes at a price, and the higher price to pay is debuggability. It's a little harder to plug a debugger to a shared object.
This implementation is much simpler and elegant than the C implementation.
Only 2 files are needed: mutest (test program written in Python using
ctypes module to access the shared object symbols) and mutest.h (the
header file that test suites should include).
Since both implementations provided by mutest share the same mutest.h,
you should define the MUTEST_PY macro when compiling the test suites if
you will run them using this implementation.
As with the C implementation, you can copy this 2 files to your project or install them at system-level and use them globally.
The procedure is even simpler than the C implementation: compile and link
you test suites as shared objects and then run the mutest program
passing the shared objects as arguments. For example:
$ cc -c -fPIC -DMUTEST_PY -o test1.o test1.c $ cc -shared -o test1.so test1.o $ cc -c -fPIC -DMUTEST_PY -o test2.o test2.c $ cc -shared -o test2.so test2.o $ mutest test1.so test2.so
That's it.
mutest program takes test suites shared objects to run as positional
arguments. It accepts the same options as the C implementation's test
program and some extra options are accepted too:
--verbose- Alias for
-v. -q,--quiet- Be quiet (no output is shown at all).
-s,--search- Search for test suites (*.so) in the current directory and add them to the list of test suites to run.
-h,--help- Show a help message and exit.
As with the C implementation, some minor dependencies are needed:
- Python (2.5 or later)
- The
nmprogram (from GNU Binutils, included in virtually any *NIX)
You will need a C compiler for building the test suites too, but technically is not needed by mutest itself ;)
- Synopsis
mu_check(expression)- Description
- Check that the
expressionevaluates to true. Continue with the test case if fail. - Availability
- Always
- Example
void mu_test(void) { mu_check(5 == 4); /* fail */ mu_check(5 == 5); /* excecuted, pass */ }
- Synopsis
mu_ensure(expression)- Description
- Check that the
expressionevaluates to true. Interrupt the test case if fail. - Availability
- Always
- Example
void mu_test(void) { mu_ensure(5 == 4); /* fail */ mu_check(5 == 5); /* not excecuted */ }
- Synopsis
mu_echeck(class, expression)- Description
- Check that the
expressionthrows a specific exceptionclass(or subclass). Continue with the test case if fail. - Availability
- C++ only
- Example
#include <stdexcept> extern "C" { void mu_test(void) { mu_echeck(std::exception, true); /* fail */ mu_echeck(std::exception, throw std::runtime_error("!")); /* excecuted, pass */ } }
- Synopsis
mu_eensure(class, expression)- Description
- Check that the
expressionthrows a specific exceptionclass(or subclass). Interrupt the test case if fail. - Availability
- C++ only
- Example
#include <stdexcept> extern "C" { void mu_test(void) { mu_eensure(std::exception, true); /* fail */ mu_echeck(std::exception, throw std::runtime_error("!")); /* not excecuted */ } }
This manual was written using reStructuredText.