<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html><head>
<title>cmocka</title>
</head>
<body>
<h1>cmocka Unit Testing Framework</h1>
<p>cmocka is a lightweight library that is used to author C unit tests.</p>

<ul>Contents
  <li><a href="#Motivation">Motivation</a></li>
  <li><a href="#Overview">Overview</a></li>
  <li><a href="#Test_Execution">Test Execution</a>
  <li><a href="#Exception_Handling">Exception Handling</a></li>
  <li><a href="#Failure_Conditions">Failure Conditions</a></li>
  <li><a href="#Assertions">Assertions</a></li>
  <ul>
    <li><a href="#Assert_Macros">Assert Macros</a></li>
  </ul>
  <li><a href="#Dynamic_Memory_Allocation">Dynamic Memory Allocation</a></li>
  <li><a href="#Mock_Functions">Mock functions</a></li>
  <ul>
    <li><a href="#Return_Values">Return Values</a></li>
    <li><a href="#Checking_Parameters">Checking Parameters</a></li>
  </ul>
  <li><a href="#Test_State">Test State</a></li>
  <li><a href="#Example">Example</a></li>
</ul>

<a name="Motivation"><h2>Motivation</h2></a>
<p>There are a variety of C unit testing frameworks available however many of
them are fairly complex and require the latest compiler technology.  Some
development requires the use of old compilers which makes it difficult to
use some unit testing frameworks.  In addition many unit testing frameworks
assume the code being tested is an application or module that is targeted to
the same platform that will ultimately execute the test.  Because of this
assumption many frameworks require the inclusion of standard C library headers
in the code module being tested which may collide with the custom or
incomplete implementation of the C library utilized by the code under test.</p>

<p>cmocka only requires a test application is linked with the standard C
library which minimizes conflicts with standard C library headers.  Also,
cmocka tries avoid the use of some of the newer features of C compilers.</p>

<p>This results in cmocka being a relatively small library that can be used
to test a variety of exotic code.  If a developer wishes to simply test an
application with the latest compiler then other unit testing frameworks maybe
preferable.</p>

<a name="Overview"><h2>Overview</h2></a>
<p>cmocka tests are compiled into stand-alone executables and linked with
the cmocka library, the standard C library and module being tested.  Any
symbols external to the module being tested should be mocked - replaced with 
functions that return values determined by the test - within the test
application.  Even though significant differences may exist between the target
execution environment of a code module and the environment used to test the
code the unit testing is still valid since its goal is to test the logic of a
code modules at a functional level and not necessarily all of its interactions
with the target execution environment.</p>

<p>It may not be possible to compile a module into a test application without
some modification, therefore the preprocessor symbol <b>UNIT_TESTING</b> should
be defined when cmocka unit test applications are compiled so code within the
module can be conditionally compiled for tests.</p>

<a name="Test_Execution"><h2>Test Execution</h2></a>
<p>cmocka unit test cases are functions with the signature
<b>void function(void **state)</b>.  cmocka test applications initialize a
table with test case function pointers using <b>unit_test*()</b> macros.  This
table is then passed to the <b>run_tests()</b> macro to execute the tests.

<b>run_tests()</b> sets up the appropriate exception / signal handlers and
other data structures prior to running each test function.   When a unit test
is complete <b>run_tests()</b> performs various checks to determine whether
the test succeeded.</p>

<h4>Using run_tests()</h4>
<a href="../src/example/run_tests.c">run_tests.c</a>
<listing>
#include &lt;stdarg.h&gt;
#include &lt;stddef.h&gt;
#include &lt;setjmp.h&gt;
#include &lt;cmocka.h&gt;

// A test case that does nothing and succeeds.
void null_test_success(void **state) {
}

int main(int argc, char* argv[]) {
    const UnitTest tests[] = {
        unit_test(null_test_success),
    };
    return run_tests(tests);
}
</listing>

<a name="Exception_Handling"><h2>Exception Handling</h2></a>
<p>Before a test function is executed by <b>run_tests()</b>,
exception / signal handlers are overridden with a handler that simply
displays an error and exits a test function if an exception occurs.  If an
exception occurs outside of a test function, for example in cmocka itself,
the application aborts execution and returns an error code.</p>

<a name="Failure_Conditions"><h2>Failure Conditions</h2></a>
<p>If a failure occurs during a test function that's executed via
<b>run_tests()</b>, the test function is aborted and the application's
execution resumes with the next test function.

Test failures are ultimately signalled via the cmocka function <b>fail()</b>.
The following events will result in the cmocka library signalling a test
failure...

<ul>
  <li><a href="#Assertions">Assertions</a></li>
  <li><a href="#Exception_Handling">Exceptions</a></li>
  <li><a href="#Dynamic_Memory_Allocation">Memory leaks</a></li>
  <li><a href="#Test_State">Mismatched setup and tear down functions</a></li>
  <li><a href="#Return_Values">Missing mock return values</a></li>
  <li><a href="#Return_Values">Unused mock return values</a></li>
  <li><a href="#Checking_Parameters">Missing expected parameter values</a></li>
  <li><a href="#Checking_Parameters">Unused expected parameter values</a></li>
</ul>
</p>

<a name="Assertions"><h2>Assertions</h2></a>
<p>Runtime assert macros like the standard C library's <b>assert()</b> should
be redefined in modules being tested to use cmocka's <b>mock_assert()</b>
function.  Normally <b>mock_assert()</b> signals a
<a href="#Failure_Conditions">test failure</a>.  If a function is called using
the <b>expect_assert_failure()</b> macro, any calls to <b>mock_assert()</b>
within the function will result in the execution of the test.  If no
calls to <b>mock_assert()</b> occur during the function called via
<b>expect_assert_failure()</b> a test failure is signalled.</p>

<h4>Using mock_assert()</h4>
<a href="../src/example/assert_module.c">assert_module.c</a>
<listing>
#include &lt;assert.h&gt;

// If unit testing is enabled override assert with mock_assert().
#if UNIT_TESTING
extern void mock_assert(const int result, const char* const expression, 
                        const char * const file, const int line);
#undef assert
#define assert(expression) \
    mock_assert((int)(expression), #expression, __FILE__, __LINE__);
#endif // UNIT_TESTING

void increment_value(int * const value) {
    assert(value);
    (*value) ++;
}

void decrement_value(int * const value) {
    if (value) {
        *value --;
    }
}
</listing>
<a href="../src/example/assert_module_test.c">assert_module_test.c</a>
<listing>
#include &lt;stdarg.h&gt;
#include &lt;stddef.h&gt;
#include &lt;setjmp.h&gt;
#include &lt;cmocka.h&gt;

extern void increment_value(int * const value);

/* This test case will fail but the assert is caught by run_tests() and the
 * next test is executed. */
void increment_value_fail(void **state) {
    increment_value(NULL);
}

// This test case succeeds since increment_value() asserts on the NULL pointer.
void increment_value_assert(void **state) {
    expect_assert_failure(increment_value(NULL));
}

/* This test case fails since decrement_value() doesn't assert on a NULL
 * pointer. */
void decrement_value_fail(void **state) {
    expect_assert_failure(decrement_value(NULL));
}

int main(int argc, char *argv[]) {
    const UnitTest tests[] = {
        unit_test(increment_value_fail),
        unit_test(increment_value_assert),
        unit_test(decrement_value_fail),
    };
    return run_tests(tests);
}
</listing>

<h3><a name="Assert_Macros">Assert Macros</a></h3>

<p>cmocka provides an assortment of assert macros that tests applications
should use use in preference to the C standard library's assert macro.  On an
assertion failure a cmocka assert macro will write the failure to the
standard error stream and signal a test failure.  Due to limitations of the
C language the general C standard library assert() and cmocka's
assert_true() and assert_false() macros can only display the expression that
caused the assert failure.  cmocka's type specific assert macros,
assert_{type}_equal() and assert_{type}_not_equal(), display the data that
caused the assertion failure which increases data visibility aiding
debugging of failing test cases.</p>

<h4>Using assert_{type}_equal() macros</h4>
<a href="../src/example/assert_macro.c">assert_macro.c</a>
<listing>
#include &lt;string.h&gt;

static const char* status_code_strings[] = {
    "Address not found",
    "Connection dropped",
    "Connection timed out",
};

const char* get_status_code_string(const unsigned int status_code) {
    return status_code_strings[status_code];
};

unsigned int string_to_status_code(const char* const status_code_string) {
    unsigned int i;
    for (i = 0; i < sizeof(status_code_strings) /
                    sizeof(status_code_strings[0]); i++) {
        if (strcmp(status_code_strings[i], status_code_string) == 0) {
            return i;
        }
    }
    return ~0U;
}
</listing>
<a href="../src/example/assert_macro_test.c">assert_macro_test.c</a>
<listing>
#include &lt;stdarg.h&gt;
#include &lt;stddef.h&gt;
#include &lt;setjmp.h&gt;
#include &lt;cmocka.h&gt;

extern const char* get_status_code_string(const unsigned int status_code);
extern unsigned int string_to_status_code(
    const char* const status_code_string);

/* This test will fail since the string returned by get_status_code_string(0)
 * doesn't match "Connection timed out". */
void get_status_code_string_test(void **state) {
    assert_string_equal(get_status_code_string(0), "Address not found");
    assert_string_equal(get_status_code_string(1), "Connection timed out");
}

// This test will fail since the status code of "Connection timed out" isn't 1
void string_to_status_code_test(void **state) {
    assert_int_equal(string_to_status_code("Address not found"), 0);
    assert_int_equal(string_to_status_code("Connection timed out"), 1);
}

int main(int argc, char *argv[]) {
    const UnitTest tests[] = {
        unit_test(get_status_code_string_test),
        unit_test(string_to_status_code_test),
    };
    return run_tests(tests);
}
</listing>

<a name="Dynamic_Memory_Allocation"><h2>Dynamic Memory Allocation</h2></a>

<p>To test for memory leaks, buffer overflows and underflows a module being
tested by cmocka should replace calls to <b>malloc()</b>, <b>calloc()</b> and
<b>free()</b> to <b>test_malloc()</b>, <b>test_calloc()</b> and
<b>test_free()</b> respectively.  Each time a block is deallocated using
<b>test_free()</b> it is checked for corruption, if a corrupt block is found
a <a href="#Failure_Conditions">test failure</a> is signalled.  All blocks
allocated using the <b>test_*()</b> allocation functions are tracked by the
cmocka library.  When a test completes if any allocated blocks (memory leaks)
remain they are reported and a test failure is signalled.</p>
<p>For simplicity cmocka currently executes all tests in one process.
Therefore all test cases in a test application share a single address space
which means memory corruption from a single test case could potentially cause
the test application to exit prematurely.</p>

<h4>Using cmocka's Allocators</h4>
<a href="../src/example/allocate_module.c">allocate_module.c</a>
<listing>
#include &lt;malloc.h&gt;

#if UNIT_TESTING
extern void* _test_malloc(const size_t size, const char* file, const int line);
extern void* _test_calloc(const size_t number_of_elements, const size_t size, 
                          const char* file, const int line);
extern void _test_free(void* const ptr, const char* file, const int line);

#define malloc(size) _test_malloc(size, __FILE__, __LINE__)
#define calloc(num, size) _test_calloc(num, size, __FILE__, __LINE__)
#define free(ptr) _test_free(ptr, __FILE__, __LINE__)
#endif // UNIT_TESTING

void leak_memory() {
    int * const temporary = (int*)malloc(sizeof(int));
    *temporary = 0;
}

void buffer_overflow() {
    char * const memory = (char*)malloc(sizeof(int));
    memory[sizeof(int)] = '!';
    free(memory);
}

void buffer_underflow() {
    char * const memory = (char*)malloc(sizeof(int));
    memory[-1] = '!';
    free(memory);
}
</listing>
<a href="../src/example/allocate_module_test.c">allocate_module_test.c</a>
<listing>
#include &lt;stdarg.h&gt;
#include &lt;stddef.h&gt;
#include &lt;setjmp.h&gt;
#include &lt;cmocka.h&gt;

extern void leak_memory();
extern void buffer_overflow();
extern void buffer_underflow();

// Test case that fails as leak_memory() leaks a dynamically allocated block.
void leak_memory_test(void **state) {
    leak_memory();
}

// Test case that fails as buffer_overflow() corrupts an allocated block.
void buffer_overflow_test(void **state) {
    buffer_overflow();
}

// Test case that fails as buffer_underflow() corrupts an allocated block.
void buffer_underflow_test(void **state) {
    buffer_underflow();
}

int main(int argc, char* argv[]) {
    const UnitTest tests[] = {
        unit_test(leak_memory_test),
        unit_test(buffer_overflow_test),
        unit_test(buffer_underflow_test),
    };
    return run_tests(tests);
}
</listing>

<a name="Mock_Functions"><h2>Mock Functions</h2></a>

<p>A unit test should ideally isolate the function or module being tested
from any external dependencies.  This can be performed using mock functions
that are either statically or dynamically linked with the module being tested.
Mock functions must be statically linked when the code being tested directly 
references external functions.  Dynamic linking is simply the process of 
setting a function pointer in a table used by the tested module to reference 
a mock function defined in the unit test.</p>

<a name="Return_Values"><h3>Return Values</h3></a>

<p>In order to simplify the implementation of mock functions cmocka provides
functionality which stores return values for mock functions in each test
case using <b>will_return()</b>.  These values are then returned by each mock 
function using calls to <b>mock()</b>.

Values passed to <b>will_return()</b> are added to a queue for each function 
specified.  Each successive call to <b>mock()</b> from a function removes a
return value from the queue.  This makes it possible for a mock function to use
multiple calls to <b>mock()</b> to return output parameters in addition to a
return value.  In addition this allows the specification of return values for 
multiple calls to a mock function.</p>

<h4>Using will_return()</h4>
<a name="../src/example/database.h" href="database.h">database.h</a>
<listing>
typedef struct DatabaseConnection DatabaseConnection;

/* Function that takes an SQL query string and sets results to an array of
 * pointers with the result of the query.  The value returned specifies the
 * number of items in the returned array of results.  The returned array of
 * results are statically allocated and should not be deallocated using free()
 */
typedef unsigned int (*QueryDatabase)(
    DatabaseConnection* const connection, const char * const query_string,
    void *** const results);

// Connection to a database.
struct DatabaseConnection {
    const char *url;
    unsigned int port;
    QueryDatabase query_database;
};

// Connect to a database.
DatabaseConnection* connect_to_database(const char * const url,
                                        const unsigned int port);
</listing>
<a href="../src/example/customer_database.c">customer_database.c</a>
<listing>
#include &lt;stddef.h&gt;
#include &lt;stdio.h&gt;
#include &lt;database.h&gt;
#ifdef _WIN32
#define snprintf _snprintf
#endif // _WIN32

// Connect to the database containing customer information.
DatabaseConnection* connect_to_customer_database() {
    return connect_to_database("customers.abcd.org", 321);
}

/* Find the ID of a customer by his/her name returning a value > 0 if
 * successful, 0 otherwise. */
unsigned int get_customer_id_by_name(
        DatabaseConnection * const connection,
        const char * const customer_name) {
    char query_string[256];
    int number_of_results;
    void **results;
    snprintf(query_string, sizeof(query_string),
             "SELECT ID FROM CUSTOMERS WHERE NAME = %s", customer_name);
    number_of_results = connection->query_database(connection, query_string,
                                                   &results);
    if (number_of_results != 1) {
        return -1;
    }
    return (unsigned int)results[0];
}
</listing>
<a href="../src/example/customer_database_test.c">customer_database_test.c</a>
<listing>
#include &lt;stdarg.h&gt;
#include &lt;stddef.h&gt;
#include &lt;setjmp.h&gt;
#include &lt;cmocka.h&gt;
#include &lt;database.h&gt;


extern DatabaseConnection* connect_to_customer_database();
extern unsigned int get_customer_id_by_name(
    DatabaseConnection * const connection, const char * const customer_name);

// Mock query database function.
unsigned int mock_query_database(
        DatabaseConnection* const connection, const char * const query_string,
        void *** const results) {
    *results = (void**)mock();
    return (unsigned int)mock();
}

// Mock of the connect to database function.
DatabaseConnection* connect_to_database(const char * const database_url,
                                        const unsigned int port) {
    return (DatabaseConnection*)mock();
}

void test_connect_to_customer_database(void **state) {
    will_return(connect_to_database, 0x0DA7ABA53);
    assert_true(connect_to_customer_database() ==
                (DatabaseConnection*)0x0DA7ABA53);
}

/* This test fails as the mock function connect_to_database() will have no
 * value to return. */
void fail_connect_to_customer_database(void **state) {
    will_return(connect_to_database, 0x0DA7ABA53);
    assert_true(connect_to_customer_database() ==
                (DatabaseConnection*)0x0DA7ABA53);
}

void test_get_customer_id_by_name(void **state) {
    DatabaseConnection connection = {
        "somedatabase.somewhere.com", 12345678, mock_query_database
    };
    // Return a single customer ID when mock_query_database() is called.
    int customer_ids = 543;
    will_return(mock_query_database, &customer_ids);
    will_return(mock_query_database, 1);
    assert_int_equal(get_customer_id_by_name(&connection, "john doe"), 543);
}

int main(int argc, char* argv[]) {
    const UnitTest tests[] = {
        unit_test(test_connect_to_customer_database),
        unit_test(fail_connect_to_customer_database),
        unit_test(test_get_customer_id_by_name),
    };
    return run_tests(tests);
}
</listing>

<a name="Checking_Parameters"><h3>Checking Parameters</h3></a>
<p>In addition to storing the return values of mock functions, cmocka
provides functionality to store expected values for mock function parameters
using the expect_*() functions provided.  A mock function parameter can then
be validated using the check_expected() macro.

<p>Successive calls to expect_*() macros for a parameter queues values to
check the specified parameter.  check_expected() checks a function parameter
against the next value queued using expect_*(), if the parameter check fails a
test failure is signalled.  In addition if check_expected() is called and
no more parameter values are queued a test failure occurs.</p>

<h4>Using expect_*()</h4>
<a href="../src/example/product_database.c">product_database.c</a>
<listing>
#include &lt;database.h&gt;

// Connect to the database containing customer information.
DatabaseConnection* connect_to_product_database() {
    return connect_to_database("products.abcd.org", 322);
}
</listing>
<a href="../src/example/product_database_test.c">product_database_test.c</a>
<listing>
#include &lt;stdarg.h&gt;
#include &lt;stddef.h&gt;
#include &lt;setjmp.h&gt;
#include &lt;cmocka.h&gt;
#include &lt;database.h&gt;

extern DatabaseConnection* connect_to_product_database();

/* Mock connect to database function.
 * NOTE: This mock function is very general could be shared between tests
 * that use the imaginary database.h module. */
DatabaseConnection* connect_to_database(const char * const url,
                                        const unsigned int port) {
    check_expected(url);
    check_expected(port);
    return (DatabaseConnection*)mock();
}

void test_connect_to_product_database(void **state) {
    expect_string(connect_to_database, url, "products.abcd.org");
    expect_value(connect_to_database, port, 322);
    will_return(connect_to_database, 0xDA7ABA53);
    assert_int_equal(connect_to_product_database(), 0xDA7ABA53);
}

/* This test will fail since the expected URL is different to the URL that is
 * passed to connect_to_database() by connect_to_product_database(). */
void test_connect_to_product_database_bad_url(void **state) {
    expect_string(connect_to_database, url, "products.abcd.com");
    expect_value(connect_to_database, port, 322);
    will_return(connect_to_database, 0xDA7ABA53);
    assert_int_equal((int)connect_to_product_database(), 0xDA7ABA53);
}

/* This test will fail since the mock connect_to_database() will attempt to
 * retrieve a value for the parameter port which isn't specified by this
 * test function. */
void test_connect_to_product_database_missing_parameter(void **state) {
    expect_string(connect_to_database, url, "products.abcd.org");
    will_return(connect_to_database, 0xDA7ABA53);
    assert_int_equal((int)connect_to_product_database(), 0xDA7ABA53);
}

int main(int argc, char* argv[]) {
    const UnitTest tests[] = {
        unit_test(test_connect_to_product_database),
        unit_test(test_connect_to_product_database_bad_url),
        unit_test(test_connect_to_product_database_missing_parameter),
    };
    return run_tests(tests);
}
</listing>

<a name="Test_State"><h2>Test State</h2></a>

<p>cmocka allows the specification of multiple setup and tear down functions
for each test case.  Setup functions, specified by the <b>unit_test_setup()</b>
or <b>unit_test_setup_teardown()</b> macros allow common initialization to be
shared between multiple test cases.  In addition, tear down functions,
specified by the <b>unit_test_teardown()</b> or
<b>unit_test_setup_teardown()</b> macros provide a code path that is always
executed for a test case even when it fails.</p>

<h4>Using unit_test_setup_teardown()</h4>
<a href="../src/example/key_value.c">key_value.c</a>
<listing>
#include &lt;stddef.h&gt;
#include &lt;stdlib.h&gt;
#include &lt;string.h&gt;

typedef struct KeyValue {
    unsigned int key;
    const char* value;
} KeyValue;

static KeyValue *key_values = NULL;
static unsigned int number_of_key_values = 0;

void set_key_values(KeyValue * const new_key_values,
                    const unsigned int new_number_of_key_values) {
    key_values = new_key_values;
    number_of_key_values = new_number_of_key_values;
}

// Compare two key members of KeyValue structures.
int key_value_compare_keys(const void *a, const void *b) {
    return (int)((KeyValue*)a)->key - (int)((KeyValue*)b)->key;
}

// Search an array of key value pairs for the item with the specified value.
KeyValue* find_item_by_value(const char * const value) {
  unsigned int i;
    for (i = 0; i < number_of_key_values; i++) {
        if (strcmp(key_values[i].value, value) == 0) {
            return &key_values[i];
        }
    }
    return NULL;
}

// Sort an array of key value pairs by key.
void sort_items_by_key() {
    qsort(key_values, number_of_key_values, sizeof(*key_values),
          key_value_compare_keys);
}
</listing>
<a href="../src/example/key_value_test.c">key_value_test.c</a>
<listing>
#include &lt;stdarg.h&gt;
#include &lt;stddef.h&gt;
#include &lt;setjmp.h&gt;
#include &lt;string.h&gt;
#include &lt;cmocka.h&gt;

/* This is duplicated here from the module setup_teardown.c to reduce the
 * number of files used in this test. */
typedef struct KeyValue {
    unsigned int key;
    const char* value;
} KeyValue;

void set_key_values(KeyValue * const new_key_values,
                    const unsigned int new_number_of_key_values);
extern KeyValue* find_item_by_value(const char * const value);
extern void sort_items_by_key();

static KeyValue key_values[] = {
    { 10, "this" },
    { 52, "test" },
    { 20, "a" },
    { 13, "is" },
};

void create_key_values(void **state) {
    KeyValue * const items = (KeyValue*)test_malloc(sizeof(key_values));
    memcpy(items, key_values, sizeof(key_values));
    *state = (void*)items;
    set_key_values(items, sizeof(key_values) / sizeof(key_values[0]));
}

void destroy_key_values(void **state) {
    test_free(*state);
    set_key_values(NULL, 0);
}

void test_find_item_by_value(void **state) {
    unsigned int i;
    for (i = 0; i < sizeof(key_values) / sizeof(key_values[0]); i++) {
        KeyValue * const found  = find_item_by_value(key_values[i].value);
        assert_true(found);
        assert_int_equal(found->key, key_values[i].key);
        assert_string_equal(found->value, key_values[i].value);
    }
}

void test_sort_items_by_key(void **state) {
    unsigned int i;
    KeyValue * const kv = *state;
    sort_items_by_key();
    for (i = 1; i < sizeof(key_values) / sizeof(key_values[0]); i++) {
        assert_true(kv[i - 1].key < kv[i].key);
    }
}

int main(int argc, char* argv[]) {
    const UnitTest tests[] = {
        unit_test_setup_teardown(test_find_item_by_value, create_key_values,
                                 destroy_key_values),
        unit_test_setup_teardown(test_sort_items_by_key, create_key_values,
                                 destroy_key_values),
    };
    return run_tests(tests);
}
</listing>

<a name="Example"><h2>Example</h2></a>

<p>A small command line calculator
<a href="../src/example/calculator.c">calculator.c</a> application
and test application that full exercises the calculator application
<a href="../src/example/calculator_test.c">calculator_test.c</a>
are provided as an example of cmocka's features discussed in this document.
</p>

<hr>
<address></address>
<!-- hhmts start --> Last modified: Wed Jul 22 12:11:43 PDT 2009 <!-- hhmts end -->
</body> </html>