RSS Feed
Twitter
Orkut
In a previous article I described how to get started testing C++ code with the Google Test framework. In this article I’m going to share a few tips and tricks for testing C code.
Here are some of the challenges we face when trying to test procedural code:
■We can’t create an instance of the code under test. This means that we can’t easily get a fresh object with initialized data for every test.
■The dependencies are hardcoded. This means we can’t use dependency injection techniques to mock/fake the module dependencies.
■We can’t use Polymorphism to break the dependencies
So that only leaves us with the two dependency-breaking tools available in the language: the preprocessor and the linker.
Things to watch out for
Static initialization: You need to be able to reset your data to a known state before running each test case. It is the only way to isolate your tests from each other.
Global variables: Is you module accessing a global variable? You need to provide a fake implementation for this.
Hardware Access: In embedded systems we often have memory mapped hardware register access. You most definitely don’t want to be dereferencing from random memory addresses in your tests. A good antidote to this is to define a generic function to get the address for a given register. You can then define a version of this function for testing purposes.
An example
So how does that look in practice? Suppose we have a make-believe embedded software application for controlling a device:
#include // here stdio.h
#include // here unistd.h
#define IOMEM_BASE 0x2FF
#define VALUE_REG (IOMEM_BASE + 3)
// This must be a power of 2!
#define BUFFER_SIZE 8
#define MAX_ITEMS (BUFFER_SIZE-1)
static int my_filter[BUFFER_SIZE];
static int readIdx = 0;
static int writeIdx = 0;
int filter_len()
{
return (BUFFER_SIZE + writeIdx - readIdx) % BUFFER_SIZE;
}
void filter_add(int val) {
my_filter[writeIdx] = val;
writeIdx = (writeIdx+1) & BUFFER_SIZE-1;
if(writeIdx == readIdx) readIdx = (readIdx+1) & BUFFER_SIZE-1;
}
#ifndef TESTING
int myapp_do_dangerous_io()
{
// lets dereference an io mapped register
// - on the target it is at address IOMEM_BASE + 3
return *((int *)VALUE_REG);
}
#endif
int myapp_get_average(){
int len = filter_len();
if(0 == len)
return 0;
int sum = 0;
for(int i = 0; i < len; i++)
{
sum += my_filter[(i+readIdx)%BUFFER_SIZE];
}
return sum/len;
}
int myapp_task()
{
// get value from register
int nextval = myapp_do_dangerous_io();
// add to filter line
filter_add(nextval);
// return the average value as the next delay
return myapp_get_average();
}
int myapp_mainloop()
{
for(;;)
{
int nextloopdelay = myapp_task();
sleep(nextloopdelay);
}
}
#ifndef TESTING
int main()
{
printf("!!!Hello World!!!\n");
return myapp_mainloop();
}
#endif
How do we approach testing this nastyness?
There are some challenges to testing code of this nature, but there are also methods we can use overcome them.
Infinite loops: these guys will destroy your ability to test effectively. The best approach is to move the body of any infinite loop into it’s own function call.
Dangerous Code: what you do on hardware in production can be dangerous to do in a testing environment. In this example we have a hardware access of memory mapped IO address.
There are three ways we can deal with dilemma:
1.change the address we dereference,
2.change the function we call (at link time)
3.hide the function we call during testing using #ifdefs and provide a test fake (this is the approach I have taken here)
Incompatible Function Names: You can’t link two main functions. You need to hide one…
Static Memory: This can really hurt the independence of your tests. You really ought to re-initialize all of your static data for each test case, and thankfully there is an easy way to achieve this. All the major testing frameworks have a concept of a test fixture which allows you to call a SetUp function before execution of each test case. Use this to initialize your static data. Remember: independent tests are good tests!
The General Testing Pattern
1. Define fake functions for the dependencies you want to stub out
2. If the module depends on a global (gasp!) you need to define your fake one
3. include your module implementation (#include module.c)
4. Define a method to reset all the static data to a known state.
5. Define your tests
#include // here gtest/gtest.h
// Hide main
#define TESTING
// Hide the io function since this will segfault in testing
int fake_register;
int myapp_do_dangerous_io()
{
return fake_register;
}
#include "myapp.c"
class MyAppTestSuite : public testing::Test
{
void SetUp(){
memset(&my_filter, 0, sizeof(my_filter));
readIdx = 0;
writeIdx = 0;
}
void TearDown(){}
};
TEST_F(MyAppTestSuite, myapp_task_should_return_correct_delay_for_one_element) {
fake_register = 10;
EXPECT_EQ(10, myapp_task());
}
TEST_F(MyAppTestSuite, myapp_task_should_return_correct_delay_for_two_elements) {
fake_register = 10;
myapp_task();
fake_register = 20;
EXPECT_EQ(15, myapp_task());
}
TEST_F(MyAppTestSuite, get_average_should_return_zero_on_empty_filter) {
ASSERT_EQ(0, myapp_get_average());
}
TEST_F(MyAppTestSuite, addFirstFilterValAddsVal) {
filter_add(42);
ASSERT_EQ(42, my_filter[readIdx]);
}
TEST_F(MyAppTestSuite, addFirstReturnsCorrectAverage) {
filter_add(42);
ASSERT_EQ(42, myapp_get_average());
}
TEST_F(MyAppTestSuite, addTwoValuesReturnsCorrectAverage) {
filter_add(42);
filter_add(40);
ASSERT_EQ(41, myapp_get_average());
}
TEST_F(MyAppTestSuite, get_average_should_return_average_of_full_filter) {
for(int i = 0; i < MAX_ITEMS; i++)
{
filter_add(i); } ASSERT_EQ((0+1+2+3+4+5+6)/MAX_ITEMS, myapp_get_average());
}
TEST_F(MyAppTestSuite, get_average_should_return_average_of_wrapped_filter)
{
for(int i = 0; i < BUFFER_SIZE; i++)
{
filter_add(i);
}
ASSERT_EQ((1+2+3+4+5+6+7)/MAX_ITEMS, myapp_get_average());
}
/// ....test buffer operations.....
When talking about testing C code (especially embedded) I often hear “But what about..”
■Timing Problems. That’s right, unit testing can’t magically simulate the runtime properties of your system.
■Interrupts. This is a special case of the last point, but it is the same issue all developers come across when going multi-threaded.
■Bit-correct operations. If you are running 24-bit code on a 32-bit architecture you will not see the exact same behavior for various overflow, underflow, bit-shifting and arithmetic operations.
■I can’t possibly test this! Well, there are some classes of code that simply cannot be tested using the unit testing methodology. In my experience however, it is an extreme minority of most code bases that this applies to. The secret is to factor out impossible-to-test-code as much as possible so you don’t pollute the rest of the codebase.
Summary
Testing C code is hard. Testing legacy C code is even harder. But with the limited dependency-breaking language features we have in C (the linker and the preprocessor) we can accomplish quite a lot.
Here are some of the challenges we face when trying to test procedural code:
■We can’t create an instance of the code under test. This means that we can’t easily get a fresh object with initialized data for every test.
■The dependencies are hardcoded. This means we can’t use dependency injection techniques to mock/fake the module dependencies.
■We can’t use Polymorphism to break the dependencies
So that only leaves us with the two dependency-breaking tools available in the language: the preprocessor and the linker.
Things to watch out for
Static initialization: You need to be able to reset your data to a known state before running each test case. It is the only way to isolate your tests from each other.
Global variables: Is you module accessing a global variable? You need to provide a fake implementation for this.
Hardware Access: In embedded systems we often have memory mapped hardware register access. You most definitely don’t want to be dereferencing from random memory addresses in your tests. A good antidote to this is to define a generic function to get the address for a given register. You can then define a version of this function for testing purposes.
An example
So how does that look in practice? Suppose we have a make-believe embedded software application for controlling a device:
#include // here stdio.h
#include // here unistd.h
#define IOMEM_BASE 0x2FF
#define VALUE_REG (IOMEM_BASE + 3)
// This must be a power of 2!
#define BUFFER_SIZE 8
#define MAX_ITEMS (BUFFER_SIZE-1)
static int my_filter[BUFFER_SIZE];
static int readIdx = 0;
static int writeIdx = 0;
int filter_len()
{
return (BUFFER_SIZE + writeIdx - readIdx) % BUFFER_SIZE;
}
void filter_add(int val) {
my_filter[writeIdx] = val;
writeIdx = (writeIdx+1) & BUFFER_SIZE-1;
if(writeIdx == readIdx) readIdx = (readIdx+1) & BUFFER_SIZE-1;
}
#ifndef TESTING
int myapp_do_dangerous_io()
{
// lets dereference an io mapped register
// - on the target it is at address IOMEM_BASE + 3
return *((int *)VALUE_REG);
}
#endif
int myapp_get_average(){
int len = filter_len();
if(0 == len)
return 0;
int sum = 0;
for(int i = 0; i < len; i++)
{
sum += my_filter[(i+readIdx)%BUFFER_SIZE];
}
return sum/len;
}
int myapp_task()
{
// get value from register
int nextval = myapp_do_dangerous_io();
// add to filter line
filter_add(nextval);
// return the average value as the next delay
return myapp_get_average();
}
int myapp_mainloop()
{
for(;;)
{
int nextloopdelay = myapp_task();
sleep(nextloopdelay);
}
}
#ifndef TESTING
int main()
{
printf("!!!Hello World!!!\n");
return myapp_mainloop();
}
#endif
How do we approach testing this nastyness?
There are some challenges to testing code of this nature, but there are also methods we can use overcome them.
Infinite loops: these guys will destroy your ability to test effectively. The best approach is to move the body of any infinite loop into it’s own function call.
Dangerous Code: what you do on hardware in production can be dangerous to do in a testing environment. In this example we have a hardware access of memory mapped IO address.
There are three ways we can deal with dilemma:
1.change the address we dereference,
2.change the function we call (at link time)
3.hide the function we call during testing using #ifdefs and provide a test fake (this is the approach I have taken here)
Incompatible Function Names: You can’t link two main functions. You need to hide one…
Static Memory: This can really hurt the independence of your tests. You really ought to re-initialize all of your static data for each test case, and thankfully there is an easy way to achieve this. All the major testing frameworks have a concept of a test fixture which allows you to call a SetUp function before execution of each test case. Use this to initialize your static data. Remember: independent tests are good tests!
The General Testing Pattern
1. Define fake functions for the dependencies you want to stub out
2. If the module depends on a global (gasp!) you need to define your fake one
3. include your module implementation (#include module.c)
4. Define a method to reset all the static data to a known state.
5. Define your tests
#include // here gtest/gtest.h
// Hide main
#define TESTING
// Hide the io function since this will segfault in testing
int fake_register;
int myapp_do_dangerous_io()
{
return fake_register;
}
#include "myapp.c"
class MyAppTestSuite : public testing::Test
{
void SetUp(){
memset(&my_filter, 0, sizeof(my_filter));
readIdx = 0;
writeIdx = 0;
}
void TearDown(){}
};
TEST_F(MyAppTestSuite, myapp_task_should_return_correct_delay_for_one_element) {
fake_register = 10;
EXPECT_EQ(10, myapp_task());
}
TEST_F(MyAppTestSuite, myapp_task_should_return_correct_delay_for_two_elements) {
fake_register = 10;
myapp_task();
fake_register = 20;
EXPECT_EQ(15, myapp_task());
}
TEST_F(MyAppTestSuite, get_average_should_return_zero_on_empty_filter) {
ASSERT_EQ(0, myapp_get_average());
}
TEST_F(MyAppTestSuite, addFirstFilterValAddsVal) {
filter_add(42);
ASSERT_EQ(42, my_filter[readIdx]);
}
TEST_F(MyAppTestSuite, addFirstReturnsCorrectAverage) {
filter_add(42);
ASSERT_EQ(42, myapp_get_average());
}
TEST_F(MyAppTestSuite, addTwoValuesReturnsCorrectAverage) {
filter_add(42);
filter_add(40);
ASSERT_EQ(41, myapp_get_average());
}
TEST_F(MyAppTestSuite, get_average_should_return_average_of_full_filter) {
for(int i = 0; i < MAX_ITEMS; i++)
{
filter_add(i); } ASSERT_EQ((0+1+2+3+4+5+6)/MAX_ITEMS, myapp_get_average());
}
TEST_F(MyAppTestSuite, get_average_should_return_average_of_wrapped_filter)
{
for(int i = 0; i < BUFFER_SIZE; i++)
{
filter_add(i);
}
ASSERT_EQ((1+2+3+4+5+6+7)/MAX_ITEMS, myapp_get_average());
}
/// ....test buffer operations.....
When talking about testing C code (especially embedded) I often hear “But what about..”
■Timing Problems. That’s right, unit testing can’t magically simulate the runtime properties of your system.
■Interrupts. This is a special case of the last point, but it is the same issue all developers come across when going multi-threaded.
■Bit-correct operations. If you are running 24-bit code on a 32-bit architecture you will not see the exact same behavior for various overflow, underflow, bit-shifting and arithmetic operations.
■I can’t possibly test this! Well, there are some classes of code that simply cannot be tested using the unit testing methodology. In my experience however, it is an extreme minority of most code bases that this applies to. The secret is to factor out impossible-to-test-code as much as possible so you don’t pollute the rest of the codebase.
Summary
Testing C code is hard. Testing legacy C code is even harder. But with the limited dependency-breaking language features we have in C (the linker and the preprocessor) we can accomplish quite a lot.
