| <html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>D-Bus Test Plan</title><meta name="generator" content="DocBook XSL Stylesheets V1.75.2"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="article" title="D-Bus Test Plan"><div class="titlepage"><div><div><h2 class="title"><a name="index"></a>D-Bus Test Plan</h2></div><div><div class="authorgroup"><div class="author"><h3 class="author"><span class="firstname">Anders</span> <span class="surname">Carlsson</span></h3><div class="affiliation"><span class="orgname">CodeFactory AB<br></span><div class="address"><p><code class="email"><<a class="email" href="mailto:andersca@codefactory.se">andersca@codefactory.se</a>></code></p></div></div></div></div></div></div><hr></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="sect1"><a href="#introduction">Introduction</a></span></dt><dd><dl><dt><span class="sect2"><a href="#importance-of-testing">The importance of testing</a></span></dt></dl></dd><dt><span class="sect1"><a href="#client-library">Testing the D-Bus client library</a></span></dt><dd><dl><dt><span class="sect2"><a href="#data-structures">Data Structures</a></span></dt><dt><span class="sect2"><a href="#message-loader">Message loader</a></span></dt><dt><span class="sect2"><a href="#authentication">Authentication</a></span></dt></dl></dd><dt><span class="sect1"><a href="#daemon">Testing the D-Bus bus daemon</a></span></dt><dd><dl><dt><span class="sect2"><a href="#debug-transport">The debug transport</a></span></dt><dt><span class="sect2"><a href="#bus-test">The bus-test program</a></span></dt></dl></dd><dt><span class="sect1"><a href="#other-tests">Other tests</a></span></dt><dd><dl><dt><span class="sect2"><a href="#oom-robustness">Out-Of-Memory robustness</a></span></dt><dt><span class="sect2"><a href="#leaks-and-other-stuff">Memory leaks and code robustness</a></span></dt></dl></dd></dl></div><div class="sect1" title="Introduction"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="introduction"></a>Introduction</h2></div></div></div><p> | |
| This document tries to explain the details of the test plan for D-Bus | |
| </p><div class="sect2" title="The importance of testing"><div class="titlepage"><div><div><h3 class="title"><a name="importance-of-testing"></a>The importance of testing</h3></div></div></div><p> | |
| As with any big library or program, testing is important. It | |
| can help find bugs and regressions and make the code better | |
| overall. | |
| </p><p> | |
| D-Bus is a large and complex piece of software (about 25,000 | |
| lines of code for the client library, and 2,500 lines of code | |
| for the bus daemon) and it's therefore important to try to make sure | |
| that all parts of the software is functioning correctly. | |
| </p><p> | |
| D-Bus can be built with support for testing by passing | |
| <code class="literal">--enable-tests</code>. to the configure script. It | |
| is recommended that production systems build without testing | |
| since that reduces the D-Bus client library size. | |
| </p></div></div><div class="sect1" title="Testing the D-Bus client library"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="client-library"></a>Testing the D-Bus client library</h2></div></div></div><p> | |
| The tests for the client library consist of the dbus-test | |
| program which is a unit test for all aspects of the client | |
| library. Whenever a bug in the client library is found and | |
| fixed, a test is added to make sure that the bug won't occur again. | |
| </p><div class="sect2" title="Data Structures"><div class="titlepage"><div><div><h3 class="title"><a name="data-structures"></a>Data Structures</h3></div></div></div><p> | |
| The D-Bus client library consists of some data structures that | |
| are used internally; a linked list class, a hashtable class and | |
| a string class. All aspects of those are tested by dbus-test. | |
| </p></div><div class="sect2" title="Message loader"><div class="titlepage"><div><div><h3 class="title"><a name="message-loader"></a>Message loader</h3></div></div></div><p> | |
| The message loader is the part of D-Bus that takes messages in | |
| raw character form and parses them, turning them into DBusMessages. | |
| </p><p> | |
| This is one of the parts of D-Bus that | |
| <span class="emphasis"><em>must</em></span> be absolutely bug-free and | |
| robust. The message loader should be able to handle invalid | |
| and incomplete messages without crashing. Not doing so is a | |
| serious issue and can easily result in D-Bus being exploitable | |
| to DoS attacks. | |
| </p><p> | |
| To solve these problems, there is a testing feature called the | |
| Message Builder. The message builder can take a serialized | |
| message in string-form and convert it into a raw character | |
| string which can then be loaded by the message loader. | |
| </p><div class="figure"><a name="idp59024"></a><p class="title"><b>Figure 1. Example of a message in string form</b></p><div class="figure-contents"><pre class="programlisting"> | |
| # Standard org.freedesktop.DBus.Hello message | |
| VALID_HEADER | |
| FIELD_NAME name | |
| TYPE STRING | |
| STRING 'org.freedesktop.DBus.Hello' | |
| FIELD_NAME srvc | |
| TYPE STRING | |
| STRING 'org.freedesktop.DBus' | |
| ALIGN 8 | |
| END_LENGTH Header | |
| START_LENGTH Body | |
| END_LENGTH Body | |
| </pre></div></div><br class="figure-break"><p> | |
| The file format of messages in string form is documented in | |
| the D-Bus Reference Manual. | |
| </p><p> | |
| The message test part of dbus-test is using the message | |
| builder to build different kinds of messages, both valid, | |
| invalid, and invalid ones, to make sure that the loader won't | |
| crash or leak memory of any of those, and that the loader | |
| knows if a message is valid or not. | |
| </p><p> | |
| There is also a test program called | |
| <code class="literal">break-loader</code> that loads a message in | |
| string-form into raw character form using the message | |
| builder. It then randomly changes the message, it can for | |
| example replace single bytes of data or modify the length of | |
| the message. This is to simulate network errors. The | |
| break-loader program saves all the messages leading to errors | |
| so it can easily be run for a long period of time. | |
| </p></div><div class="sect2" title="Authentication"><div class="titlepage"><div><div><h3 class="title"><a name="authentication"></a>Authentication</h3></div></div></div><p> | |
| For testing authentication, there is a testing feature that | |
| can read authentication sequences from a file and play them | |
| back to a dummy server and client to make sure that | |
| authentication is working according to the specification. | |
| </p><div class="figure"><a name="idp65712"></a><p class="title"><b>Figure 2. Example of an authentication script</b></p><div class="figure-contents"><pre class="programlisting"> | |
| ## this tests a successful auth of type EXTERNAL | |
| SERVER | |
| SEND 'AUTH EXTERNAL USERNAME_HEX' | |
| EXPECT_COMMAND OK | |
| EXPECT_STATE WAITING_FOR_INPUT | |
| SEND 'BEGIN' | |
| EXPECT_STATE AUTHENTICATED | |
| </pre></div></div><br class="figure-break"></div></div><div class="sect1" title="Testing the D-Bus bus daemon"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="daemon"></a>Testing the D-Bus bus daemon</h2></div></div></div><p> | |
| Since the D-Bus bus daemon is using the D-Bus client library it | |
| will benefit from all tests done on the client library, but | |
| there is still the issue of testing client-server communication. | |
| This is more complicated since it it may require another process | |
| running. | |
| </p><div class="sect2" title="The debug transport"><div class="titlepage"><div><div><h3 class="title"><a name="debug-transport"></a>The debug transport</h3></div></div></div><p> | |
| In D-Bus, a <span class="emphasis"><em>transport</em></span> is a class that | |
| handles sending and receiving raw data over a certain | |
| medium. The transport that is used most in D-Bus is the UNIX | |
| transport with sends and recevies data over a UNIX socket. A | |
| transport that tunnels data through X11 client messages is | |
| also under development. | |
| </p><p> | |
| The D-Bus debug transport is a specialized transport that | |
| works in-process. This means that a client and server that | |
| exists in the same process can talk to eachother without using | |
| a socket. | |
| </p></div><div class="sect2" title="The bus-test program"><div class="titlepage"><div><div><h3 class="title"><a name="bus-test"></a>The bus-test program</h3></div></div></div><p> | |
| The bus-test program is a program that is used to test various | |
| parts of the D-Bus bus daemon; robustness and that it conforms | |
| to the specifications. | |
| </p><p> | |
| The test program has the necessary code from the bus daemon | |
| linked in, and it uses the debug transport for | |
| communication. This means that the bus daemon code can be | |
| tested without the real bus actually running, which makes | |
| testing easier. | |
| </p><p> | |
| The bus-test program should test all major features of the | |
| bus, such as service registration, notification when things | |
| occurs and message matching. | |
| </p></div></div><div class="sect1" title="Other tests"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="other-tests"></a>Other tests</h2></div></div></div><div class="sect2" title="Out-Of-Memory robustness"><div class="titlepage"><div><div><h3 class="title"><a name="oom-robustness"></a>Out-Of-Memory robustness</h3></div></div></div><p> | |
| Since D-Bus should be able to be used in embedded devices, and | |
| also as a system service, it should be able to cope with | |
| low-memory situations without exiting or crashing. | |
| </p><p> | |
| In practice, this means that both the client and server code | |
| must be able to handle dbus_malloc returning NULL. | |
| </p><p> | |
| To test this, two environment variables | |
| exist. <code class="literal">DBUS_MALLOC_FAIL_NTH</code> will make every | |
| nth call to dbus_malloc return NULL, and | |
| <code class="literal">DBUS_MALLOC_FAIL_GREATER_THAN</code> will make any | |
| dbus_malloc call with a request for more than the specified | |
| number of bytes fail. | |
| </p></div><div class="sect2" title="Memory leaks and code robustness"><div class="titlepage"><div><div><h3 class="title"><a name="leaks-and-other-stuff"></a>Memory leaks and code robustness</h3></div></div></div><p> | |
| Naturally there are some things that tests can't be written | |
| for, for example things like memory leaks and out-of-bounds | |
| memory reading or writing. | |
| </p><p> | |
| Luckily there exists good tools for catching such errors. One | |
| free good tool is <a class="ulink" href="http://devel-home.kde.org/~sewardj/" target="_top">Valgrind</a>, which runs the program in a | |
| virtual CPU which makes catching errors easy. All test programs can be run under Valgrind, | |
| </p></div></div></div></body></html> |