Testing a project¶
You can add tests to your projects using the qi_add_test
functions.
You can then run the tests with:
$ qibuild test foo
Note for CTest users¶
qibuild test
does not use ctest
executable, and qi_add_test
is
not calling upstream add_test
function.
The reasons for this are:
qibuild test
is just actually callingqitest run
which is able to run any test executable, not just those coming from aCMake
project.- we needed to be able to run the tests after they have been deployed, and
CTest
does not support this use case - Having a test runner implemented in Python made it possible for us to implement
more features such as running tests with
valgrind
, or specify a CPU mask for each test.
This means that you cannot use set_test_properties
for a test created
with qi_add_test
Also note that qibuild test
is intended to be used with Jenkins, instead of CDash
.
qibuild test features¶
qibuild test
will always generate JUnit-like XML files toproject/build-tests/results
, so you do not have to use any test framework to generate the XML for you.- The tests are run from
<build>/sdk/bin
by default. - If your test is a simple executable and you only care about the return code, the generated XML will contain the full output of the program and the return code.
<testsuite name="test">
<testcase name="test_foo" status="run">
<failure message="Return code: 2">
<![CDATA[
ERROR: ...
]]>
</failure>
</testcase>
</testsuite>
- If you use
qi_create_gtest
, the test will be called with the correct--gtest-output
function for you - If your GTest test times out, or segfaults, it sometimes leaves an invalid XML
file behind. But
qibuild test
will handle this case nicely, producing a nice XML Junit files with an error message about the time out or the segfault.
<testsuite name="test">
<testcase name="test_foo" status="run">
<failure message="Timed out (2 s)">
</failure>
</testcase>
</testsuite>