XML Formats

Reporting test results in the XML format is the preferred way since it includes more fields and can also be used for NFT test reporting. Currently MeeGo test-definition and Google Test Framework XML report formats are supported. Apache Ant JUnit task reports may works as well since Google Test FW report is based on that one.

Google Test FW reports are currently interpreted so that if the status of a test case is run and it has no <error> or <failure> nodes as children the test case result is Pass, and if it has those child nodes the result is Fail. If status is something else than run the result will be N/A.

CSV format can also be used, but it is not preferred.

Simple MeeGo test-definition Example

<?xml version="1.0" encoding="utf-8"?>
<testresults>
  <suite name="Bluetooth">
    <set name="Functional tests" feature="BT">
      <case name="BT Device Scan" TC_ID="BT_SCAN_1284" result="PASS"/>
      <case name="BT Device Pair" TC_ID="BT_PAIR_5AD" result="FAIL"/>
    </set>
  </suite>
</testresults>

Extended options

QA Reports has added support for reporting test metrics (on test report level) using XML input files. Notice: XML files with additional metrics do not comply to MeeGo test-definition schema so you cannot use the same files with other MeeGo test tools. If metrics are reported the created report contains section Metrics above the Test Results section (both sections can be hidden if needed).

Metrics are grouped in both the XML file and on the created report. Selected metrics can be drawn on a line chart (current + two previous values included as in the Result Summary section). An example XML file with metrics and test cases could look like this:

<?xml version="1.0" encoding="utf-8"?>
<testresults>
  <metrics>
    <group name="Reliability Summary">
      <metric name="First crash" value="38" unit="hours" chart="true"/>
      <metric name="Average CPU load" value="37" unit="%"/>
    </group>
    <group name="Load Summary">
      <metric name="Response time under load" value="5.9" unit="ms" chart="true"/>
    </group>
  </metrics>
  <suite name="Suite">
    <set name="Set">
      <case name="Case" result="PASS"/>
    </set>
  </suite>
</testresults>

Simple Google Test FW Example

<?xml version="1.0" encoding="utf-8"?>
<testsuites name="Bluetooth">
  <testsuite name="Functional tests">
    <testcase name="BT Device Scan" status="run"/>
    <testcase name="BT Device Pair" status="run">
      <error message="Pairing failed"/>
    </testcase>
  </testsuite>
</testsuites>