Post-processing outputs

`XML output files`_ that are generated during the test execution can be post-processed afterwards by the Rebot tool, which is an integral part of Robot Framework. It is used automatically when test reports and logs are generated during the test execution, and using it separately allows creating custom reports and logs as well as combining and merging results.

Using Rebot

Synopsis

rebot [options] robot_outputs
python|jython|ipy -m robot.rebot [options] robot_outputs
python|jython|ipy path/to/robot/rebot.py [options] robot_outputs
java -jar robotframework.jar rebot [options] robot_outputs

The most common way to use Rebot is using the rebot `runner script`_. Alternatively it is possible to execute the installed `robot.rebot module`__ or `robot/rebot.py file`__ directly using the selected interpreter. The final alternative is using the `standalone JAR distribution`_.

注釈

Versions prior to Robot Framework 3.0 installed the rebot script only with Python and used jyrebot and ipyrebot scripts with Jython and IronPython, respectively. These scripts are still installed, but the plan is to deprecate and remove them in the future.

Specifying options and arguments

The basic syntax for using Rebot is exactly the same as when `starting test execution`_ and also most of the command line options are identical. The main difference is that arguments to Rebot are `XML output files`_ instead of test data files or directories.

Return codes with Rebot

Return codes from Rebot are exactly same as when `running tests`__.

Creating different reports and logs

You can use Rebot for creating the same reports and logs that are created automatically during the test execution. Of course, it is not sensible to create the exactly same files, but, for example, having one report with all test cases and another with only some subset of tests can be useful:

rebot output.xml
rebot path/to/output_file.xml
rebot --include smoke --name Smoke_Tests c:\results\output.xml

Another common usage is creating only the output file when running tests (log and report generation can be disabled with --log NONE --report NONE) and generating logs and reports later. Tests can, for example, be executed on different environments, output files collected to a central place, and reports and logs created there. This approach can also work very well if generating reports and logs takes a lot of time when running tests on Jython. Disabling log and report generation and generating them later with Rebot can save a lot of time and use less memory.

Combining outputs

An important feature in Rebot is its ability to combine outputs from different test execution rounds. This capability allows, for example, running the same test cases on different environments and generating an overall report from all outputs. Combining outputs is extremely easy, all that needs to be done is giving several output files as arguments:

rebot output1.xml output2.xml
rebot outputs/*.xml

When outputs are combined, a new top-level test suite is created so that test suites in the given output files are its child suites. This works the same way when `multiple test data files or directories are executed`__, and also in this case the name of the top-level test suite is created by joining child suite names with an ampersand (&) and spaces. These automatically generated names are not that good, and it is often a good idea to use --name to give a more meaningful name:

rebot --name Browser_Compatibility firefox.xml opera.xml safari.xml ie.xml
rebot --include smoke --name Smoke_Tests c:\results\*.xml

Merging outputs

If same tests are re-executed or a single test suite executed in pieces, combining results like discussed above creates an unnecessary top-level test suite. In these cases it is typically better to merge results instead. Merging is done by using --merge option which changes the way how Rebot combines two or more output files. This option itself takes no arguments and all other command line options can be used with it normally:

rebot --merge --name Example --critical regression original.xml merged.xml

How merging works in practice is explained in the following sections discussing its two main use cases.

Merging re-executed tests

There is often a need to re-execute a subset of tests, for example, after fixing a bug in the system under test or in the tests themselves. This can be accomplished by `selecting test cases`_ by names (--test and --suite options), tags (--include and --exclude), or by previous status (--rerunfailed).

Combining re-execution results with the original results using the default combining outputs approach does not work too well. The main problem is that you get separate test suites and possibly already fixed failures are also shown. In this situation it is better to use --merge (-R) option to tell Rebot to merge the results instead. In practice this means that tests from the latter test runs replace tests in the original. The usage is best illustrated by a practical example using --rerunfailed and --merge together:

robot --output original.xml tests                          # first execute all tests
robot --rerunfailed original.xml --output rerun.xml tests  # then re-execute failing
rebot --merge original.xml rerun.xml                       # finally merge results

The message of the merged tests contains a note that results have been replaced. The message also shows the old status and message of the test.

Merged results must always have same top-level test suite. Tests and suites in merged outputs that are not found from the original output are added into the resulting output. How this works in practice is discussed in the next section.

注釈

Merging re-executed results is a new feature in Robot Framework 2.8.4. Prior to Robot Framework 2.8.6 new tests or suites in merged outputs were skipped and merging was done using nowadays deprecated --rerunmerge option.

Merging suites executed in pieces

Another important use case for the --merge option is merging results got when running a test suite in pieces using, for example, --include and --exclude options:

robot --include smoke --output smoke.xml tests   # first run some tests
robot --exclude smoke --output others.xml tests  # then run others
rebot --merge smoke.xml others.xml               # finally merge results

When merging outputs like this, the resulting output contains all tests and suites found from all given output files. If some test is found from multiple outputs, latest results replace the earlier ones like explained in the previous section. Also this merging strategy requires the top-level test suites to be same in all outputs.