Skip to content

Test reports ​

Collect test results from infrastructure jobs and investigate failures in a run's Tests tab. Ordyn accepts JUnit XML and TeamCity test service messages from Runner scripts and executables, including tasks running in Docker.

Job Designer can configure reporting; Job Operator can review results.

Collect JUnit XML ​

Configure your test runner to write JUnit XML into the shared job workspace. In the job editor, open Features, enable Test reports, and enter Report paths. For example:

text
+:**/junit*.xml
-:node_modules/**

Paths are relative to the workspace, including for Docker tasks. Enter individual files, directories, or Ant patterns, separated by lines or commas. * matches within a directory, ** spans directories, and -: excludes matches. Use a unique filename for each report and keep it available until collection finishes. Absolute paths, parent-directory traversal, and symbolic links are unsupported.

Choose When no reports are found according to whether XML output is required. Fail the run when tests fail makes reported failures fail an otherwise successful run; disable it when results are informational. Collection or parsing errors still fail an otherwise successful run.

For Docker tasks, XML reports are collected after the task finishes. Keep report files in the workspace until collection completes; files deleted during the task cannot be collected. Standard output, standard error, and direct test service messages remain live during execution.

Configured paths apply throughout the job. Collection finishes before workspace cleanup, including after failure, cancellation, or timeout. Reporting preserves an existing failure, cancellation, or timeout outcome.

Register reports from a script ​

A script can register reports by writing a TeamCity service message to standard output:

text
##teamcity[importData type='junit' path='reports/**/*.xml' whenNoDataPublished='error']

Emit the message in the task that produces the reports. Its paths apply until that task finishes, so the XML must be ready by then. Messages can register files before they exist or while they are growing. An import message works without manually configured report paths; enable Test reports with empty paths when you want to configure whether failed tests fail the run.

Ordyn skips files whose modification time predates the run. Add parseOutOfDate='true' when importing an existing report is intentional. whenNoDataPublished accepts nothing, info, warning, or error; error makes missing reports fail the run. Only type='junit' is supported.

Publish tests directly ​

Test tools that emit TeamCity test messages can publish results without XML:

text
##teamcity[testSuiteStarted name='Checkout']
##teamcity[testStarted name='accepts a valid order']
##teamcity[testFailed name='accepts a valid order' message='Unexpected total' details='Expected 12, received 10']
##teamcity[testFinished name='accepts a valid order' duration='42']
##teamcity[testSuiteFinished name='Checkout']

Write one complete message per line to standard output. duration is in milliseconds. Attribute values use TeamCity escaping: || for a pipe, |' for an apostrophe, |n for a newline, |r for a carriage return, and |[ or |] for brackets.

Use testIgnored for skipped tests and testStdOut or testStdErr with an out attribute for test output. Parallel tests need distinct flowId values on their messages. With testRetrySupport enabled='true', a later passing attempt mutes earlier direct-message failures with the same suite and name. XML results remain separate; publishing the same test through both methods counts it twice.

Review results and XML ​

Open an infrastructure run in Operations → Infrastructure Jobs or Reporting → Job Runs. Tests appears when results are available. Search or filter the results and select a test to inspect its failure and output.

Open Reports beside the run tabs to inspect collection problems and download XML. Incomplete reporting retains successfully received tests, but those results do not establish that the whole test run passed.

Original XML is retained as a job artifact, with Download XML available to users who can download the run's artifacts. The download includes content omitted from the Tests view. XML follows artifact access and malware-scanning requirements; deleting or pruning the run also removes its reports and standalone XML artifacts. See run retention.

Large reports and limitations ​

The default limits allow 1 GiB per XML file, 10 GiB of XML, 10,000 report files, and one million test cases per run. Administrators can adjust these limits. Individual stored test fields are limited to 8 MiB and marked when truncated; the original XML keeps the collected content. A TeamCity service message can contain up to 16 MiB.

Collection has a 30-minute finalization deadline by default. If a Runner remains unavailable or collection cannot finish, the run retains its received results and reports incomplete collection.

JUnit XML must contain a testsuite or testsuites root. Nested suites and XML namespaces are supported. DTDs and custom entities are rejected. Files may grow during collection, but rewriting or replacing collected content is unsupported; give separate test invocations separate report filenames. Test reporting applies to infrastructure jobs; endpoint jobs do not collect these results.

XML artifacts collected through job-wide report paths are labeled Job-level report. Reports registered by a step retain that step association.