Get Help

JSON

The whole run as one JSON document: the counts, and every result with its message, timing and the definition that produced it.

When to use it

For anything that reads the results by program: a pipeline step that fails the build on a failed test, a script that mails someone when a suite’s pass rate drops, a job that loads results somewhere CAT has no output for. JSON is read natively by PowerShell, Python and every CI platform. It carries everything — the complete test definition included — so nothing has to be looked up in the project file afterwards. For a human reading the file, YAML is the same data in a friendlier shape; for a test report in a pipeline, JUnit or TRX.

How to set it 

Output: json

writes TestResults/cat-test-results-{timestamp}.json next to the project file. To choose the name:

Output:
- Format: json
  File: results/latest.json

See Settings. Not available on the Starter and Professional plans (MS Excel only).

What it looks like

The sample run — the first result in full, the other two elided:

{
  "executionGuid": "78084065-0bde-4628-b49f-f977c0f803df",
  "startedOn": "2026-08-21T09:06:37.4369592+00:00",
  "finishedOn": "2026-08-21T09:06:44.7333889+00:00",
  "duration": "00:00:07.2964297",
  "results": [
    {
      "executionGuid": "2ecab40f-fbcd-4d94-a2a0-28b1134f9a5c",
      "testResult": "Passed",
      "message": "The sets match.\n\nDescription: Every airline in the source must land in the dimension, nothing more.\nFirst Data Source: DWH\nFirst Query:\n\nSELECT 'CSA' AS Code UNION ALL SELECT 'LH' ORDER BY Code\n\nSecond Data Source: DWH\nSecond Query:\n\nSELECT 'CSA' AS Code UNION ALL SELECT 'LH' ORDER BY Code\n\n",
      "rawMessage": "The sets match.",
      "startTime": "2026-08-21T09:06:38.2510955+00:00",
      "endTime": "2026-08-21T09:06:38.3266711+00:00",
      "duration": "00:00:00.0755756",
      "exceptionMessage": null,
      "exceptions": null,
      "testDefinition": {
        "testDefinitionID": "aa6f2ccd-6e99-4c0e-8f4d-775c7904a628",
        "suite": "Smoke tests",
        "name": "Airline dimension matches the source",
        "testFullName": "[Smoke tests].[Airline dimension matches the source]",
        "description": "Every airline in the source must land in the dimension, nothing more.",
        "firstDataSource": "DWH",
        "firstQuery": "SELECT 'CSA' AS Code UNION ALL SELECT 'LH' ORDER BY Code",
        "secondDataSource": "DWH",
        "secondQuery": "SELECT 'CSA' AS Code UNION ALL SELECT 'LH' ORDER BY Code",
        "expectation": "sets match",
        "timeout": 0,
        "maximumErrorsLogged": 1,
        "tags": [],
        "settings": {
          "Name": "Airline dimension matches the source",
          "Suite": "Smoke tests",
          "Description": "Every airline in the source must land in the dimension, nothing more.",
          "First data source": "DWH",
          "First query": "SELECT 'CSA' AS Code UNION ALL SELECT 'LH' ORDER BY Code",
          "Second data source": "DWH",
          "Second query": "SELECT 'CSA' AS Code UNION ALL SELECT 'LH' ORDER BY Code",
          "Expectation": "sets match"
        }
      },
      "testDefinitionGuid": "aa6f2ccd-6e99-4c0e-8f4d-775c7904a628",
      "testName": "Airline dimension matches the source",
      "testFullName": "[Smoke tests].[Airline dimension matches the source]",
      "testSuite": "Smoke tests",
      "testCase": null,
      "description": "Every airline in the source must land in the dimension, nothing more.",
      "firstDataSource": "DWH",
      "firstQuery": "SELECT 'CSA' AS Code UNION ALL SELECT 'LH' ORDER BY Code",
      "secondDataSource": "DWH",
      "secondQuery": "SELECT 'CSA' AS Code UNION ALL SELECT 'LH' ORDER BY Code",
      "expectation": "sets match",
      "numberOfErrors": null,
      "threadNumber": 0
    },
    { "testResult": "Failed", "rawMessage": "No row was expected, but at least 1 row exists. …", "…": "…" },
    { "testResult": "Error", "exceptionMessage": "Invalid object name 'dbo.Flights'.", "…": "…" }
  ],
  "passedCount": 1,
  "failedCount": 1,
  "errorCount": 1,
  "inconclusiveCount": 0,
  "totalCount": 3,
  "passRate": 33.3,
  "failedRate": 33.3,
  "errorRate": 33.3,
  "inconclusiveRate": 0
}

Details

One document, the run at the top. executionGuid, startedOn, finishedOn, duration, then results — one object per test — and after them the counts and rates of the whole run (passedCount, failedCount, errorCount, inconclusiveCount, totalCount, passRate, …). Keys are camelCase.

One object per result. testResult (Passed · Failed · Error · Inconclusive), message and rawMessage as described on Properties, startTime / endTime / duration, threadNumber, numberOfErrors, exceptionMessage plus exceptions — the exception chain with type and stack trace, for Error only — and testDefinition: the complete definition as CAT resolved it, including settings, the properties exactly as they were written in the source. Properties of the definition come twice: resolved (firstQuery) and as written before environment variables were expanded (firstQueryRaw). The shown sample leaves most of the definition out; the file has all of it.

Dates and durations. ISO 8601 with offset for the times (2026-08-21T09:06:38.2510955+00:00), hh:mm:ss.fffffff for durations.

The file starts with a byte-order mark (UTF-8 BOM). PowerShell’s ConvertFrom-Json and .NET readers skip it; Python’s json.load does not — open the file with encoding="utf-8-sig".

$run = Get-Content TestResults/latest.json -Raw | ConvertFrom-Json
$run.results | Where-Object testResult -ne Passed | Select-Object testFullName, testResult, rawMessage
if ($run.failedCount + $run.errorCount -gt 0) { exit 1 }

import json
with open("TestResults/latest.json", encoding="utf-8-sig") as f:
    run = json.load(f)
failed = [r for r in run["results"] if r["testResult"] != "Passed"]

Written once, after the run, whatever the results; a run that is killed leaves no file. A pipeline that must fail on a failed test reads failedCount and errorCount from here — no CAT tool turns results into an exit code on its own.

  • YAML — the same document in YAML.
  • Properties — what each field holds.
  • Results — the result values.
← Properties
YAML →