get_test_results

Returns the result of every test of the last run in this process — verdict, message, duration, the queries — as objects.

Signature #

cat.get_test_results()

Parameters #

None.

What it does #

Returns the per-test results CAT kept from the last invoke_project / invoke_tests in this process — the same collection as get_test_results_summary().Results. A new run replaces them; close_project keeps them.

Returns #

A .NET collection of TestOutput objects: TestResult (an enum — str(r.TestResult) gives Passed, Failed, Error, Inconclusive), TestFullName, TestSuite, TestCase, TestName, Description, Message (the verdict’s explanation, including the sample of offending rows), RawMessage, ExceptionMessage, FirstDataSource, FirstQuery, SecondDataSource, SecondQuery, Expectation, NumberOfErrors, StartTime, EndTime, Duration, ThreadNumber, TestDefinition, TestDefinitionGuid.

Raises #

The sign-in and plan check (first call of the process) raises the CatPortalError family listed on the Introduction page. Before any run in this process: AttributeError: 'NoneType' object has no attribute 'Results'.

Examples #

cat.invoke_project("D:/Testing/DwhTests.cat.yaml")
for r in cat.get_test_results():
    if str(r.TestResult) != "Passed":
        print(r.TestResult, r.TestFullName)
        print("   ", r.Message)

# the failed tests as a DataFrame
import pandas as pd
failed = [{"test": r.TestFullName, "message": r.Message}
          for r in cat.get_test_results() if str(r.TestResult) == "Failed"]
df = pd.DataFrame(failed)

Related #

• Introduction — session model, logging, .NET objects, exceptions, sign-in.
• get_test_results_summary — the counts.
• Outputs — the same results as files or tables, written by the run itself.