CAT 1.2.0

Log Number of Errors

CAT has again a new preview feature that originates from user feedback: Number of Errors. Let’s say you have a test with set is empty expectation. By default CAT stops the evaluation of a test immediatelly when it retrieves the very first row. You can set Maximum Errors Logged to a number up to 50 (introduced recently). That influences only how many rows you see in the error message. But how many rows / errors there are in total? You can now instruct CAT to find out:

Tests:
- Name: There are no new errors in the error log
  Data Source: DWH
  Query: |
    SELECT  *
    FROM    Audit.ErrorLog
    WHERE   ThrownOn > DATEADD(DAY, -14, SYSUTCDATETIME())
  Expectation: set is empty
  Maximum Errors Logged: 5
  Log Number of Errors: true # this setting is new (default is false)

CAT in that case retrieves the entire set and enriches the error message with the information: No row was expected, but exactly 9 rows exist.

Important: this setting can have significant negative effect on performance. Use it only when you really need it - when you really do something with the result.

In the first phase this is introduced only for set is empty expectation. The others may follow, depending on your feedback. Log Number of Errors is set per test, there is no global setting.

See full documentation.

Export of Test Results to Database

There is new Number of Errors property on the output. CAT can output this value to relational databases (SQL Server and ORACLE). If you already output test results to a database, you don’t need to do anything, unless you need the new property on the output. CAT now automatically adapts to the structure of the table or procedure/package you provide in the project file:

Output:
- Database: DWH
  Table: Test.CatTestResults

CAT now checks the structure of the target table (or parameters of a procedure). If the name matches existing output property, such as the new Number of Errors, it will use it in INSERT statement (or in the procedure call). If not, no problem.

Also, all the naming conventions CAT supports can be used, e.g., NumberOfErrors, NUMBER_OF_ERRORS, number_of_errors, Number of errors. You don’t adapt your database to CAT, CAT will adapt to your database.

If you export test results into a SQL server table and want the new property on the output, just do something like this:

-- change schema and/or table name
ALTER TABLE Test.CatTestResults ADD NumberOfErrors INT NULL;

New Fields in Database Output

Number of Errors is not the only new field in the database (but also JSON and YAML) output. There is a new RawMessage output property - it contains the message from the test, except other existing properties. (The Message property cannot be changed, it is used by Azure DevOps, GitLab and other automation systems.) You can also export Tags.

Again, alter your table or procedure to get any/all of the new properties on the output. See the full documentation to check the complete list of columns / procedure parameters you can have in the output.

Database Output for SqlServer@2

In previous versions of CAT, you had an option to output CAT test results only to SQL Server and ORACLE. But only SqlServer@1 provider was supported - and that was very confusing. Beginning with this release, CAT supports export of test results also for SqlServer@2 provider (the newer one that supports service principals, MFA etc.). PostgreSQL and MySQL are not yet supported, but that’s coming soon.

That’s it :-). Happy testing.

CAT team

← CAT 1.3.0
CAT 1.1.0 →