CAT 0.14.0

This release is entirely about metadata driven tests.

Some time ago, we were relying on generating tests into a relational database table or other place. Then, we read the test defintitions from there. This required a boring and cumbersome preprocessing step.

CAT allows you to read tests definitions from any provider. You can also use e.g., a stored procedure or plain SQL, if it returns necessary columns:

That does NOT require a preprocessing step, but it is cumbersome to prepare, hardly ledgible and simply uggly. There must be a better way…

Let’s say the example shown above compares data from two data sources:

Data Sources:
- Provider: SqlServer@1
  Connection string: '%BIG_BANG_PROD%' # environment variable used for connection string
  Name: BigBangPROD
- Provider: SqlServer@1
  Connection string: '%BIG_BANG_UAT%' # environment variable used for connection string
  Name: BigBangUAT

Let’s just define a named set - set of data with some logical name. We can then use it throughout our project file:

Named sets:
- Name: fact and dimension tables
  Data source: BigBangPROD
  Query: |
    SELECT  *
    FROM INFORMATION_SCHEMA.TABLES AS t
    WHERE t.TABLE_SCHEMA in ('Fact', 'Dim')

And define the test. Number of rows in fact and dimensions tables set defined above determines how many tests will be generated. Use %ColumnName% where needed:

Tests:
- Metadata: fact and dimension tables
  TestSuite: INT Compare Count of Dim, Fact at UATxPROD
  TestCase: Compare count of rows from two Databases
  TestName: '[%TABLE_SCHEMA%].[%TABLE_NAME%]'
  Description: This test checks that count of rows in the tables is the same
  FirstDataSource: BigBangUAT
  FirstQuery: |
    SELECT COUNT (*) FROM [%TABLE_SCHEMA%].[%TABLE_NAME%] WHERE [CompanyID] = '1'
  SecondDataSource: BigBangPROD
  SecondQuery: |
    SELECT COUNT (*) FROM [%TABLE_SCHEMA%].[%TABLE_NAME%]
  Expectation: sets match

And that’s it :) Compare the ledgibility with the SQL version. And of course, you can use metadata returned by any provider (CSV, MS Excel, …).