Introduction

Introduction to very basic concepts in CAT. If you are new to CAT, read carefully.

What is CAT about #

Data is prone to problems. It has issues. Even if it doesn’t, it is easy to screw it up. Data is often “travelling” through the company systems - e.g., records are created in a CRM system, loaded into DWH and from DWH to Power BI workspace for reporting. Data teams release new versions of these solutions. There is a big potential for things to go wrong.

It is a best practice to have automated tests that give you enough confidence that all is working and all the data is trustworthy. With CAT you achieve this easily and quickly and will have your data under control.

If you end up here, we expect you have some data and you want to trust it.

This is technical reference documentation — for the value behind CAT and its use cases, read introduction to CAT.

Basic terms #

In CAT you need to define:

Data sources

Where is the data you want to test?

Queries

Questions asked against your data sources — written directly in a test, or defined on their own under a name

Tests

Simple queries against data and your expectations about the results

Outputs

Optionally you can define the format for results of your tests (MS Excel, export to database and many others)

These definitions live in a project file — directly, or in other places the project file points to. Each linked section documents one definition’s properties, wherever the definition is stored.

Data sources #

Every company has lots of data. Very often, the data is in various formats:

A Data Source in CAT is a definition of where the data is and how to connect to it. You give the data source a friendly name (such as CRM, DWH, Power BI, …). This friendly name is then used in your tests, where you define your expectation about the data in the system. This way you don’t have to repeat all connection information in every test - you just specify that your test examines data in e.g., DWH.

For details, see Data sources.

Tests #

A TestDefinition describes your expectations about the data. E.g., you expect that a SQL statement issued against your CRM system (MySQL) returns exactly the same data as another SQL statement issued against your DWH (SQL Server). Or that none of your tables in a Power BI model is empty.

Example of a simple smoke test:

• Properties
• YAML

Tests:
- Test Suite:   Smoke tests
  Test Name:    Customer dimension table has active customers
  Data Source:  DWH
  Query: |
                SELECT *
                FROM Dimension.Customer
                WHERE IsActive = 1
  Expectation:  set is not empty

You can define your tests wherever you want - in YAML (as in the example), in MS Excel, in a relational database table. You can also very easily author them in the GUI - CAT Studio.

For details, see Tests.

Outputs #

Outputs are results of your tests in a format you need. You may or may not need outputs.

By default, every CAT tool shows test results in its own way:

For that, you don’t need to set anything - that always works. But there may be various situations, where you need test results also in other formats:

• Developers or business users often need results of tests in MS Excel format
• You want to easily build reports about how the data is doing, for your stakeholders - so you need test results in a relational database table
• Azure DevOps or other automation tools need special file formats in order to display nice charts and details of test results
• You may be tasked to automate further processes and you need test results in JSON
• and others.

And you may even need more outputs, not only one.

In CAT you can define these outputs in seconds - it really takes a few clicks in CAT Studio or a few lines in YAML.

For details, see Outputs.