run
Runs the tests of a project file — all of them, or a subset chosen by name or tags — prints the results and writes the project's outputs.
Synopsis
catcli run [-p <path>] [-f <text>] [-i <tags>] [-e <tags>] [-n] [-g] [-l <level>] [-q]
Options
| Short | Long | Value | Meaning |
|---|---|---|---|
-p |
--project |
path | Project file, or a directory with exactly one *.cat.yaml. Default: the current directory. |
-f |
--filter |
text | Run only tests whose full name contains the text. Substring match, not case-sensitive, no wildcards. |
-i |
--includeTags |
tags | Run only tests that carry at least one of the tags. Several tags separated by , or ;. Not case-sensitive. |
-e |
--excludeTags |
tags | Run only tests that carry none of the tags. Same format as -i. |
-n |
--noOutputs |
Skip every output defined in the project (files, database tables, …); only the console output remains. | |
-g |
--progress |
Show progress bars while the project loads. Honoured only when logging is None. |
|
-l |
--loggingLevel |
level | None (default), Error, Information, Debug, … — see Logging. |
-q |
--quiet |
Hide the “working offline” notice. |
-f, -i and -e combine: a test runs only if it satisfies every condition given.
What it does
Opens the project (data sources, test lists, named sets — everything the project file points to), runs the sign-in and plan check, selects the tests that pass the filters, runs them with the project’s thread settings, and finally writes the project’s outputs unless -n is given. Tests with an Inconclusive result are counted but are not failures — they were not run because the plan limits the number of tests per run (see Results).
The run reads the project file fresh each time; there is no cache between runs.
Output
One line per test as it finishes — a coloured dot, the duration, the full name and the result:
• 00:00:00.0076693 - [SourceDataTests].[0].[Check if the table DIM.AIRLINES is not empty.] finished with result Passed
• 00:00:00.0018578 - [SourceDataTests].[2].[Check if the table DIM.DESTINATIONS is not empty.] finished with result Failed
then a breakdown bar and the totals:
😺 passed: 41 😿 failed: 1 🙀 error: 0 🐱 inconclusive: 0
Total: 42 tests Pass Rate: 97.6 % Duration: 00:00:03
Details of failed tests — the message, the sample of offending rows, the queries — are in the project’s outputs (Excel, JSON, database, …); on the console they appear only with -l Error or more, which prints them as the test finishes. To step through failed tests interactively, use open.
Exit code
0 when the run finished, whatever the test results — a run with failed tests also exits 0. A pipeline that must fail on failed tests has to read an output (for example the JUnit or JSON output) or the console text. 1 when the project could not be opened or the run aborted; 2–7 from the sign-in and plan check — see Exit codes.
Examples
# the project is in the current directory
catcli run
# a specific project file
catcli run -p "D:\Testing\DwhTests.cat.yaml"
# only tests whose full name contains "gate"
catcli run -f gate
# only tests tagged Departures or Passengers, but not the ones tagged ManualOnly
catcli run -i "Departures, Passengers" -e ManualOnly
# ad-hoc run: no Excel/JSON/database outputs, log what CAT is doing
catcli run -n -l Information
# scheduled run (Enterprise): keep the manual-only tests out
catcli run -p "D:\Testing\DwhTests.cat.yaml" -e ManualOnly
Related
- Introduction — conventions, sign-in, exit codes.
show— see what would run before running it.- Tests — properties —
Tags, full name, results. - Outputs — where the results go.
- Run your tests — the same task across all tools.