run
Runs the tests of a project file — all of them, or a subset chosen by name or tags — prints the results and writes the project's outputs.
Synopsis
catcli run [-p <path>] [-f <pattern>] [-i <tags>] [-e <tags>] [-n] [-g] [-l <level>] [-q] [--exitCode <value>]
Options
| Short | Long | Value | Meaning |
|---|---|---|---|
-p |
--project |
path | Project file, or a directory with exactly one *.cat.yaml. Default: the current directory. |
-f |
--filter |
pattern | Run only tests whose full name matches the pattern. * any text, ? one character, not case-sensitive, the square brackets of the full name are ignored. |
-i |
--includeTags |
tags | Run only tests that carry at least one of the tags. Several tags separated by , or ;. Not case-sensitive. |
-e |
--excludeTags |
tags | Run only tests that carry none of the tags. Same format as -i. |
-n |
--noOutputs |
Skip every output defined in the project (files, database tables, …); only the console output remains. | |
-g |
--progress |
Show progress bars while the project loads. Honoured only when logging is None. |
|
-l |
--loggingLevel |
level | None (default), Error, Information, Debug, … — see Logging. |
-q |
--quiet |
Hide the “working offline” notice. | |
--exitCode |
failures | run |
What the exit code reports. failures (default): 10 unless every selected test passed. run: 0 whenever the run finished, whatever the test results. Not case-sensitive. |
-f, -i and -e combine: a test runs only if it satisfies every condition given.
What it does
Opens the project (data sources, test lists, named sets — everything the project file points to), runs the sign-in and plan check, selects the tests that pass the filters, runs them with the project’s thread settings, and finally writes the project’s outputs unless -n is given. Tests with an Inconclusive result are counted but are not failures: on the Starter plan, the daily limit of checks was reached before they ran (see Results).
The run reads the project file fresh each time; there is no cache between runs.
Output
One line per test as it finishes — a coloured dot, the duration, the full name and the result:
• 00:00:00.0076693 - [SourceDataTests].[0].[Check if the table DIM.AIRLINES is not empty.] finished with result Passed
• 00:00:00.0018578 - [SourceDataTests].[2].[Check if the table DIM.DESTINATIONS is not empty.] finished with result Failed
then a breakdown bar and the totals:
😺 passed: 41 😿 failed: 1 🙀 error: 0 🐱 inconclusive: 0
Total: 42 tests Pass Rate: 97.6 % Duration: 00:00:03
On the Starter plan, a line with the checks used and left today follows the totals:
Checks today: 380 of 500 used, 120 left. The limit resets at midnight (in 5 h 12 min).
When CAT Portal cannot be reached, it starts Checks today on this machine: and says (CAT Portal could not be reached). When the run needed more checks than were left, one more line says how many tests were not evaluated, when the limit resets and where to upgrade; those tests are Inconclusive. See Checks per day (Starter plan).
Details of failed tests — the message, the sample of offending rows, the queries — are in the project’s outputs (Excel, JSON, database, …); on the console they appear only with -l Error or more, which prints them as the test finishes. To step through failed tests interactively, use open.
Exit code
0 when every selected test passed. 10 when any selected test is Failed, Error or Inconclusive, or when no test was selected at all — a filter that matched nothing is not a pass. --exitCode run restores 0 whenever the run finished, whatever the test results — the behaviour of every CLI release before 3.0. 1 when the project could not be opened or the run aborted; 2–8 from the sign-in and plan check — see Exit codes. On the Starter plan, 7 also when the project has outputs, when no checks are left today, or when CAT can reach neither CAT Portal nor its count of checks on this computer; no test runs, and the message is on standard error.
Examples
# the project is in the current directory
catcli run
# a specific project file
catcli run -p "D:\Testing\DwhTests.cat.yaml"
# only tests whose full name contains "gate"
catcli run -f "*gate*"
# only tests tagged Departures or Passengers, but not the ones tagged ManualOnly
catcli run -i "Departures, Passengers" -e ManualOnly
# ad-hoc run: no Excel/JSON/database outputs, log what CAT is doing
catcli run -n -l Information
# scheduled run (Enterprise): keep the manual-only tests out
catcli run -p "D:\Testing\DwhTests.cat.yaml" -e ManualOnly
# keep the pre-3.0 behaviour: exit 0 whatever the results
catcli run --exitCode run
Up to 2.x, run exited 0 whatever the results; --exitCode run restores that behaviour for a pipeline or scheduler that relies on it.
Related
- Introduction — conventions, sign-in, exit codes.
show— see what would run before running it.- Tests — properties —
Tags, full name, results. - Outputs — where the results go.
- Run your tests — the same task across all tools.