Get Help

run

Runs the tests of a project file — all of them, or a subset chosen by name or tags — prints the results and writes the project's outputs.

Synopsis 

catcli run [-p <path>] [-f <pattern>] [-i <tags>] [-e <tags>] [-n] [-g] [-l <level>] [-q] [--exitCode <value>]

Options

Short Long Value Meaning
-p --project path Project file, or a directory with exactly one *.cat.yaml. Default: the current directory.
-f --filter pattern Run only tests whose full name matches the pattern. * any text, ? one character, not case-sensitive, the square brackets of the full name are ignored.
-i --includeTags tags Run only tests that carry at least one of the tags. Several tags separated by , or ;. Not case-sensitive.
-e --excludeTags tags Run only tests that carry none of the tags. Same format as -i.
-n --noOutputs Skip every output defined in the project (files, database tables, …); only the console output remains.
-g --progress Show progress bars while the project loads. Honoured only when logging is None.
-l --loggingLevel level None (default), Error, Information, Debug, … — see Logging.
-q --quiet Hide the “working offline” notice.
--exitCode failures | run What the exit code reports. failures (default): 10 unless every selected test passed. run: 0 whenever the run finished, whatever the test results. Not case-sensitive.

-f, -i and -e combine: a test runs only if it satisfies every condition given.

What it does

Opens the project (data sources, test lists, named sets — everything the project file points to), runs the sign-in and plan check, selects the tests that pass the filters, runs them with the project’s thread settings, and finally writes the project’s outputs unless -n is given. Tests with an Inconclusive result are counted but are not failures — they were not run because the plan limits the number of tests per run (see Results).

The run reads the project file fresh each time; there is no cache between runs.

Output

One line per test as it finishes — a coloured dot, the duration, the full name and the result:

• 00:00:00.0076693 - [SourceDataTests].[0].[Check if the table DIM.AIRLINES is not empty.] finished with result Passed
• 00:00:00.0018578 - [SourceDataTests].[2].[Check if the table DIM.DESTINATIONS is not empty.] finished with result Failed

then a breakdown bar and the totals:

😺 passed: 41     😿 failed: 1     🙀 error: 0     🐱 inconclusive: 0

Total:  42  tests    Pass Rate:  97.6 %    Duration:  00:00:03

Details of failed tests — the message, the sample of offending rows, the queries — are in the project’s outputs (Excel, JSON, database, …); on the console they appear only with -l Error or more, which prints them as the test finishes. To step through failed tests interactively, use open.

Exit code

0 when every selected test passed. 10 when any selected test is Failed, Error or Inconclusive, or when no test was selected at all — a filter that matched nothing is not a pass. --exitCode run restores 0 whenever the run finished, whatever the test results — the behaviour of every CLI release before 3.0. 1 when the project could not be opened or the run aborted; 2–7 from the sign-in and plan check — see Exit codes.

Examples 

# the project is in the current directory
catcli run

# a specific project file
catcli run -p "D:\Testing\DwhTests.cat.yaml"

# only tests whose full name contains "gate"
catcli run -f "*gate*"

# only tests tagged Departures or Passengers, but not the ones tagged ManualOnly
catcli run -i "Departures, Passengers" -e ManualOnly

# ad-hoc run: no Excel/JSON/database outputs, log what CAT is doing
catcli run -n -l Information

# scheduled run (Enterprise): keep the manual-only tests out
catcli run -p "D:\Testing\DwhTests.cat.yaml" -e ManualOnly

# keep the pre-3.0 behaviour: exit 0 whatever the results
catcli run --exitCode run

Up to 2.x, run exited 0 whatever the results; --exitCode run restores that behaviour for a pipeline or scheduler that relies on it.

← Installation
show →