--- title: "Run your tests" description: "Run everything in a project file, or only the part you care about right now — with any CAT tool" url: "https://docs.justcat.it/how-to-guides/organize-and-run-tests/run-your-tests/" --- # Run your tests Every CAT tool runs the same project file. CAT Studio while you write tests, the PowerShell module or CAT CLI when you script or schedule them, the Python module when they are part of a notebook or a Python pipeline — the project file does not care. ## Run everything **CAT Studio** Open the project (**Project** page → *Open existing project*, or a recent one), go to **Tests** and press **F9** — *Run all tests*. Results fill the grid as tests finish; click a test to read its message. **CAT CLI** ```text cd D:\AutomatedTests\Flights catcli run ``` With no path, `catcli run` takes the one `.cat.yaml` file in the current directory. Otherwise name it: `catcli run --project D:\AutomatedTests\Flights\Flights.cat.yaml` (a directory works too, when it holds exactly one project file). See [`run`](https://docs.justcat.it/reference/cat-cli/run/ "catcli run"). **PowerShell module** ```powershell cd D:\AutomatedTests\Flights Invoke-CatProject ``` With no path, `Invoke-CatProject` takes the one `.cat.yaml` file in the current location. Otherwise name it: `Invoke-CatProject -Path D:\AutomatedTests\Flights\Flights.cat.yaml` (a directory works too, when it holds exactly one project file). See [`Invoke-CatProject`](https://docs.justcat.it/reference/powershell-module/invoke-catproject/ "Invoke-CatProject"). **Python module** ```python import justcatit as cat summary = cat.invoke_project("D:/AutomatedTests/Flights/Flights.cat.yaml") ``` The path is required — a file, or a directory with exactly one project file. See [`invoke_project`](https://docs.justcat.it/reference/python-module/invoke-project/ "invoke_project"). ## Read the result A command-line run prints one line per test as it finishes and a summary at the end: ```text • 00:00:00.0755756 - [Smoke tests].[Airline dimension matches the source] finished with result Passed • 00:00:00.1389169 - [Data quality].[No booking without a passenger] finished with result Failed • 00:00:06.2492205 - [Smoke tests].[Flights table is loaded] finished with result Error 😺 passed: 1 😿 failed: 1 🙀 error: 1 🐱 inconclusive: 0 ``` Three things to read: * **A result per test** — `Passed`, `Failed`, `Error` (or `Inconclusive`); see [Results](https://docs.justcat.it/reference/tests/results/ "Results"). * **The summary** — how many of each, how long it took. * **The log**, when you turn it on. Anything can go wrong — a typo in the project file or in a query, a data source that does not exist, a missing permission — and the log is what says which. CAT CLI is quiet by default (`-l Information` to see it, `-l Error` for the failed tests' details only); the PowerShell module logs at `Information` by default (`-LoggingLevel None` to silence it). The details of a failed test — the message with the sample of offending rows, the queries — are in CAT Studio's test detail, in the PowerShell module's `Get-CatTestResult`, in the Python summary's `Results`, and in every [output](https://docs.justcat.it/reference/outputs/introduction/ "Outputs") you ask for — an Excel file, JSON, a database table, a report for your pipeline. ## Run only a part of the project Test bases grow; sooner or later running everything on every occasion stops making sense. Two mechanisms narrow a run down, and they combine — a test runs only when it satisfies every condition you give. ### By name A filter runs the tests whose [full name](https://docs.justcat.it/reference/tests/properties/#full-name "Full name") — `[Suite].[Order].[Test case].[Name]` — contains the text. Substring, not case-sensitive, no wildcards. Because the full name is built from the categorization, a filter is also how you run one suite or one test case. **CAT Studio** Filter the grid — the column headers filter and search — tick the tests you want, and press **F8**, *Run selected tests*. **CAT CLI** ```text catcli run --filter "Airline" catcli run --filter "[Smoke tests]" catcli run --filter "[Smoke tests].[Dimensions]" ``` **PowerShell module** ```powershell Invoke-CatProject -Filter "Airline" Invoke-CatProject -Filter "[Smoke tests]" Invoke-CatProject -Filter "[Smoke tests].[Dimensions]" ``` **Python module** ```python cat.invoke_project(path, filter="Airline") cat.invoke_project(path, filter="[Smoke tests]") cat.invoke_project(path, filter="[Smoke tests].[Dimensions]") ``` ### By tag Names were chosen for other reasons; tags are the deliberate way of marking a group. Add `Tags` to a test — one, or several separated by a comma or a semicolon — then include or exclude by tag. Matching is not case-sensitive; *include* runs tests that carry at least one of the listed tags, *exclude* skips tests that carry any of them. ```yaml Tests: - Name: Full reconciliation of the fact table Tags: departures, slow # … ``` Excluding is the more useful direction: tag the tests that should never run unattended — long ones, tests against production, tests someone has to look at — `ManualOnly`, and exclude that tag wherever the project is scheduled. The tests stay in the project and stay maintained; the pipeline stops tripping over them. **CAT Studio** The run button's dropdown on the **Tests** page has the include and the exclude tag lists; they combine with the ticked tests. **CAT CLI** ```text catcli run --includeTags "departures, passengers" catcli run --excludeTags ManualOnly ``` **PowerShell module** ```powershell Invoke-CatProject -IncludeTag departures, passengers Invoke-CatProject -ExcludeTag ManualOnly ``` **Python module** ```python cat.invoke_project(path, include_tags="departures, passengers") cat.invoke_project(path, exclude_tags="ManualOnly") ``` ## Related * [`catcli run`](https://docs.justcat.it/reference/cat-cli/run/ "catcli run") · [`Invoke-CatProject`](https://docs.justcat.it/reference/powershell-module/invoke-catproject/ "Invoke-CatProject") · [`invoke_project`](https://docs.justcat.it/reference/python-module/invoke-project/ "invoke_project") · [Tests in CAT Studio](https://docs.justcat.it/reference/cat-studio/tests/ "Tests in CAT Studio") — every option of each tool. * [Outputs](https://docs.justcat.it/reference/outputs/introduction/ "Outputs") — where the results go. * [Run tests in parallel](https://docs.justcat.it/how-to-guides/organize-and-run-tests/run-tests-in-parallel/ "Run tests in parallel") — when a run takes too long.