Integrations
Running CAT from a pipeline or a scheduler is one command. This page lists what that command needs around it — plan, license key, tool, secrets, results, exit codes — and points to the platform guides.
Plan it first
Before the first pipeline step, settle three questions — and draw the answer; it is the picture that makes the allow-list request, the upgrade procedure and the “why can’t it see the warehouse” conversation short.
| Question | The options | What the answer decides |
|---|---|---|
| Where does it run? | a hosted agent · a self-hosted agent or server · a scheduler host | Its network position — CAT must reach every data source the tests compare, and the best tests compare systems (source vs warehouse, warehouse vs Power BI); a place inside one of them sees half the picture. Hosted agents are fresh every run: install and key in the pipeline. Self-hosted: install and key once, under the agent’s account. |
| How is it managed? | installed by the pipeline, pinned in a variable · pre-installed and upgraded with the server | Who owns the version and the key: where CAT_VERSION and CAT_LICENSE_KEY live, who bumps the version after testing it, who rotates the key. From CAT 3.0 the key is a hard requirement for unattended runs — decide where it is set before the upgrade. |
| What must it reach? | package source at install time · data sources at run time · nothing else | The allow-list (table under Tool below) and the firewall rules for the data sources. |
| Where do results go? | a report file the platform renders · a database table · Excel · JSON | How the run becomes a verdict — the publish step or a summary check, never CAT’s exit code — and where a person looks when something is red. |
Draw it as above with your names in the boxes — the agent pool, the warehouse, the workspace — and the rest of this page fills in the facts.
Plan
Automated use — a run started by a CI/CD platform or a scheduler — is an Enterprise plan feature. The Starter, Professional and Team plans are interactive-only: a run they would otherwise allow is refused when CAT sees it was launched by automation, with The 'Team' plan allows interactive use only; CAT was launched by 'GitHub Actions'. Run CAT interactively, or upgrade your plan. (CAT CLI exit code 6; PowerShell error CatPortal.InteractiveUsageRequired; Python InteractiveUsageError). See Compare plans.
Enforced from CAT 3.0: a run started by a CI/CD platform or a scheduler without a valid Enterprise license key stops before any test runs. Earlier versions let such runs through; when a pipeline moves to 3.0, add the license-key step before bumping the version — which is one more reason to pin the version and keep it in a variable.
CAT recognizes automation two ways, and refuses only on a confident match — nothing detected means the run is allowed:
| Detected by | Launchers |
|---|---|
| Environment variable of the platform | GitHub Actions · GitLab CI · Azure Pipelines · Jenkins · TeamCity · CircleCI · Travis CI · Bitbucket Pipelines · AppVeyor · AWS CodeBuild · Google Cloud Build · a Kubernetes pod |
| Parent process | cron · SQL Server Agent · Windows Task Scheduler · Control-M · Azure Data Factory self-hosted integration runtime |
Sign-in: the license key
A pipeline has no browser and nobody to type a password. On the Enterprise plan CAT signs in with nothing: the license key stored on the machine unlocks every tool, also without Internet access. Set it once on a self-hosted agent, or once per run on a hosted agent from a secret:
| Tool | Set the key |
|---|---|
| CAT CLI | catcli instance --setLicenseKey "$CAT_LICENSE_KEY" — see instance |
| PowerShell module | Set-CatInstance -LicenseKey $env:CAT_LICENSE_KEY — see Set-CatInstance |
| Python module | cat.set_instance(os.environ["CAT_LICENSE_KEY"]) — see set_instance |
The key is shared by all CAT tools on the machine. See Get CAT license and Apply a license key.
Tool
| Tool | Runs where | Install on the agent |
|---|---|---|
| CAT CLI | any Windows runner; no PowerShell needed | winget install DataTools.CATCLI, or unpack the release ZIP |
| PowerShell module | wherever PowerShell 7 runs — Windows and Linux agents, containers | Install-Module CAT -AcceptLicense -Force |
| Python module | Windows runners; notebooks — Databricks, Microsoft Fabric | pip install justcatit |
One command runs the project: catcli run, Invoke-CatProject, invoke_project(). The project file, the test definitions and everything they reference are read from the checkout; relative paths resolve against the project file.
Pre-installed on the agent, or installed by the pipeline
The platform guides install CAT in a pipeline step, because that works on any agent — hosted ones included. The other normal setup, on self-hosted agents, is to treat CAT as a component of the machine: install it once (pinned), set the license key once under the account the agent runs as, and manage updates with the same process as the rest of the server’s software. The pipeline step is then just the run command, and the agent needs no access to the package sources at all.
For networks that allow-list outbound addresses, this is what each route reaches:
| When | Reaches |
|---|---|
| Installing in the pipeline | the package source of the tool — the PowerShell Gallery (www.powershellgallery.com), WinGet’s source, PyPI (pypi.org, files.pythonhosted.org) — or the Releases page of this site (docs.justcat.it) for the CAT CLI installer |
| Running tests | your data sources — nothing else. With the license key in place CAT signs in to nothing and needs no other address to run |
Pin the version, keep version and key in variables
A pipeline that installs “the latest CAT” changes behaviour the day a new version ships — with 3.0, a pipeline that has no license-key step stops working the day it picks the new version up. Pin the version, and keep the two values every CAT step needs — the version and the license key — in pipeline variables (a variable group, CI/CD variables, repository secrets, Jenkins credentials), not in the script: upgrading CAT is then one variable change in one place, and the key is a secret the platform masks.
| Tool | Pinned install |
|---|---|
| CAT CLI | winget install DataTools.CATCLI --version $env:CAT_VERSION, or the release ZIP of that version |
| PowerShell module | Install-Module CAT -RequiredVersion $env:CAT_VERSION -AcceptLicense -Force |
| Python module | pip install justcatit==$CAT_VERSION |
followed by the key from the table above (Set-CatInstance -LicenseKey $env:CAT_LICENSE_KEY, …). The platform guides below use CAT_VERSION and CAT_LICENSE_KEY this way.
Secrets
Connection strings, passwords and tokens never go into the project file. Write %NAME% where the value belongs — Connection string: "%DWH_CONNECTION_STRING%" — and CAT reads the environment variable NAME when it opens the project. Every platform can inject a secret as an environment variable of the step, so the project file runs unchanged on a laptop and in the pipeline. See Environment variables and Work with passwords.
Results and exit codes
Platforms render test results from a report file: ask for one with the Output setting — JUnit is read by Azure DevOps, GitLab, GitHub Actions and Jenkins; TRX by Azure DevOps and Visual Studio. Files are written after the run; a database output writes each test as it finishes.
No CAT tool fails the step on a failed test. catcli run exits 0 when the run completed, whatever the results; Invoke-CatProject throws nothing and sets no exit code; invoke_project() returns the summary. A pipeline that should go red on a failed test does one of two things: let the publish step fail on test failures (Azure DevOps failTaskOnFailedTests, Jenkins’ JUnit plugin, the GitHub publish action), or read the summary itself — Get-CatTestResultSummary / the returned summary in Python / the JSON output’s failedCount and errorCount — and exit non-zero. A non-zero exit code from CAT means the run did not complete: the project could not be opened, or the sign-in and plan check refused it — see the tool’s exit codes.
Platform guides
Step by step, with screenshots:
- CAT in Azure DevOps Pipelines · Azure DevOps Classic Releases
- CAT in GitLab
- CAT in GitHub Actions
- CAT in Jenkins
- SQL Server Agent · CAT in Databricks notebooks · CAT and Microsoft Fabric
Every other platform that can run a shell command works the same way; the guides are starting points, not finished pipelines.