https://docs.justcat.it/how-to-guides/data-platforms/ Recent content in Data platforms on Hugo -- gohugo.io Fri, 21 Aug 2026 10:00:00 +0200 https://docs.justcat.it/how-to-guides/data-platforms/test-databricks/ Tue, 21 Nov 2023 06:59:23 +0100 https://docs.justcat.it/how-to-guides/data-platforms/test-databricks/ What you connect to # Any compute that can run SQL over your tables, through the same driver and the same connection string — only two values differ: Compute Where to copy Server hostname and HTTP path SQL warehouse — classic or serverless; the right choice for tests SQL → SQL Warehouses → the warehouse → Connection details tab All-purpose cluster Compute → the cluster → Configuration → Advanced options → JDBC/ODBC tab A warehouse that is stopped starts when CAT connects (and your run waits for it); a cluster must be running. https://docs.justcat.it/how-to-guides/data-platforms/cat-in-databricks-notebooks/ Mon, 15 Jan 2024 10:00:00 +0100 https://docs.justcat.it/how-to-guides/data-platforms/cat-in-databricks-notebooks/ Testing Databricks data — from where? # CAT tests data in Databricks Delta tables through the Odbc@1 provider and the Simba driver; it can even generate metadata-driven tests from Unity Catalog. The setup is in Test Databricks. 😺 We recommend running the tests outside the Databricks notebooks — from a development machine, a build agent or any other machine in your architecture where it makes sense. The reason is what CAT is for: checking that the data is correct end to end, which means cross-system comparisons — sources against the lakehouse, the lakehouse against Power BI, incremental loads against their sources. https://docs.justcat.it/how-to-guides/data-platforms/test-microsoft-fabric/ Mon, 15 Jan 2024 10:00:00 +0100 https://docs.justcat.it/how-to-guides/data-platforms/test-microsoft-fabric/ What you can test # Item How CAT reaches it Provider Warehouse its SQL connection string (a TDS endpoint, port 1433) SqlServer@2 Lakehouse its SQL analytics endpoint — same kind of connection string, read-only SqlServer@2 Semantic model the workspace’s XMLA endpoint Dax@2 — exactly as a Power BI workspace model, see Test a semantic model in a workspace Authentication is Microsoft Entra ID only — your account interactively, a service principal unattended. https://docs.justcat.it/how-to-guides/data-platforms/test-dataverse/ Fri, 21 Aug 2026 10:00:00 +0200 https://docs.justcat.it/how-to-guides/data-platforms/test-dataverse/ Prerequisites # The environment’s TDS endpoint is enabled — Power Platform admin center → Environments → the environment → Settings → Product → Features → Enable TDS endpoint. It is on by default; an administrator may have turned it off. With user-level control on, the account also needs the Allow user to access TDS endpoint privilege. Microsoft Entra ID authentication only — your own account (MFA prompts included) interactively, a service principal with access to the environment for unattended runs. https://docs.justcat.it/how-to-guides/data-platforms/test-csv-and-excel-files/ Fri, 21 Aug 2026 10:00:00 +0200 https://docs.justcat.it/how-to-guides/data-platforms/test-csv-and-excel-files/ What you can test # File Provider What happens CSV (and any delimited text file) Csv@2 every file becomes a table of an in-memory DuckDB database for the run; full SQL Excel .xlsx Excel@2 every worksheet becomes a table, named "<data source>"."<sheet>"; full SQL YAML Yaml@1 a list in a YAML file is a table — the place for a small table of expected values No driver (the Visual C++ Redistributable on Windows). https://docs.justcat.it/how-to-guides/data-platforms/query-csv-and-excel-with-duckdb-sql/ Thu, 17 Aug 2023 11:40:56 +0100 https://docs.justcat.it/how-to-guides/data-platforms/query-csv-and-excel-with-duckdb-sql/ Csv@2 and Excel@2 do not query your files directly: they load them into an in-memory DuckDB database and run the queries there. Every worksheet, and every CSV file, becomes a table named "<data source name>"."<sheet or file name>" — Aero.DIM_PLANES for a sheet DIM_PLANES of a data source Aero. The rest is DuckDB’s SQL dialect, and almost everything you are used to from a database works. The examples below cover what tests need most; the complete reference is the DuckDB SQL documentation.