--- title: "Databricks@1" description: "Databricks@1 — Databricks warehouses and clusters through the Apache Arrow ADBC driver, no driver install" url: "https://docs.justcat.it/reference/data-sources/providers/databricks-1/" --- # Databricks@1 ## Connects to Databricks on Azure, AWS and Google Cloud: SQL warehouses (classic and serverless — the right choice for tests) and all-purpose clusters. Copy the two values a connection needs from the workspace: | Where | What you get | |---|---| | SQL warehouse → **Connection details** tab | Server hostname and HTTP path | | Cluster → **Advanced options** → **JDBC/ODBC** tab | Server hostname and HTTP path | ## Example **Properties** Name : MyDatabricksData Provider : Databricks@1 Technology : Databricks Connection string : ``` adbc.spark.host=%DATABRICKS_HOST%; adbc.spark.path=%DATABRICKS_HTTP_PATH%; adbc.spark.auth_type=token; adbc.spark.token=%DATABRICKS_TOKEN%; ``` **YAML** ```yaml Data sources: - Name: MyDatabricksData Provider: Databricks@1 Technology: Databricks Connection string: > adbc.spark.host=%DATABRICKS_HOST%; adbc.spark.path=%DATABRICKS_HTTP_PATH%; adbc.spark.auth_type=token; adbc.spark.token=%DATABRICKS_TOKEN%; ``` ## Connection string The keys are the driver's — `Key=Value` pairs separated by semicolons, handed to the Apache Arrow ADBC driver's connection object exactly as written. The keys you will use most: | Key | Meaning | |---|---| | `adbc.spark.host` | Server hostname from the workspace's connection details, e.g. `adb-1234567890123456.7.azuredatabricks.net` or `dbc-a1b2c3d4-e5f6.cloud.databricks.com` — no `https://` | | `adbc.spark.path` | HTTP path of the SQL warehouse (`/sql/1.0/warehouses/`) or cluster (`sql/protocolv1/o//`) | | `adbc.spark.auth_type` | `token` for a personal access token, `oauth` for a service principal | | `adbc.spark.token` | The personal access token (with `adbc.spark.auth_type=token`) | | `adbc.databricks.oauth.grant_type`, `adbc.databricks.oauth.client_id`, `adbc.databricks.oauth.client_secret` | Service principal credentials (with `adbc.spark.auth_type=oauth`) | | `adbc.connection.catalog`, `adbc.connection.db_schema` | Optional defaults for unqualified names | | `adbc.spark.port` | Optional, `443` | A service principal authenticates with `grant_type=client_credentials` and its client id and secret instead of a token: **Properties** Name : MyDatabricksData Provider : Databricks@1 Technology : Databricks Connection string : ``` adbc.spark.host=%DATABRICKS_HOST%; adbc.spark.path=%DATABRICKS_HTTP_PATH%; adbc.spark.auth_type=oauth; adbc.databricks.oauth.grant_type=client_credentials; adbc.databricks.oauth.client_id=%DATABRICKS_CLIENT_ID%; adbc.databricks.oauth.client_secret=%DATABRICKS_CLIENT_SECRET%; ``` **YAML** ```yaml Data sources: - Name: MyDatabricksData Provider: Databricks@1 Technology: Databricks Connection string: > adbc.spark.host=%DATABRICKS_HOST%; adbc.spark.path=%DATABRICKS_HTTP_PATH%; adbc.spark.auth_type=oauth; adbc.databricks.oauth.grant_type=client_credentials; adbc.databricks.oauth.client_id=%DATABRICKS_CLIENT_ID%; adbc.databricks.oauth.client_secret=%DATABRICKS_CLIENT_SECRET%; ``` Any other parameter the driver documents passes through unchanged — for example `adbc.databricks.cloudfetch.enabled=false`, or a server-side `SET` through `adbc.databricks.ssp_=…`. CAT reads no key from the string, adds none and validates nothing; a missing or misspelt key fails with the driver's own error message, at the data source. The full parameter reference is in the driver's own documentation: the inherited Spark keys in the Spark driver readme, and the Databricks-specific keys in the readme on the NuGet package page. A token, a secret, or the whole string, can come from an environment variable — `%NAME%` is replaced with the value of `NAME`; see [How every provider is used](https://docs.justcat.it/reference/data-sources/providers/introduction/#how-every-provider-is-used "How every provider is used"). ## Settings None beyond `Connection string`. ## Query One SQL statement in Databricks SQL that returns a result set. Use three-part names (`catalog.schema.table`), or set `adbc.connection.catalog` and `adbc.connection.db_schema` in the connection string to default unqualified names. ## Prerequisites None. The driver ships with CAT — Windows, Linux and macOS alike, nothing to install. A warehouse that is stopped starts when CAT connects, and the first test on it waits rather than fails; a cluster must already be running. ## Types | Databricks type | Arrives as | |---|---| | DECIMAL | a number; a value beyond 28 digits arrives as text | | TIMESTAMP, TIMESTAMP_NTZ | date and time, in the session time zone | | DATE | a date | | STRUCT | JSON text | | ARRAY, MAP, INTERVAL | text | ## Serving test and data source definitions `Databricks@1` can also serve [lists](https://docs.justcat.it/reference/project-file/lists/ "Lists") of definitions: any statement that returns the columns of a [test](https://docs.justcat.it/reference/tests/properties/ "Test properties"), [data source](https://docs.justcat.it/reference/data-sources/properties/ "Data source properties") or [query](https://docs.justcat.it/reference/queries/properties/ "Query properties") definition works. Ready-made table scripts are in [Store definitions in a database](https://docs.justcat.it/how-to-guides/organize-and-run-tests/store-definitions-in-a-database/ "Store definitions in a database"). ## Related * [Technologies](https://docs.justcat.it/reference/data-sources/technologies/ "Technologies") — what CAT can test, with the provider behind each. * [Test Databricks](https://docs.justcat.it/how-to-guides/data-platforms/test-databricks/ "Test Databricks") — a full walk-through, with the guided form in CAT Studio. * [Odbc@1](https://docs.justcat.it/reference/data-sources/providers/odbc-1/ "Odbc@1") — through the ODBC driver instead, when your organisation standardises on it.