Docs

Connecting Databricks

How you connect a Databricks workspace to Emolysis: sign-in, OAuth, workspace and compute, the catalog mirror, sampling and what a deploy makes.

Planned reference

Updated Oct 23, 2026

Docs menu · 25 pagesConnecting Databricks

Status

In private development. Early access by invitation. The names, options and formats on this page can change before the first release.

On this page

Databricks is the first platform Emolysis targets. This page walks through connecting a workspace and what happens after you do. The Databricks platform page has the summary.

Before you start

You need:

  • A Databricks workspace with Unity Catalog.
  • An account that can read the catalogs and schemas you want to work with, use a SQL warehouse, create jobs, write files in its own workspace folder, and read and write the pipeline's tables. In Databricks terms, that is USE CATALOG, USE SCHEMA and SELECT on what you read, CREATE TABLE and MODIFY where the pipeline writes, CAN USE on the SQL warehouse, and permission to create jobs in the workspace you deploy to, which workspace users have unless an admin has restricted it.
  • For a first try, a development workspace or a dedicated catalog and schema, so that nothing you test touches production.

You also need a way to reach a model: your own model key, on any plan, or a paid plan's model usage. See Model keys and credits.

The flow

  1. Choose Connect, then Databricks, in the app, or run emo connect databricks, or use the connect skill in your editor.
  2. Sign in to Emolysis. If you are not signed in yet, you see a short code. Confirm it in the browser, where you sign in with your GitHub, Microsoft or Google account. The code expires after 15 minutes.
  3. Authorize with Databricks. You sign in with Databricks' own OAuth flow. A personal access token is accepted for automation, such as CI and scripts. The credentials are kept in your machine's credential store and are never sent to Emolysis.
  4. Pick a workspace and compute. Emolysis lists the workspaces your account can reach. Pick one, then the SQL warehouse or cluster that samples and deploys should use.
  5. Ready. The Databricks profile loads and your catalog appears in the app. If you use an editor or coding agent that supports MCP, add one configuration entry for emo mcp; setup buttons on the web dashboard are planned.
Shell
emo connect databricks

What happens after you connect

  • Catalog mirror. Emolysis reads the catalogs, schemas, tables and columns your account can see into a local mirror, so the agent can find sources without querying the platform for each request. Metadata only; no table data is copied.
  • Sampling, on request. When you ask for a sample, Emolysis reads a small slice of a table through your SQL warehouse into DuckDB. That query runs on your compute.
  • Templates. Only Databricks templates are offered in this project, and the Databricks platform profile governs how code is written.
  • The project file. The project's emolysis.yml names databricks as its platform. See Projects and emolysis.yml.

What leaves your machine

Your Databricks credentials stay in your machine's credential store. Table and column names, schemas and sample values can appear in the prompts the agent sends to the model provider you choose, because the agent needs them to write code. With a paid plan's model usage, those prompts pass through Emolysis's metering service on the way to the model provider; that service is designed not to store or log their content. The catalog mirror itself stays on your machine. See Data handling.

What a deploy produces

A green run deploys as a Databricks Asset Bundle with its jobs or pipelines, run on your workspace's compute under your account. The Databricks CLI is not needed: with it installed, Emolysis deploys the bundle; without it, Emolysis creates the same job through the Databricks API, with the pipeline's SQL files in your workspace folder. See Deploying and the Asset Bundle entry in the glossary.

Permissions and traps the profile knows

Unity Catalog permissions decide what you, and your deployed jobs, can read and write. A job runs as an identity, and that identity needs its own grants; passing a test on your machine does not prove the job can read the table. Code can behave differently on serverless and classic compute. The Databricks lint rules flag both before anything runs.

Disconnecting

Removing the connection deletes the stored credentials from your machine.

Not on Databricks?

Snowflake and Microsoft Fabric are planned after Databricks, PostgreSQL is in development as a source, and SQL Server is planned as a source.