Skip to content
ClickRuntime

Automate Databricks

Get a morning list of failed job runs, catch jobs that are taking far too long, and watch DBU spend from your billing system tables. ClickRuntime calls your workspace's REST API and runs SQL on a SQL warehouse you choose. Describe what you want to your AI assistant, check what it will do, and it keeps running.

Add to ChatGPTAdd to Claude
Works inChatGPTClaudeClaude CodeCursorCodexVS CodeGemini CLIZedopencodeAny MCP client

Try asking

Create a ClickRuntime automation: every day at 7am, list the Databricks job runs from the last 24 hours that failed and email me the job names, error states and links.

Paste it into your assistant. It tells you what it will send or change before it runs live.

Ideas

What you can automate in Databricks

One overnight digest of failed jobs, a nudge when a run goes past two hours, and a budget alert on yesterday's DBUs. These are ready-made starting points. Anything else you can describe works too.
Daily

Get a morning email of failed Databricks jobs

Each morning one email lists every job run that failed overnight, with links to each.

See how it works
Daily

Get an alert when daily DBU spend goes over budget

If yesterday's DBU usage passed your budget, you know first thing in the morning.

See how it works
On a schedule

Get a Slack alert when a job runs too long

Any run still going after two hours is flagged in Slack once, so stuck jobs don't burn compute all night.

See how it works Your own logic

Any other automation you can describe in words

Your own rules, schedule and AI steps with Databricks, plus any other app. Describe it and your assistant builds it.

See how to build your own

Build your own

Build any Databricks automation you can describe

The ideas above are ready-made starting points, not a fixed list. Describe what Databricks should do in your own words, with your conditions, other apps or an AI step, and your assistant writes it, tests it and runs it on ClickRuntime.

  • Your own rules

    Conditions, thresholds, filters and exceptions, written the way you'd explain them to a colleague.

  • Any number of apps

    One automation can read from one app and update several others. Over 248 apps are ready, and anything with an API works too.

  • AI where it helps

    Summaries, classification, extraction and drafts use ClickRuntime's built-in AI, with no extra key.

  • On a schedule or on each event

    Runs as often as every 5 minutes, or the moment an app sends an event, and keeps track of what it already handled.

For example, ask your assistant

Every morning, list Databricks job runs that failed overnight or took more than twice their usual duration, with the error message for each.

Every 15 minutes, check my Databricks jobs and post any failed run to Slack with the job name, the failing task and a link to the run.

When a production Databricks job fails, open a Jira issue with the error details, post it in Slack, and close the issue automatically when the next run succeeds.

Beyond chat

Have Databricks work when you're not in the chat

Connecting Databricks to ChatGPT or Claude lets you ask about it while you chat. ClickRuntime handles the part that should happen on its own: checks on a schedule and actions on each new event, set up from the same chat and running after you close it.

Setup

Connect Databricks

Databricks connects with a personal access token from your workspace, plus your workspace URL. A service principal with its own OAuth secret is the alternative Databricks recommends.

Databricks

Data / Lakehouse

  1. In your Databricks workspace, click your username in the top bar and open Settings, then Developer.
  2. Next to Access tokens, choose Manage, then Generate new token. Set a name, lifetime and scopes.
  3. Copy the token (it starts with dapi); Databricks shows it only once.
  4. In the ClickRuntime dashboard, open Connections & secrets and save the token and your workspace URL as a Databricks secret.

Good to know

Databricks limits that affect automations

  • Workspace admins can turn off personal access tokens or cap how long they last. If you can't create one, ask your admin or use a service principal.
  • A token that isn't used for 90 days is revoked automatically, so a rarely run automation may need a new token.
  • Cost alerts query system.billing.usage, so they need a SQL warehouse ID and read access to system tables.

FAQ

Questions about automating Databricks

What can ClickRuntime see in my Databricks workspace?
What the token's user and scopes allow. These automations read job runs and query billing system tables; they don't start, cancel or edit jobs.
Databricks already emails on job failure. Why use this?
Native notifications send one message per failure. The daily report gathers every failed run from the last 24 hours into one email with links, and the long-running alert catches jobs that haven't failed but are stuck.
Do I need a SQL warehouse?
Only for automations that run SQL, such as the DBU cost alert. Job reports use the Jobs API and don't need one.
How quickly will I hear about a long-running job?
Within about 15 minutes of it passing your limit, since active runs are checked every 15 minutes. You get one alert per run, not one per check.
Do I need AI for Databricks automations?
No. These list runs and compare numbers. If you want failures explained in plain English, that uses the AI that comes with every ClickRuntime plan, so you don't need your own AI key.
Is this a Databricks connector for ChatGPT or Claude?
Not for chatting with Databricks data. A chat connector lets your assistant read and change Databricks while you're in the conversation. ClickRuntime is for the work that should happen without you: you set up an automation from the chat once, and it keeps running on a schedule or on each Databricks event.

Related pages

Keep exploring

Tell your assistant what Databricks should do

Free plan. No credit card.

Add to ChatGPTAdd to Claude