Databricks serverless vs classic compute: which is cheaper?

Short answer: Serverless jobs compute has a higher $/DBU, but that rate includes the VMs, and you don't pay for a cluster sitting through startup or idling. It usually wins for short, frequent or spiky jobs, where a classic cluster spends a big share of its billed time starting up. Classic usually wins for long, steady jobs, especially when you've cut the VM bill with spot instances, reservations or savings plans. The only reliable test is to run the job both ways and compare.

Why the $/DBU comparison misleads

On a classic jobs cluster you pay twice: Databricks bills DBUs, and your cloud bills the VMs. On serverless you pay once. Databricks says it plainly: "for serverless services, the DBU cost already includes the virtual machine costs" (Databricks docs). So comparing $0.15/DBU for classic Jobs on AWS with $0.35/DBU for serverless is apples to oranges.

A fairer number is the classic all-in cost per DBU: what you pay Databricks plus the cloud, divided by the DBUs burned. Here it is for each cloud's default instance with Photon on (Photon is on by default for classic jobs compute created in the UI, and always on for serverless, per Databricks docs):

*Classic all-in = (DBU/h × $/DBU + VM on-demand $/h) ÷ DBU/h for one node of m5d.2xlarge (AWS), D8ds v5 (Azure) or n2-standard-8 (GCP), Photon on, Premium tier. Pulled from DBU Watch's daily data.

The catch: serverless and classic don't burn the same number of DBUs for the same job. Serverless picks its own instance types and scaling, so you can't predict its DBU count from your cluster config. Databricks' own guidance is to "run and benchmark a representative or specific workload and then analyze the billing system table" (Databricks serverless FAQ).

Where the money actually goes on classic

A classic job cluster costs money for more than the minutes your code runs. The VMs are running, and billing, while the cluster boots, runs init scripts and installs libraries, and while it scales and shuts down. If a job runs for 5 minutes and the cluster spends another 5 or 6 starting and stopping, roughly half the bill buys nothing. Serverless works differently: Databricks says serverless jobs "bill only when a command or query runs" and that there's "no charge for idle capacity" (Databricks). That's why it shines on short jobs.

Serverless also has two performance modes on one SKU. Standard mode "uses less compute to reduce costs" and tolerates "slightly higher startup latency of 4 to 6 minutes"; Performance optimized starts and runs faster. "Both modes use the same SKU, but standard performance mode consumes fewer DBUs" (Databricks docs). Databricks recommends Performance optimized only for workloads that need to launch within a minute (Databricks pricing page).

Worked examples

These are examples, not benchmarks. Assumptions: AWS us-east-1 (N. Virginia), Premium tier, 1 driver + 4 workers of m5d.2xlarge (8 vCPU, 32 GB), Photon on, on-demand EC2 at $0.452/hour per node, and 6 extra billed minutes per run for startup and shutdown (check your own cluster event log; it varies). That cluster costs $5.24/hour all-in ($2.98 Databricks + $2.26 EC2). Serverless jobs list at $0.35/DBU in this region.

Example A: a 60-minute nightly ETL job

• Classic cost per run: $5.24 × 66/60 = $5.76, or $172.91 a month at 30 runs.
• Serverless breaks even at 16.5 DBUs per run ($5.76 ÷ $0.35).
• The classic cluster itself burns 19.9 DBUs during the 60-minute runtime. So serverless wins only if it does the same work in under 0.83× the DBUs the classic cluster uses.

Whether that holds depends on how well serverless sizes your job. On a long job, startup is a small slice (9% here), so the classic cluster isn't wasting much. And if your EC2 is on spot or a savings plan, the classic side gets cheaper still. This is the kind of job where classic often holds its own.

Example B: a 5-minute job every hour

• Classic cost per run: $5.24 × 11/60 = $0.96, or $691.65 a month at 720 runs.
• Serverless breaks even at 2.74 DBUs per run.
• The classic cluster burns only 1.66 DBUs of useful runtime per run. Serverless can use up to 1.66× that and still come out cheaper, because 55% of every classic run is startup and shutdown.

Short, frequent jobs are where serverless most often wins. If you can't move to serverless, the classic fix is to run several tasks on one shared job cluster so you pay startup once per job, not per task (Databricks docs).

What about Photon off?

Without Photon, the same classic cluster costs $3.29/hour instead of $5.24. That lowers the classic side, but serverless always runs Photon, so you're no longer comparing the same engine. See Is Photon worth it? for that trade-off.

When each one wins

Lean serverless when:

• Jobs are short (minutes) or run many times a day, so startup is a big share of billed time.
• Load is spiky or unpredictable and your clusters sit oversized "just in case."
• Nobody owns cluster tuning. Serverless "automatically and continuously optimizes infrastructure" (Databricks docs), and a badly tuned classic cluster is expensive.
• Latency isn't critical, so you can use Standard mode and its lower DBU use.

Lean classic when:

• Jobs run for hours on a steady, well-understood cluster size.
• You get real VM discounts (spot workers, reservations, savings plans). Those only apply to the classic VM line, since serverless VMs are bundled into the DBU.
• You need things serverless doesn't support. It needs Unity Catalog and standard access mode (docs), and it has limits like restricted streaming triggers (docs).

How to test it on your own job

1. Clone the job and switch the copy to serverless (Standard mode unless you need fast starts).
2. Let both run for a week.
3. Pull serverless DBUs per run from system.billing.usage, where serverless usage carries the job_id, job_run_id and product_features.performance_target (docs; sample queries in Monitor serverless cost). Records can take up to 24 hours to show up (docs).
4. For the classic run, add the DBU cost and the VM cost from your cloud bill, or use our calculator's on-demand estimate.
5. Enter the measured DBUs in the serverless vs. classic calculator.

Serverless rates differ by region (AWS Premium serverless jobs ranges from $0.35 to $0.46/DBU in our data), and Databricks sometimes runs promotional discounts on specific SKUs (Databricks). Check your region in the price table, and we'll flag any change in the price history.