5 documented Databricks Compute cost inefficiencies, each with how to detect it and how to fix it.
Databricks ComputeCER-0189
Running varied workload types (e.g., ETL pipelines, ML training, SQL dashboards) on the same cluster introduces inefficiencies. Each workload has different runtime characteristics, scaling needs, and performance sensitivities. When mixed...
Databricks ComputeCER-0222
Autoscaling is a core mechanism for aligning compute supply with workload demand, yet it's often underutilized or misconfigured. In older clusters or ad-hoc environments, autoscaling may be disabled by default or set with tight min/max...
Databricks ComputeCER-0195
Photon is frequently enabled by default across Databricks workspaces, including for development, testing, and low-concurrency workloads. In these non-production contexts, job runtimes are typically shorter, SLAs are relaxed or nonexistent,...
Databricks ComputeCER-0523
Databricks instance pools keep a set of ready cloud instances so classic clusters can start and scale faster. While an instance sits idle in a pool, Databricks charges no DBUs for it, but the cloud provider still bills the virtual machine....
Databricks ComputeCER-0530
Every classic Databricks cluster, job cluster and pipeline runs a specific Databricks Runtime (DBR) version chosen when the compute is defined. Databricks releases runtimes on a regular cadence, and its cost-optimization best practices...