7 documented Databricks Clusters cost inefficiencies, each with how to detect it and how to fix it.
Databricks ClustersCER-0098
In non-production environments - such as development, testing, and experimentation-many teams default to on-demand nodes out of habit or caution. However, Databricks offers built-in support for using spot instances safely. Its job...
Databricks ClustersCER-0159
Photon is enabled by default on many Databricks compute configurations. While it can accelerate certain SQL and DataFrame operations, its performance benefits are workload-specific and may not justify the increased DBU cost. Many...
Databricks ClustersCER-0022
In many environments, users launch Databricks clusters for development or analysis and forget to shut them down after use. When no auto-termination policy is configured, these clusters remain active indefinitely, incurring unnecessary...
Databricks ClustersCER-0116
Databricks supports AWS Graviton-based instances for most workloads, including Spark jobs, data engineering pipelines, and interactive notebooks. These instances offer significant cost advantages over traditional x86-based VMs, with...
Databricks ClustersCER-0125
Databricks users can select from a wide range of instance types for cluster driver and worker nodes. Without guardrails, teams may choose high-cost configurations (e.g., 16xlarge nodes) that exceed workload requirements. This results in...
Databricks ClustersCER-0067
Interactive clusters are often left running between periods of active use. To mitigate idle charges, Databricks provides an "autotermination" setting that shuts down clusters after a period of inactivity. However, if the termination period...
Databricks ClustersCER-0008
Interactive clusters are intended for development and ad-hoc analysis, remaining active until manually terminated or until their auto-termination inactivity timeout elapses. When used to run scheduled jobs or production workflows, they...
