Delta Lake

2 documented Delta Lake cost inefficiencies, each with how to detect it and how to fix it.

2 inefficiencies

Delta LakeCER-0177
Missing Delta Optimization Features for High-Volume Tables
In many Databricks environments, large Delta tables are created without enabling standard optimization features like liquid clustering (or, on older tables, partitioning and Z-Ordering). Without these, queries scanning large datasets may...
Storage
Delta LakeCER-0529
Unvacuumed Delta Tables Retaining Unreferenced Data Files
Delta Lake never modifies data files in place. Every UPDATE, DELETE, MERGE, overwrite and OPTIMIZE writes new files and marks the old ones as removed in the transaction log, but the old files stay in cloud object storage so earlier table...
Storage