Explanation
Why the waste happens and who it affects.
Soft-deleted data is billed like any other stored data for the whole retention period. For buckets holding long-lived data this is a modest insurance cost, but for buckets whose objects are written and deleted quickly, the soft-deleted bytes can exceed the live bytes.
Google warns that if a bucket primarily contains temporary data, soft delete can result in significantly increased storage costs, and names typical cases: staging files, query results, source parts of composite objects and workloads with frequent writes and deletes. Pipelines that rename objects by copying and deleting them, or that rewrite the same objects daily, create the same effect.
Billing model
The pricing dimensions that drive this cost.
- Soft-deleted storage
- Billed at the storage rate of each object's storage class, the same as live and noncurrent objects, until the retention period expires
- Retention duration
- 7 days by default, configurable from 7 to 90 days, or 0 to disable
- Early deletion
- Objects in Nearline, Coldline or Archive that are soft deleted before their minimum storage duration incur early deletion charges, reduced by the length of the retention duration
- Restores
- Restoring soft-deleted objects is billed as operations, and retrieval fees do not apply
How to detect
4 checks to find it in your estate.
- Review the Cloud Storage Soft Delete recommender; it recommends disabling soft delete on buckets whose soft delete costs are over 50% of their monthly Cloud Storage cost and at least 20 USD, and shows the estimated monthly savings
- Chart storage.googleapis.com/storage/v2/total_bytes (or v2/total_byte_seconds) by the type label and compare soft-deleted-object with live-object bytes per bucket
- Identify buckets used for temporary data: staging, landing, scratch, query results, compose sources and checkpoints
- Check for pipelines that rename objects with copy plus delete, which soft-delete the original on every rename
How to fix
4 ways to remove the waste.
- Disable soft delete on buckets that hold only temporary data with gcloud storage buckets update --clear-soft-delete gs://BUCKET, or keep the minimum 7-day retention where some protection is wanted; already soft-deleted objects stay until their original retention expires
- Move temporary objects into dedicated buckets with soft delete disabled and keep durable data in buckets that retain it
- Rename objects with the objects.move API instead of copy plus delete, so renames do not leave soft-deleted copies
- Set an organization or folder default with tags so new scratch buckets are not created with soft delete, and weigh each change against the higher risk of data loss from accidental or malicious deletion
Documentation
Vendor references for pricing and configuration.