Explanation
Why the waste happens and who it affects.
The alternative is capacity pricing through BigQuery editions, where compute is billed per slot-hour with autoscaling and optional baseline and committed slots, independent of how many bytes a query scans. For projects that run a large, steady volume of scheduled transformations, BI dashboards and scan-heavy queries, the same work can cost less as slot-hours than as TiB processed, and Enterprise and Enterprise Plus commitments add a further discount (20% for one year, 40% for three years).
Projects stay on on-demand pricing because it is the default and requires no setup, even after query volume has grown far beyond the free tier. Google's Well-Architected Framework cost pillar recommends reducing query processing costs for steady-state workloads by using capacity-based pricing, and the BigQuery edition slot recommender produces cost-optimized recommendations specifically for on-demand workloads, including the cost of converting projects to the Enterprise edition. The opposite problem, reservations that are larger than the workload needs, is a separate inefficiency.
Billing model
The pricing dimensions that drive this cost.
BigQuery offers two compute pricing models that can be mixed per project.
- On-demand compute
- Billed per TiB processed by each query, with the first 1 TiB per month free and up to 2,000 concurrent slots per project
- Editions autoscaling
- Standard, Enterprise and Enterprise Plus reservations bill autoscaled slots per slot-hour, with a one-minute minimum by default
- Baseline and committed slots
- Enterprise and Enterprise Plus can add baseline slots and 1-year or 3-year capacity commitments, billed whether or not the slots are used
- Project assignment
- A project assigned to a reservation stops paying on-demand rates for the job types covered by the assignment
How to detect
4 checks to find it in your estate.
- Open Workload management, Slot estimator in the BigQuery console and select an on-demand workload to see the edition slot recommender's cost-optimized options (pay as you go, 1-year and 3-year commitments) for the organization or specific projects, based on the past 30 days
- List recommendations from google.bigquery.capacityCommitments.Recommender with gcloud or the Recommender API for on-demand projects
- From INFORMATION_SCHEMA.JOBS, sum total_bytes_billed (on-demand cost driver) and total_slot_ms (slot-hours the same work consumed) per project over 30 to 90 days to compare the two models for heavy projects
- Look for projects whose on-demand charges are consistently high and steady month to month rather than spiky
How to fix
5 ways to remove the waste.
- Model the move with the slot estimator and custom pricing, then create a reservation in the right edition with autoscaling and assign the heavy projects to it; keep spiky or low-volume projects on on-demand, since both models can run side by side
- Choose the edition by required features, not only price: Standard edition has no BigQuery ML, customer-managed keys, row-level security or capacity commitments, so projects that use those need Enterprise or higher
- Add baseline slots or a 1-year or 3-year commitment only for the steady floor of slot usage shown by the recommender, keeping the sum of baselines at or below committed slots; commitments are billed for their full term
- Set a maximum reservation size and review slot utilization after the switch, since capacity pricing replaces a per-query cost with a per-slot-hour cost that must be actively managed
- Combine with query-level savings such as partitioning and clustering, which reduce slot time as well as bytes
Documentation
Vendor references for pricing and configuration.