# Exactly-Once Streaming Mode Where At-Least-Once Is Sufficient in Dataflow Canonical: https://www.pointfive.co/efficiency-hub/inefficiencies/exactly-once-streaming-mode-where-at-least-once-is-sufficient-in-dataflow Every Dataflow streaming job runs in exactly-once mode unless configured otherwise. By: PointFive Updated: 2026-09-28 [Cloud Efficiency Hub](https://www.pointfive.co/efficiency-hub) The short version Every Dataflow streaming job runs in exactly-once mode unless configured otherwise. PointFive Research Cloud cost research at PointFive GCP service [GCP Dataflow](https://www.pointfive.co/efficiency-hub/cloud-services/gcp-dataflow) Category [Compute](https://www.pointfive.co/efficiency-hub/service-category/compute) Reference CER-0490 Type Inefficient Configuration ## Explanation Why the waste happens and who it affects. Exactly-once guarantees that records are not dropped or duplicated, which requires Dataflow to deduplicate and checkpoint state along the pipeline. Google states that at-least-once mode, which allows occasional duplicates, can significantly reduce the cost and latency of a pipeline. Many streaming pipelines do not need the stronger guarantee: map-only pipelines with no aggregations such as log processing, change data capture and ETL, pipelines whose sink deduplicates or is idempotent, and pipelines writing to sinks that cannot guarantee exactly-once delivery anyway, such as Pub/Sub. Reading from Pub/Sub is also significantly optimized in at-least-once mode. These jobs keep paying for exactly-once because it is the default and the mode is chosen once at launch. ## Billing model The pricing dimensions that drive this cost. Streaming jobs bill per second, per job; rates vary by region. Streaming worker vCPU and memory Billed per vCPU-hour and GiB-hour for streaming workers Streaming Engine Compute Units With resource-based billing, Streaming Engine backend resources are metered and billed in Streaming Engine Compute Units Mode impact Exactly-once processing uses more worker and Streaming Engine resources than at-least-once for the same input Persistent Disk Billed per worker at normal rates, independent of mode ## How to detect 4 checks to find it in your estate. - Check the Streaming mode shown under Job info on the Dataflow Jobs page or job details panel for each streaming job - Classify pipelines: flag those without aggregations (no counts, sums, means or windowed combines), those that are pure transformations, and those that write to Pub/Sub or to BigQuery through the Storage Write API - Confirm whether downstream consumers already deduplicate (for example by a primary key or MERGE in BigQuery) or are idempotent - Rank candidates by Streaming Engine Compute Unit and worker spend in the Cloud Billing export ## How to fix 4 ways to remove the waste. - Launch eligible pipelines with --dataflowServiceOptions=streaming\_mode\_at\_least\_once; the mode cannot be changed in place, so start a replacement job and drain or cancel the old one - For BigQuery sinks, use the STORAGE\_API\_AT\_LEAST\_ONCE write method; at-least-once mode is not compatible with the FILE\_LOADS method - Keep exactly-once for pipelines with aggregations, business-critical results that must not double count, and non-idempotent transforms such as appending timestamps - Enable Streaming Engine with resource-based billing, which at-least-once mode requires, and compare resource usage before and after the switch ## Documentation Vendor references for pricing and configuration. - [Set the pipeline streaming mode docs.cloud.google.com](https://docs.cloud.google.com/dataflow/docs/guides/streaming-modes) - [Best practices for Dataflow cost optimization docs.cloud.google.com](https://docs.cloud.google.com/dataflow/docs/optimize-costs) - [Dataflow pricing cloud.google.com](https://cloud.google.com/dataflow/pricing) - [Exactly-once in Dataflow docs.cloud.google.com](https://docs.cloud.google.com/dataflow/docs/concepts/exactly-once) ## Related inefficiencies [Browse the library](https://www.pointfive.co/efficiency-hub) - GCP Dataflow CER-0244 ### [Idle Dataflow Workers Running After Pipeline Failure](https://www.pointfive.co/efficiency-hub/inefficiencies/idle-dataflow-workers-running-after-pipeline-failure-f6b1a) When a Dataflow pipeline fails - often due to dependency issues, misconfigurations, or data format mismatches-its worker instances may remain active temporarily until the service terminates them. In some cases, misconfigured jobs, stuck... Compute - GCP Dataflow CER-0251 ### [Pipeline Breaks from Outdated Dependency Images in Dataflow](https://www.pointfive.co/efficiency-hub/inefficiencies/pipeline-breaks-from-outdated-dependency-images-in-dataflow-fbd63) In restricted or isolated network environments, Dataflow workers often cannot reach the public internet to download runtime dependencies. To operate securely, organizations build custom worker images that bundle required libraries.... Compute - GCP Dataflow CER-0489 ### [Missed FlexRS for Delay-Tolerant Batch Dataflow Jobs](https://www.pointfive.co/efficiency-hub/inefficiencies/missed-flexrs-for-delay-tolerant-batch-dataflow-jobs) Flexible Resource Scheduling (FlexRS) is a Dataflow option for batch pipelines that trades start time for price. Dataflow queues the job and starts it within six hours of creation, and runs it on a mix of preemptible and regular VMs with... Compute --- Source: the public page above. Product screenshots and illustrative interfaces are examples, not live customer data.