Explanation
Why the waste happens and who it affects.
Dense Kubernetes nodes, sidecar-heavy pods, short-lived job containers and crash-looping pods can push the average container count above that allotment, and every container above it is billed per container-hour at the on-demand rate. Datadog meters containers in five-minute increments, so even short bursts of many containers add billable container-hours.
The waste has two forms: monitoring containers nobody needs data from, such as sidecars, init-style utility containers or whole non-production namespaces, and paying on-demand rates for a steady container baseline that could be covered by prepaid containers. It grows on Agent-monitored Kubernetes and ECS clusters as sidecars and microservices are added while the host count, and therefore the allotment, stays flat.
Billing model
The pricing dimensions that drive this cost.
Container usage is compared each hour against an allotment made of included containers plus any contracted container commitment.
- Included containers
- 5 containers per host on Pro and 10 per host on Enterprise, averaged across all infrastructure
- On-demand containers
- Containers above the allotment, billed per container per hour ($0.002 per container-hour on the public pricing page)
- Prepaid containers
- A contracted container commitment billed per container per month ($1 on the public pricing page) that raises the allotment
- Five-minute metering
- Container counts are sampled every five minutes, averaged over each hour, and the allotment is deducted hour by hour
How to detect
4 checks to find it in your estate.
- Chart datadog.estimated_usage.containers and datadog.estimated_usage.containers.by_tag against the allotment (hosts times 5 or 10, plus any prepaid containers) to see when and where usage goes over
- In Plan and Usage, use the Billable view of Usage Details to see how many container-hours are billed on demand each month
- Break container usage down by kube_namespace, image and container name to find sidecars, utility containers and non-production namespaces that add many containers but are not used in dashboards or monitors
- Look for pods in CrashLoopBackOff and high-churn Jobs or CronJobs: any container that runs for more than ten seconds counts in that metering interval
How to fix
5 ways to remove the waste.
- Exclude containers that do not need monitoring with Container Discovery Management, for example DD_CONTAINER_EXCLUDE by image, name or kube_namespace (Agent 7.20+), or the ad.datadoghq.com/exclude pod annotation; excluded containers send no metrics or logs
- Keep the default pause container exclusion enabled (DD_EXCLUDE_PAUSE_CONTAINER), which stops pause containers from counting as billable
- If a steady container count above the allotment remains, buy prepaid containers for that baseline through your account team instead of paying on-demand container-hours
- Fix crash-looping workloads and consolidate low-value sidecars; each removed container reduces billable container-hours on every node where it runs
- Tradeoff: excluded containers disappear from Datadog metrics and logs, so confirm no monitor, SLO or dashboard depends on them; kubernetes.containers.running and docker.containers.running still count all containers
Documentation
Vendor references for pricing and configuration.