Explanation
Why the waste happens and who it affects.
When an integration is connected to an account, subscription or project without host tag filters, dev and test VMs, batch workers and short-lived autoscaled instances are all counted, even if the team only meant to monitor production. On Azure, nodes in App Service Plans are counted as hosts too, and on Google Cloud, services such as Dataflow can create billable GCE hosts.
The integration also brings in other billable usage: on AWS, active Lambda functions and CloudWatch custom metrics are billed, and on Azure, enabling App Insights custom metric collection pulls in every custom metric within the integration's scope. The symptom is a billable host count well above the number of hosts running the Agent, typically after whole cloud organizations are connected to Datadog in one step.
Billing model
The pricing dimensions that drive this cost.
Integration-discovered resources are billed on the same meters as Agent-monitored ones, and a host is not double-billed when it runs both.
- Infrastructure host
- Any VM picked up by a cloud integration or running the Agent, billed on the high-water mark of the lower 99 percent of hourly host counts
- Serverless functions
- AWS Lambda billed on the average number of functions executed and monitored per hour across the month
- Custom metrics
- CloudWatch custom metrics and Azure App Insights custom metrics collected by the integration are billed as Datadog custom metrics
- Non-billable resources
- Metrics for services such as ELB, RDS and DynamoDB on AWS, or Cloud SQL and Pub/Sub on Google Cloud, do not add to monthly billing
How to detect
4 checks to find it in your estate.
- In the Infrastructure List, find hosts that show only the cloud provider logo or report only aws.*, azure.* or gcp.* metrics: these are billed hosts discovered by the integration with no Agent
- Open each AWS account, Azure app registration and Google Cloud project in the integration tile and check whether the host tag filter (Limit Metric Collection to Specific Resources on AWS, Optionally limit metrics collection to hosts with tag on Azure and Google Cloud) is empty
- Compare the billable host count in Plan and Usage with the number of hosts running the Agent, and break integration-only hosts down by account, subscription, project and env tag
- Check which AWS accounts send CloudWatch custom metrics and Lambda functions, and whether Azure App Insights custom metric collection is enabled on subscriptions that do not need it
How to fix
5 ways to remove the waste.
- Add host tag filters to each integration, either an inclusion list such as datadog:monitored or env:production, or exclusions with the ! notation such as !datadog:no, and make sure the tags are applied to the cloud resources
- Remove non-production accounts, subscriptions or projects from the integration if they do not need Datadog monitoring, or give them a tight filter
- Limit Lambda and CloudWatch custom metric collection on AWS to tagged resources, filter Azure App Service Plans by tag, and turn off App Insights custom metric collection where it is not used
- Preconditions: filters only affect hosts without a running Agent, since Agent hosts are always billed; previously discovered hosts can stay in the Infrastructure List for up to two hours after a filter change without counting toward billing
- On AWS, if EC2 instances require IMDSv2, use Agent 7.64.0 or later or set ec2_prefer_imdsv2 so Agent hosts are not billed twice
Documentation
Vendor references for pricing and configuration.