Explanation
Why the waste happens and who it affects.
When sampling is turned off, set to keep 100% of traces, or never configured on older instrumentation, telemetry volume grows in step with traffic, and every record is billed as Log Analytics ingestion. Verbose logging makes it worse: Information or Debug level log output from application code, frameworks and hosts such as Azure Functions lands in the AppTraces table and often becomes the largest billed table in the workspace.
Common causes are sampling disabled during troubleshooting and never restored, a sampler ratio of 1.0 copied into every environment, and log levels left at Information in production. Microsoft calls OpenTelemetry sampling the primary tool for tuning how much data Application Insights collects. Metrics are not sampled, so alerting on them keeps working, and the Application Insights sampler keeps whole traces together, so a sampled data set still supports end-to-end transaction views.
Billing model
The pricing dimensions that drive this cost.
Workspace-based Application Insights stores its data in a Log Analytics workspace, so ingestion and retention are billed by that workspace.
- Log ingestion
- Billed per GB of data sent to the workspace, based on the billed size of each record (_BilledSize)
- Interactive retention
- Application Insights data is kept for 90 days without a retention charge; longer retention is billed per GB per month
- Sampling
- Reduces the number of traces (and, by default, the logs tied to unsampled traces) sent from the application; metrics are never sampled
- Ingestion sampling
- Drops data at the Azure Monitor ingestion point, with less control over which traces are kept
How to detect
5 checks to find it in your estate.
- Run the Microsoft validation query union requests,dependencies,pageViews,browserTimings,exceptions,traces | where timestamp > ago(1d) | summarize RetainedPercentage = 100/avg(itemCount) by bin(timestamp, 1h), itemType; a RetainedPercentage of 100 for high-volume types means nothing is being sampled
- Rank tables in the linked workspace by billed volume with search * | where _IsBillable == true | summarize sum(_BilledSize) by $table, and look for AppTraces, AppDependencies and AppRequests at the top
- Break down AppTraces by AppRoleName, logger category (Properties['Category']) and SeverityLevel to find applications logging at Information or Debug level in production
- Check application configuration for OTEL_TRACES_SAMPLER set to always_on or a ratio of 1.0, sampling ratios of 1.0 in code, or APPLICATIONINSIGHTS_SAMPLING_PERCENTAGE of 100 for the Java agent
- Use the workspace Usage workbook in Log Analytics Workspace Insights to see which Application Insights resources contribute most ingestion
How to fix
6 ways to remove the waste.
- Configure fixed-rate sampling (microsoft.fixed_percentage, for example a ratio of 0.1) or rate-limited sampling (microsoft.rate_limited, maximum traces per second) in the Azure Monitor OpenTelemetry distro; environment variables override code settings. If Failures and Performance views look incomplete, raise the rate
- Keep trace-based sampling for logs enabled so logs belonging to unsampled traces are dropped, and set application log levels to export only ERROR, adding WARN only where it is actionable
- Disable instrumentation and host logging that nobody uses, limit Ajax call reporting in the JavaScript SDK, and avoid high-cardinality custom metric dimensions and 'Enable alerting on custom metric dimensions' unless needed
- Use ingestion sampling (Usage and estimated costs, Data Sampling) only when application code cannot be changed, since it drops data without regard to trace boundaries
- Set a daily cap with an alert at a threshold such as 90% as a last-resort guard, not as the primary cost control, because reaching it stops ingestion until the next day
- Migrate any remaining classic Application Insights resources to workspace-based resources so they can use commitment tiers, Basic Logs and table-level retention
Documentation
Vendor references for pricing and configuration.
- Sampling in Azure Application Insights with OpenTelemetrylearn.microsoft.com
- Configuring OpenTelemetry in Application Insightslearn.microsoft.com
- Cost optimization in Azure Monitorlearn.microsoft.com
- Azure Monitor Logs Cost Calculations And Optionslearn.microsoft.com
- Troubleshoot High Data Ingestion in Application Insightslearn.microsoft.com
- Azure Monitor pricingazure.microsoft.com