# Idle SageMaker Real-Time Inference Endpoint Canonical: https://www.pointfive.co/efficiency-hub/inefficiencies/idle-sagemaker-real-time-inference-endpoint A SageMaker real-time inference endpoint keeps its instances running and billed for as long as the endpoint is in service, whether or not any... By: PointFive Updated: 2026-09-28 [Cloud Efficiency Hub](https://www.pointfive.co/efficiency-hub) The short version A SageMaker real-time inference endpoint keeps its instances running and billed for as long as the endpoint is in service, whether or not any application calls it. PointFive Research Cloud cost research at PointFive AWS service [AWS SageMaker](https://www.pointfive.co/efficiency-hub/cloud-services/aws-sagemaker) Category [AI](https://www.pointfive.co/efficiency-hub/service-category/ai) Reference CER-0559 Type Idle or Unused Resource ## Explanation Why the waste happens and who it affects. Endpoints are created for model evaluations, demos, A/B tests, proofs of concept and previous model versions, and they are often left in place after the traffic moves to a new endpoint or the project ends. Because models are frequently hosted on GPU or accelerator instances, and production variants may run several instances for availability, a single forgotten endpoint can be expensive. The waste is easy to overlook because an idle endpoint reports no errors and nothing breaks. It is common in teams that deploy endpoints from notebooks or pipelines without a teardown step, and in accounts where each model version gets its own endpoint. AWS Compute Optimizer includes SageMaker endpoints in its idle resource recommendations, using zero invocations over the lookback period as the idle signal. ## Billing model The pricing dimensions that drive this cost. Real-time inference is billed for the instances behind the endpoint, not for the requests it serves. Instance usage Each instance behind a production variant is charged for the instance type chosen for as long as the endpoint is running Data processed Data processed in and out of the endpoint is charged separately per GB Scale to zero Endpoints that host inference components can scale in to zero instances, removing instance charges while there is no traffic Serverless inference On-demand serverless inference is billed only for the compute used to process requests, by the millisecond, plus data processed, so idle time is not charged (Provisioned Concurrency adds a charge for keeping capacity warm) ## How to detect 5 checks to find it in your estate. - Review AWS Compute Optimizer idle recommendations for Amazon SageMaker endpoints, which flag endpoints with zero invocations over the 14-day lookback period - In CloudWatch (AWS/SageMaker namespace), check the Sum of Invocations per EndpointName and VariantName over at least 14 to 30 days; no datapoints or a sum of zero means the endpoint served no requests - Check instance metrics in /aws/sagemaker/Endpoints (CPUUtilization, GPUUtilization, MemoryUtilization) to confirm the instances are doing no work, and note the instance type and count to size the waste - List endpoints with list-endpoints and compare creation time and last update with active model versions and application configuration to find superseded or experimental endpoints - Confirm with the model owner that no scheduled or low-frequency caller, such as a monthly batch or a failover path, depends on the endpoint ## How to fix 5 ways to remove the waste. - Delete endpoints that are no longer used with delete-endpoint; the endpoint configuration and model can be kept so the endpoint can be recreated later if needed - For endpoints with intermittent traffic, move the model to inference components and allow scale in to zero by setting MinInstanceCount to 0 in ManagedInstanceScaling and registering each inference component with a minimum capacity of 0; add a step scaling policy triggered by the NoCapacityInvocationFailures metric, and accept that scaling out from zero takes several minutes during which requests fail - Move spiky workloads that can tolerate variable p99 latency to serverless inference, and latency-insensitive workloads to asynchronous inference, which can scale down to zero - Consolidate several lightly used endpoints onto a multi-model or multi-container endpoint so they share instances - Add an owner and expiry tag to endpoints created for experiments, and include endpoint deletion in the teardown step of notebooks and pipelines ## Documentation Vendor references for pricing and configuration. - [Viewing idle resource recommendations docs.aws.amazon.com](https://docs.aws.amazon.com/compute-optimizer/latest/ug/view-idle-recommendations.html) - [Inference cost optimization best practices docs.aws.amazon.com](https://docs.aws.amazon.com/sagemaker/latest/dg/inference-cost-optimization.html) - [Scale an endpoint to zero instances docs.aws.amazon.com](https://docs.aws.amazon.com/sagemaker/latest/dg/endpoint-auto-scaling-zero-instances.html) - [Amazon SageMaker AI metrics in Amazon CloudWatch docs.aws.amazon.com](https://docs.aws.amazon.com/sagemaker/latest/dg/monitoring-cloudwatch.html) - [Amazon SageMaker AI Pricing aws.amazon.com](https://aws.amazon.com/sagemaker/ai/pricing/) ## Related inefficiencies [Browse the library](https://www.pointfive.co/efficiency-hub) - AWS SageMaker CER-0333 ### [Idle SageMaker Notebook Instances Left Running Continuously](https://www.pointfive.co/efficiency-hub/inefficiencies/idle-sagemaker-notebook-instances-left-running-continuously) SageMaker notebook instances are billed continuously while in an active state - and critically, they do not automatically shut down when idle. Closing a browser tab, shutting down a Jupyter kernel, or simply walking away does not stop the... AI - AWS SageMaker CER-0383 ### [SageMaker Studio Applications Without Idle Shutdown](https://www.pointfive.co/efficiency-hub/inefficiencies/sagemaker-studio-applications-without-idle-shutdown) In SageMaker Studio, each running JupyterLab or Code Editor application runs on an instance that is billed for as long as the application is running, whether or not the user is doing anything. Data scientists routinely leave spaces running... AI - AWS SageMaker CER-0384 ### [On-Demand SageMaker Training Jobs Without Managed Spot Training](https://www.pointfive.co/efficiency-hub/inefficiencies/on-demand-sagemaker-training-jobs-without-managed-spot-training) SageMaker training and hyperparameter tuning jobs run on on-demand instances unless managed spot training is explicitly enabled on the job. Many training workloads tolerate interruption: experiments, scheduled retraining, tuning sweeps and... AI --- Source: the public page above. Product screenshots and illustrative interfaces are examples, not live customer data.