It's no secret that AI powered user experiences continue to redefine applications of all sizes. For Microsoft customers, Azure OpenAI services are becoming foundational to how organizations build products that leverage AI. But these workloads don't just reshape user experience, they reshape cloud costs. Worldwide AI spending is set to reach approximately $1.5 trillion in 2025, according to Gartner.
Rising costs aren't slowing the enthusiasm for innovation. But while AI adoption continues to grow, budgets are tightening. Teams now feel pressure to bring the same level of cost visibility and governance to AI as they do for cloud at large.
This is easier said than done. Azure OpenAI workloads behave differently from traditional cloud services. Token usage changes with prompt design and user behavior. Model choice affects both performance and cost. Shared endpoints obscure ownership.
Though the tooling is still maturing, the direction is clear. FinOps for AI is the new necessity. Visibility is the first step, but visibility alone won't reduce costs. To truly optimize AI spend, organizations must achieve ongoing efficiency.
In this paper, we share PointFive's research on how to achieve continuous efficiency, identifying four cost-saving patterns in Azure OpenAI deployments.
Azure OpenAI offers two primary billing approaches, Pay-As-You-Go (PAYG) and Provisioned Throughput Units (PTUs). Each serves different workload needs and pricing models.
The PAYG model is token based. Tokens are the basic units (like text, photos, videos) that Azure OpenAI models read and generate. Tracking tokens is difficult because usage shifts with prompt design, model choice, and user behavior, which creates unpredictable cost patterns. Input tokens represent the prompt, while output tokens represent the units the model returns. Output tokens add significant variability because the model expands on the prompt in ways that reflect its internal reasoning rather than the input’s size.In the PAYG model, you are charged based on actual token usage across both input and output tokens. Pricing is defined per 1,000 tokens and varies by model (e.g., GPT-3.5, GPT-4o, GPT-4 Turbo). This creates cost differences across workloads. PAYG also applies to embeddings (like vector generation) and other model operations that generate tokens (reranking or chained calls). This model works best for workloads with low or variable traffic, such as experimentation, QA, or episodic usage.
Differing from token based pricing, PTUs allocate a dedicated slice of capacity to your OpenAI deployment. This pricing model gives you consistent throughput and low-latency guarantees. You pay based on the number of provisioned PTUs. PTUs do not bill on usage; they bill for full capacity even when usage is low. PTUs are model specific, so each model family requires its own PTU allocation with distinct performance characteristics. PTUs are best suited for production-grade scenarios where stable performance and response time are critical.
PTUs come in two billing formats:
💡 Key insight: The choice between PTU and PAYG isn’t purely a cost decision. It’s a performance trade-off. While PAYG may appear cheaper under certain workloads, PTUs offer consistent throughput and latency that may be essential for user experience or SLA commitments. As such, any shift between pricing models should be informed not just by utilization metrics, but also by a clear understanding of the workload’s criticality, performance needs, and business context. What saves money in one environment might introduce risk (like performance degradation, SLA and reliability issues) in another.
Through our early analysis, we’ve identified four key areas where cost optimization for Azure OpenAI is both viable and impactful. These patterns represent a blend of common inefficiencies and emerging FinOps levers for AI workloads.
Optimization opportunity: If a deployment is using PTUs at full capacity with consistent traffic, such as a production chatbot, inference layer, or RAG engine, it’s a strong candidate for reserved PTUs. Switching from on-demand PTUs to reserved PTUs can significantly reduce costs. Azure supports monthly or annual reservations, with discounts of up to ~80% compared to on-demand rates.
How it’s overlooked: Many teams adopt on-demand PTUs during experimentation or early launch phases because usage patterns are uncertain. But once traffic stabilizes, continuing to pay the higher on-demand rate creates ongoing, hidden waste. Monthly reservations are relatively flexible and easy to adopt, making this a low-effort, high-impact lever for cost savings.
What to watch for: This opportunity is most applicable to workloads that run 24/7 and consume consistent throughput. The change is a simple pricing switch. It requires no changes to code, quota levels, or deployment configuration.
Optimization opportunity: Azure OpenAI prices PTUs based on provisioned capacity, not usage. If quotas are overestimated and utilization is below 70% on a sustained basis, that idle capacity still incurs full cost and is wasteful. Azure provides a metric (ProvisionedUtilization) that shows the percentage of capacity being consumed.
How it’s overlooked: Rightsizing is one of the most direct ways to reduce cost while maintaining performance guarantees. It involves reviewing capacity allocation in light of observed traffic patterns and scaling PTUs down where safe.
What to watch for: Sustained utilization below 70% for five or more days, or flat utilization with minimal variance over extended periods, especially for newer GPT-4 deployments, may signal a need to trim quotas.
Optimization opportunity: Development and QA environments often run intermittently and with low traffic. In these cases, PTUs (even when rightsized) may be overkill. Switching to PAYG means paying only for actual token usage, which can lead to better alignment between cost and value.
How it’s overlooked: PTUs make sense for production due to their performance guarantees, guarantees that often aren’t needed for test or sandbox setups. PAYG also avoids the operational effort of provisioning, scaling, and monitoring quotas for low-stakes environments.
What to watch for: Low-utilization PTU deployments tagged as “dev”, “test”, or “non-prod” are strong candidates for migration to PAYG as long as this shift does not introduce unacceptable performance degradation.
Optimization opportunity: Not all workloads run continuously. Some are tied to business cycles, weekly reports, marketing campaigns, or seasonal spikes. For these, keeping PTUs provisioned 24/7 results in excess cost during idle windows. A better strategy is to dynamically scale PTU capacity up and down on a schedule.
How it’s overlooked: Azure does not currently offer native PTU scheduling, but teams can build automation via the API to provision and deprovision capacity on a predefined schedule. This reduces waste without losing performance during peak windows.
What to watch for: Recurring traffic patterns, idle time blocks, or workloads tied to calendar-based campaigns are strong candidates for schedule-based scaling.
PointFive helps organizations achieve ongoing efficiency by eliminating cloud waste and reclaiming engineering focus. We do this by bringing visibility to the full spectrum of infrastructure costs, including AI services.
To achieve ongoing efficiency at scale, PointFive takes a holistic approach to reducing cloud spend. We focus on Cloud Efficiency Posture Management rather than budgeting or cost reporting. CEPM is an engineering-native approach that continuously monitors infrastructure for inefficiencies, gives engineering teams the context they need to understand and validate issues quickly, and streamlines remediation with tools like AI prompt remediation and 1-Click fixes.
Beyond detection, PointFive’s research team develops and maintains a library of 200+ validated savings opportunities across modern cloud architectures, including support for Azure OpenAI workloads. PointFive’s reliable recommendations and seamless remediation workflows give teams confidence that their infrastructure is efficient, optimized, and well managed. To support these insights, our platform connects directly to cloud APIs via an agent-less integration. It continuously analyzes resource metadata, metrics, and billing data, and surfaces actionable, policy ready recommendations for FinOps and platform teams.
From Kubernetes to serverless to now AI, our mission remains the same: turn complexity into cost control. Ready to see PointFive in action? Book a demo.
As organizations scale their use of AI, cost optimization must evolve alongside model performance and deployment strategies. At PointFive, we are committed to equipping FinOps and platform engineering teams with the insights and tools they need to manage Azure OpenAI workloads efficiently and responsibly. Our goal is to support better decision-making grounded in business context, performance needs, and financial accountability.
