Explanation
Why the waste happens and who it affects.
Consumers written as tight receive loops, running on containers, EC2 instances or scheduled workers, therefore issue a constant stream of ReceiveMessage calls, and most of them come back empty on quiet queues. Every one of those calls is a billable SQS request.
The pattern is most costly on low-traffic queues that have many pollers, such as per-tenant or per-environment queues, dead-letter queues watched by a worker, and development queues that nobody sends to but whose consumers keep running. Short polling can also return false empty responses when messages are available but not on the sampled servers, which prompts even more polling. AWS documents long polling as a way to reduce the cost of using SQS, and it is a single queue attribute or request parameter. Lambda event source mappings already use long polling, so this mainly affects custom consumers.
Billing model
The pricing dimensions that drive this cost.
SQS bills by request, and empty responses are requests too.
- Requests
- Every Amazon SQS action counts as a request, including ReceiveMessage calls that return no messages
- Payload chunks
- Each 64 KB chunk of a payload is billed as one request
- Batching
- A single request can carry 1 to 10 messages, up to a total payload of 1 MiB
- Free tier
- All customers get the first 1 million SQS requests each month free
How to detect
5 checks to find it in your estate.
- Compare the CloudWatch metric NumberOfEmptyReceives with NumberOfMessagesReceived per queue; AWS notes that high empty-receive counts can indicate that consumers use short polling or that polling is inefficient
- Check each queue's ReceiveMessageWaitTimeSeconds attribute (get-queue-attributes); a value of 0 combined with consumers that do not set WaitTimeSeconds means short polling is in effect
- Review consumer code and SDK configuration for ReceiveMessage calls without WaitTimeSeconds, or with WaitTimeSeconds set to 0, and for fixed polling loops with little or no sleep
- Look at SQS request charges in Cost Explorer against actual message volume; request counts far above what message traffic explains usually come from empty receives
- List queues with running consumers but little or no NumberOfMessagesSent, such as idle development or dead-letter queues
How to fix
5 ways to remove the waste.
- Enable long polling by setting ReceiveMessageWaitTimeSeconds on the queue or WaitTimeSeconds on each ReceiveMessage call; AWS recommends 20 seconds (the maximum) in most cases, or a shorter value down to 1 second if needed
- Make sure the HTTP client response timeout for ReceiveMessage is longer than WaitTimeSeconds to avoid HTTP errors, and adjust clients that are not AWS SDKs or that are configured with shorter timeouts
- When one application long-polls several queues, use one thread per queue so waiting on an empty queue does not block processing of others
- Receive, send and delete in batches of up to 10 messages per request to reduce request counts further
- Stop or scale down consumers attached to queues that no longer receive messages
Documentation
Vendor references for pricing and configuration.