Why Your Cloud Bill Doubled Overnight: A FinOps Debugging Guide
The Slack message is always the same shape: someone from finance asks why last month’s AWS bill is 2.3x normal, and nobody on the engineering side has an answer ready. Not because the team is careless — because most cost dashboards are built to show that spend happened, not why.
This is the debugging path that actually finds the cause, in the order that finds it fastest.
Step 1: Confirm It’s Real, Not a Billing Artifact
Before treating this as an incident, rule out the boring explanations:
- Reserved Instance or Savings Plan expiry. A one- or three-year commitment lapsing silently reverts everything it covered to on-demand pricing — often a 40–60% jump with zero change in actual usage.
- A billing cycle quirk. Months with five weekends, or a provider’s mid-cycle pricing change, can shift totals a few percent without anything being wrong.
- Currency or discount program changes. Enterprise Discount Program renewals and FX shifts on non-USD accounts both move the top-line number independent of consumption.
If none of those explain the gap, it’s a real spike — move to step 2.
Step 2: Go Straight to Cost Explorer, Grouped by Service, Then by Usage Type
Skip the dashboard’s default monthly view. Group by service, then drill into the top mover by usage type (not just “EC2” — you want “EC2: BoxUsage:m5.4xlarge” vs “EC2: DataTransfer-Out-Bytes”). This alone identifies the culprit in a large majority of spikes, because cost problems concentrate: one service, one usage type, one account.
Common patterns, ranked by how often they’re the actual cause:
- Data transfer, not compute. Cross-AZ or cross-region transfer between microservices, or an S3 bucket suddenly serving public traffic, shows up as a “small” line item that quietly became the biggest one.
- A forgotten autoscaling ceiling. Someone raised the max instance count for a load test and never lowered it back.
- Orphaned resources. Unattached EBS volumes, idle NAT Gateways, and unused Elastic IPs bill continuously and never show up in an application dashboard because nothing is using them — that’s exactly why they’re invisible.
- A new feature with no cost guardrail. A GenAI feature calling a frontier model per-request, with no caching and no token ceiling, is the newest version of this pattern and the fastest-growing one in 2026.
Step 3: Check for Tag Coverage Before You Trust Any Breakdown
If 20%+ of spend shows up as “No tag” or “Untagged,” your breakdown is lying to you by omission — the real spike could be hiding entirely inside the untagged slice. Cost allocation tags need to be applied at resource creation (via Terraform/CloudFormation defaults, not after the fact) for this analysis to be trustworthy going forward. This is worth fixing permanently, not just for this incident.
Step 4: Correlate the Timestamp With a Deploy, Not Just a Date
Cost Explorer works in daily granularity by default, which is too coarse. Cross-reference the day cost started climbing against your deploy log, feature-flag changes, and any marketing/traffic events. A spike that starts at 2pm on a Tuesday almost always maps to a specific commit or config change — pull up what shipped that day before you look anywhere else.
Step 5: Set the Guardrail So This Specific Spike Can’t Recur
Root-causing the spike is half the job. The other half is a specific, automated guardrail:
- A budget alert scoped to the usage type that caused it, not just the account total (account-level alerts fire too late to matter).
- An autoscaling max that requires a PR to change, not a console click.
- For LLM-calling features specifically: a hard per-request token ceiling and a cache layer in front of any repeated prompt pattern.
Generic “we’ll review costs monthly” doesn’t prevent a repeat of this spike — only a guardrail shaped like the actual cause does.
A cost spike is a signal, not just a bill. Most of the time it’s pointing at a real architectural gap — a caching layer that doesn’t exist, a tagging policy nobody enforces, or a ceiling nobody set. Find that, and the next spike either doesn’t happen or gets caught before finance has to ask.
Questions & Comments
Loading comments…