The $18K EBS Mistake

Last month, our AWS bill jumped 340%. The culprit? 47 detached EBS volumes from deleted EC2 instances that nobody noticed for 3 months.

How It Happened

We had a cleanup script that was supposed to delete orphaned volumes, but the volumes weren’t being tagged as “available” because they were:

1. Still attached to a snapshot backup job
2. Had pending CloudWatch alarms
3. Marked as “protect-from-acidental-deletion”

Our volumes had all three.

The Cost Breakdown

47 detached volumes × 500 GB each × $0.10/GB-month = $2,350/month
3 months = $7,050

Plus:
- Unattached EIPs: $104 over 92 days
- Old EBS snapshots: $426 over 3 months  
- Idle RDS instances (3): $18,000 over 3 months

Total: **$26,580 over 3 months**

The Cleanup Script (Fixed Version)

The fix involves checking creation dates (>7 days old), verifying no recent snapshots, and always running in dry-run mode first.

The Prevention

1. Tag everything on creation - Environment, Team, CostCenter
2. Set lifecycle policies - Auto-delete untagged resources after 7 days
3. Daily cost anomaly alerts - AWS Cost Explorer + SNS
4. Monthly resource audit - Script that lists all “zombie” resources

Lessons Learned

1. Never run destructive scripts without dry-run first
2. Tag all resources with ownership metadata
3. Automate cleanup but add safety checks (time thresholds, snapshots)
4. Monitor cost anomalies, not just total spend

Incident cost: $26,580 | Duration: 3 months | Fix: Tagging policy + automated cleanup with safeguards