cost-optimizationfinopsawsplaybook

The zombie resource hunt: finding what bills but does nothing

Zombie resources are alive on the invoice and dead in production: disks with no instance, clusters with no pods, gateways with no traffic. Here is the hunt playbook, the detection query for each type, and how to delete safely.

The C3X Team··8 min read

Quick answer

Hunt in this order, easiest signal first: unattached block storage (state equals available), unattached static IPs, load balancers with zero healthy targets, NAT gateways with near zero bytes processed, snapshots older than the retention policy, idle managed databases with zero connections for 30 days, empty Kubernetes node pools, and stopped instances whose storage still bills. Each has a single decisive metric, so detection is a query rather than a judgement call. A first hunt on an untouched account typically finds 5 to 12 percent of total spend. Delete on a tag, notify, wait 14 days, snapshot, then delete pattern to make mistakes recoverable.

A zombie resource is one that appears on the invoice every hour and serves no traffic, holds no needed data, and has no owner who would notice its absence. They are created by ordinary events: a VM deleted without its disk, a service migrated but its load balancer left in place, a project cancelled but its cluster kept running just in case. Every account accumulates them, and unlike right-sizing, removing them carries no performance risk at all.

The hunt list with detection signals

ZombieDecisive signalTypical cost each
Unattached block storageState equals available / Unattached$40/mo per 500 GB gp3
Unattached public IPNo association$3.60 to $7.30/mo
Load balancer, no targetsHealthyHostCount zero for 14 days$16 to $23/mo base
NAT gateway, no trafficBytesOutToDestination near zero$32.85/mo base
Expired snapshotsOlder than retention policy$0.05/GB-mo
Idle databaseZero connections for 30 days$125/mo for db.m5.large
Empty node poolZero scheduled podsFull node cost
Stale container imagesNot pulled in 90 days$0.10/GB-mo registry

Phase 1: the unambiguous ones

Start with resources whose zombie status is a state field rather than an inference. Block storage with state available is attached to nothing: on AWS a gp3 volume bills $0.08 per GB-month regardless, so 500 GB is $40 a month. Public IP addresses with no association bill $0.005 per hour on AWS and Azure (about $3.60 and $3.65 a month) and about $0.010 per hour on GCP for a reserved but unused static IP, roughly $7.30 a month. These need no observation window because the state itself is the proof. Expect a few hundred to a few thousand dollars a month on a mid sized account and delete after a snapshot.

Phase 2: the metric based ones

The next tier needs 14 to 30 days of telemetry. A load balancer with a HealthyHostCount of zero for two weeks is serving nothing, and its base charge is about $16.43 a month for an ALB before LCU charges. A NAT gateway processing near zero bytes is $32.85 a month of pure base charge; the cost implications are covered in NAT gateway cost optimization. A managed database with zero connections for 30 days is a strong signal, though be careful: quarterly reporting databases exist and will have zero connections for 89 days and then matter enormously. Check with the owner before deleting anything with data.

Idle compute is the highest value item in this tier. An instance under 5 percent CPU with near zero network traffic for 30 days is almost certainly a zombie, and an m5.xlarge at $0.192 per hour is about $140 a month each. The full method is in idle EC2 cost elimination.

Phase 3: the accumulating ones

Snapshots, container images, and log data do not appear as individual zombies; they appear as a line that grows every month. Snapshots at $0.05 per GB-month with a daily schedule and no expiry reach thousands of dollars within a year. Container registries at about $0.10 per GB-month hold every image ever built. The fix here is not deletion but a lifecycle policy, so the problem cannot recur. Set snapshot retention to 30 days for non-production and to whatever compliance actually requires for production, and expire registry images not pulled in 90 days.

The safe deletion protocol

Deleting the wrong thing is the only real risk, and process removes almost all of it. Use five steps. One, tag every candidate with a marked-for-deletion tag and a date. Two, notify the owner if there is one, and the owning account's team if there is not. Three, wait 14 days and re-check the signal; anything that became active drops off the list. Four, for anything holding data, take a snapshot first, which costs $0.05 per GB-month and converts a data loss incident into a cheap inconvenience. Five, delete, and record what was deleted and the monthly saving.

For high risk resources, add a stop before delete step: stop the instance, or detach the volume, and wait another week. If nothing breaks, it was a zombie. The fuller process is in the orphaned resource cleanup guide.

Expected return and how to make it stick

A first hunt on an account that has never had one typically recovers 5 to 12 percent of total spend. On a $150,000 a month bill that is $7,500 to $18,000 a month for perhaps two weeks of work by one engineer. The second hunt six months later finds far less, maybe 2 to 3 percent, which is the point.

Zombies regenerate because infrastructure is created faster than it is decommissioned. Three controls slow it: require an owner tag on creation so every resource is attributable, use Terraform so destroying a stack removes everything it created rather than leaving disks behind, and price infrastructure changes before merge against the resource catalog so the cost of what is being created is visible from the start. Pair with cloud waste elimination for the recurring rhythm.

FAQ

What counts as a zombie cloud resource?

One that bills every hour but serves no traffic, holds no needed data, and has no owner who would notice its absence. Typical examples are block storage with no instance attached, public IPs with no association, load balancers with zero healthy targets, NAT gateways processing near zero bytes, snapshots past their retention policy, databases with no connections for 30 days, and empty Kubernetes node pools.

How do I find zombie resources?

Each type has one decisive signal, so detection is a query rather than a judgement call. Start with state fields that need no observation window: block storage in the available or unattached state, and public IPs with no association. Then use 14 to 30 days of telemetry for load balancers (HealthyHostCount zero), NAT gateways (near zero bytes processed), databases (zero connections), and instances (under 5 percent CPU with near zero network).

How much does a zombie resource hunt recover?

A first hunt on an account that has never had one typically recovers 5 to 12 percent of total spend. On a $150,000 a month bill that is $7,500 to $18,000 a month for roughly two weeks of work by one engineer. A second hunt six months later finds far less, around 2 to 3 percent, which is the sign the controls are working.

How do I delete idle resources safely?

Use a five step protocol: tag every candidate with a marked-for-deletion tag and date, notify the owner or the owning account's team, wait 14 days and re-check the signal so anything that became active drops off, snapshot anything holding data (at $0.05 per GB-month that converts a data loss incident into a cheap inconvenience), then delete and record the saving. For high risk resources, stop or detach first and wait another week.

What is the risk of deleting an idle database?

Quarterly and annual workloads. A reporting database can show zero connections for 89 days and then matter enormously at quarter end. Zero connections over 30 days is a strong signal, not proof, so always confirm with an owner before deleting anything holding data, and take a snapshot first so restoration is possible if the assessment was wrong.

How do I stop zombies coming back?

Three controls. Require an owner tag at creation so every resource is attributable. Manage infrastructure with Terraform so destroying a stack removes everything it created rather than leaving disks and IPs behind. And price infrastructure changes before merge so the cost of what is being created is visible from the start. C3X handles the last one by costing Terraform changes in the pull request.

What to do next

Stop creating the next generation of zombies. C3X prices Terraform changes against a live resource catalog before merge. Start with the quickstart.

Try C3X on your own Terraform

Free and open source. No API key required. One command to install, one command to estimate.