Why Kubernetes nodes never scale down, and the six settings that fix it

Quick answer

Scale-up is easy because a pending pod is an obvious signal. Scale-down requires proving a node can be emptied safely, and a single blocking pod keeps a $280 node alive indefinitely. The usual culprits are pods without controllers, restrictive PodDisruptionBudgets, local storage, DaemonSets counted as blockers, the safe-to-evict annotation set to false, and a utilization threshold set too low. Tuning the scale-down utilization threshold from 0.5 to 0.65, shortening the unneeded time from 10 minutes to 5, and clearing blocking pods typically removes 20 to 35 percent of nodes from a cluster whose workload has not changed at all.

Every cluster autoscaler is enthusiastic about growing and reluctant about shrinking, by design. Scaling up wrongly costs you a little money. Scaling down wrongly costs you an outage. The result is a ratchet: node count climbs during every traffic peak and never fully returns.

How the decision is made

The classic Cluster Autoscaler evaluates each node against two questions. Is its resource utilization, measured as the sum of pod requests divided by allocatable capacity, below the scale-down threshold, default 0.5? And have all its pods been reschedulable elsewhere continuously for the unneeded period, default 10 minutes? Only if both hold does it cordon, drain, and terminate.

Note that it uses requests, not usage. A node whose pods request 60 percent and use 12 percent is not a candidate. That is why right-sizing requests is a prerequisite for scale-down working at all, not an unrelated optimization.

The six blockers

Blocker	Symptom	Fix
Pods with no controller	A bare Pod pins the node forever	Wrap in a Deployment or Job
Restrictive PodDisruptionBudget	minAvailable equals replicas	Allow at least one disruption
Local storage, emptyDir or hostPath	Autoscaler refuses to evict	Annotate safe-to-evict true
safe-to-evict false	Set once for a migration, never removed	Audit the annotation quarterly
kube-system pods on the node	Treated as non-evictable by default	Give them PDBs or skip the check
Node affinity pinning	Pod cannot move anywhere else	Loosen to preferred

The PodDisruptionBudget case is the most common and the most quietly expensive. A team sets minAvailable to 3 on a Deployment with exactly 3 replicas, meaning zero voluntary disruptions are ever allowed. Those three pods can now each hold a separate node open permanently: $840 per month for a correctness setting that was meant to protect availability during deploys.

The six settings to tune

Setting	Default	Suggested	Effect
scale-down-utilization-threshold	0.5	0.6 to 0.7	More nodes qualify
scale-down-unneeded-time	10m	5m	Faster reclaim
scale-down-delay-after-add	10m	5 to 10m	Avoid thrash after scale-up
max-empty-bulk-delete	10	20 to 30	Drain a big peak faster
skip-nodes-with-local-storage	true	false, with care	Unblocks emptyDir pods
skip-nodes-with-system-pods	true	false, with PDBs	Unblocks kube-system nodes

Raising the utilization threshold is the highest-impact change. At 0.5, a node whose pods request 55 percent of its capacity is considered busy. At 0.65, it becomes a candidate, and its pods usually fit elsewhere. On a 60-node cluster with requests averaging 58 percent, that one change commonly removes 10 to 14 nodes, $2,800 to $3,900 per month.

The two skip flags need judgment. Turning off skip-nodes-with-local-storage is safe when emptyDir is used as scratch space and unsafe when something treats it as durable. Turning off skip-nodes-with-system-pods requires that the kube-system workloads have PodDisruptionBudgets so they move rather than being killed.

Do not raise the utilization threshold past about 0.7 without watching what happens. Above that point the autoscaler starts removing nodes whose pods only just fit elsewhere, which produces churn: a node is drained, the pods land on other nodes, those nodes tip over the scale-up trigger, and a new node appears. Each cycle costs a few minutes of degraded capacity and, on providers with per-second billing after a one-minute minimum, a small amount of duplicated spend. A threshold of 0.6 to 0.65 with a five-minute unneeded time is the range where most clusters get the savings without the oscillation.

Consolidation beats thresholds

Threshold tuning finds nodes that are already nearly empty. It does not actively rearrange pods to create empty nodes. Karpenter-style consolidation does: it looks for nodes that could be replaced by a smaller node or removed entirely by moving their pods, and it acts. That behavioural difference is the main cost argument in theKarpenter versus Cluster Autoscalercomparison, and it typically finds another 10 to 20 percent beyond good threshold tuning.

Diagnosing your own cluster

The autoscaler writes its reasoning to a status ConfigMap and to events. For every node it declined to remove, it names the pod that blocked it. Pull that list, group by pod owner, and you will usually find that 5 to 10 workloads account for most blocked nodes. Fixing those specific workloads, rather than tuning globally, gets most of the benefit with the least risk.

Track node count against total pod requests weekly. If requests fall and node count does not, scale-down is blocked and you have a list to work through. Price the node group against theresource catalog so each blocked node carries a dollar figure, which turns an abstract autoscaler complaint into a prioritized list worth $280 a line.

FAQ

Why do my Kubernetes nodes never scale down?

Usually a blocking pod. The autoscaler will only remove a node if every pod on it can be rescheduled elsewhere, so a bare Pod with no controller, a PodDisruptionBudget allowing zero disruptions, a pod with local emptyDir or hostPath storage, a safe-to-evict false annotation, a kube-system pod, or node affinity pinning will each keep a node alive indefinitely at around $280 per month.

What is the scale-down utilization threshold and what should it be?

It is the ratio of pod requests to allocatable capacity below which a node becomes a removal candidate, defaulting to 0.5. Raising it to 0.6 or 0.65 makes more nodes eligible, and since their pods usually fit elsewhere, it commonly removes 10 to 14 nodes from a 60-node cluster averaging 58 percent requests, worth $2,800 to $3,900 per month.

How do PodDisruptionBudgets block Kubernetes scale-down?

A PDB with minAvailable equal to the replica count allows zero voluntary disruptions, so the autoscaler can never evict those pods. Three such pods spread across three nodes hold all three open permanently, roughly $840 per month. Setting minAvailable to replicas minus one, or using maxUnavailable of 1, preserves the availability intent while allowing consolidation.

Does the cluster autoscaler use requests or actual usage?

Requests. A node whose pods request 60 percent of allocatable capacity but use only 12 percent is not considered under-utilized and will not be removed. That makes right-sizing resource requests a prerequisite for scale-down working at all, not an independent optimization, since inflated requests make every node look busy.

How do I find which pods are blocking Kubernetes scale-down?

The cluster autoscaler writes its reasoning to a status ConfigMap and to Kubernetes events, naming the specific pod that blocked each node removal. Pull that list and group by pod owner. In most clusters five to ten workloads account for the majority of blocked nodes, so fixing those specific workloads gets most of the benefit with minimal risk.

How does C3X help with Kubernetes scale-down?

C3X prices node groups from Terraform against a live catalog, so each node that fails to scale down carries a concrete monthly figure. That turns an abstract autoscaler complaint into a prioritized list worth roughly $280 per line, and it verifies that threshold tuning or unblocking work actually reduced the provisioned node count.

What to do next

Give every node that will not scale down a price tag. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.