kubernetesnetworkingcost-optimizationaws

Kubernetes cross-zone traffic cost: the tax on spreading pods

Spreading pods across availability zones buys resilience and costs bandwidth. Kubernetes load balances without regard to zone by default, so most of your service traffic is billed. Here is how to fix it.

The C3X Team··7 min read

Quick answer

Kubernetes services load balance across all healthy endpoints regardless of zone, so in a three-zone cluster roughly two thirds of pod-to-pod traffic crosses a zone boundary and is billed at 0.01 dollars per GB in each direction on AWS. A cluster moving 100 TB per month internally pays about 1,340 dollars in cross-zone charges under random routing. Enabling topology aware routing, or the newer traffic distribution field set to prefer same zone, typically cuts that to under 300 dollars. The trade-off is that zone-local routing reduces spare capacity per zone, so it needs enough replicas per zone to absorb failure.

Kubernetes was designed to treat a cluster as a single pool of compute. That abstraction is what makes it pleasant to use and what makes its network bill surprising. The scheduler and the service proxy are both happy to send a request from a pod in us-east-1a to a pod in us-east-1c, and the cloud provider is happy to charge you 0.01 dollars per GB for the privilege, in each direction.

Why two thirds of traffic crosses a zone

A standard ClusterIP service with kube-proxy in iptables or IPVS mode distributes connections across all ready endpoints with no topology awareness. If your deployment has nine replicas spread evenly across three zones, a calling pod in zone A has a one in three chance of hitting a same-zone endpoint. Two thirds of requests therefore cross a zone boundary. The same applies to DNS-based service discovery and to most mesh configurations unless locality load balancing is explicitly turned on.

Internal volume/monthCross-zone shareBillable GBMonthly cost
10 TB67 percent13,400 GB134 USD
50 TB67 percent67,000 GB670 USD
100 TB67 percent134,000 GB1,340 USD
100 TB15 percent30,000 GB300 USD

The billable GB column counts both directions, which is why 100 TB of cross-zone traffic at 67 percent yields 134,000 billable GB rather than 67,000.

Topology aware routing

Kubernetes offers topology aware hints, and in newer versions a trafficDistribution field on the Service object that can be set to PreferClose. When enabled, EndpointSlices carry zone hints and kube-proxy prefers endpoints in the caller's zone. The controller only applies hints when it can distribute endpoints proportionally to zone capacity, which is a safety valve: if one zone has too few endpoints, hints are withheld and traffic falls back to cluster-wide routing rather than overloading a thin zone.

In practice this means you need a reasonable number of replicas per zone for the feature to engage. A deployment with three replicas across three zones often will not get hints, because losing one leaves a zone unserved. Nine or more replicas across three zones usually does. Once engaged, the cross-zone share typically falls to between 5 and 15 percent, which on a 100 TB per month cluster takes the charge from about 1,340 dollars to about 300 dollars.

Load balancers add their own line

North-south traffic has a separate meter. A Network Load Balancer charges 0.01 dollars per GB for cross-zone traffic when cross-zone load balancing is enabled, on top of its 0.0225 dollars per hour and NLCU charges. An Application Load Balancer includes cross-zone load balancing at no extra per-GB charge, which makes ALB the cheaper choice for workloads with heavy cross-zone distribution. If you are running an NLB in front of a three-zone service, disabling cross-zone load balancing removes the charge but makes traffic distribution uneven when zones have different numbers of targets.

Stateful workloads are the expensive case

Databases and message brokers running in the cluster replicate continuously. A three-node Kafka cluster with replication factor 3 spread across zones sends every produced byte to two other zones. Ingesting 5 TB per month means 10 TB of replication traffic, of which nearly all crosses a zone, costing about 200 dollars per month in transfer on top of the compute. Kafka's rack awareness and follower fetching can cut consumer-side cross-zone reads substantially, and similar locality features exist for most distributed data systems. The broader pattern is covered in Kafka cost optimization.

Measuring what you actually move

VPC flow logs with the pkt-srcaddr and pkt-dstaddr fields let you map traffic to zones by subnet, and most mesh and CNI implementations expose per-zone byte counters directly. Start by finding the top five service pairs by volume; in almost every cluster they account for the majority of bytes. Then check which of them are genuinely zone-spread and whether the spread is buying resilience you need.

The resilience trade-off

Zone-local routing is not free of risk. If all traffic prefers local endpoints, a zone with a thin replica count can saturate while capacity sits idle elsewhere. The correct posture is enough replicas per zone to handle that zone's share plus headroom, combined with automatic fallback to cluster-wide routing when local endpoints become unhealthy. That is precisely what the PreferClose semantics provide. The goal is not to stop spreading pods; it is to stop paying for spreading them when the traffic did not need to travel. Price your cluster topology before you deploy it against the resource catalog.

FAQ

Why does Kubernetes generate so much cross-zone traffic?

A standard ClusterIP service distributes connections across all ready endpoints with no topology awareness. In a three-zone cluster with replicas spread evenly, a calling pod has only a one in three chance of reaching a same-zone endpoint, so about two thirds of requests cross a zone boundary. On AWS that traffic is billed 0.01 dollars per GB in each direction.

How much does cross-zone traffic cost in a Kubernetes cluster?

A cluster moving 100 TB per month internally with 67 percent crossing zones generates 134,000 billable GB counting both directions, costing about 1,340 dollars per month on AWS. Reducing the cross-zone share to 15 percent through topology aware routing brings that to about 300 dollars. The exact figure scales linearly with internal traffic volume.

What is topology aware routing and does it help?

It is a Kubernetes feature, exposed as topology aware hints and more recently as a trafficDistribution field set to PreferClose, that makes kube-proxy prefer endpoints in the caller's zone. It typically cuts the cross-zone share from about 67 percent to between 5 and 15 percent. The controller withholds hints when endpoints cannot be distributed proportionally, so thin deployments may not benefit.

Does the load balancer choice affect cross-zone cost?

Yes. A Network Load Balancer charges 0.01 dollars per GB for cross-zone traffic when cross-zone load balancing is enabled, on top of hourly and capacity-unit charges. An Application Load Balancer includes cross-zone load balancing with no additional per-GB charge, making it cheaper for workloads distributing heavily across zones.

Why are stateful workloads the expensive case?

They replicate continuously. A three-node Kafka cluster with replication factor 3 spread across zones sends every produced byte to two other zones, so ingesting 5 TB per month generates 10 TB of replication traffic costing roughly 200 dollars. Rack awareness and follower fetching reduce consumer-side cross-zone reads, and similar locality features exist in most distributed data systems.

How does C3X help with Kubernetes network cost?

C3X prices infrastructure from Terraform before deployment, including node groups, subnets across availability zones, load balancers, and NAT gateways. Because cross-zone charges follow from how the cluster is laid out rather than from application code, seeing those costs in the pull request is where the placement and load balancer decisions can still be changed cheaply.

What to do next

Price your cluster layout before it runs. C3X reads your Terraform and costs it against a live resource catalog. Start with the quickstart.

Try C3X on your own Terraform

Free and open source. No API key required. One command to install, one command to estimate.