networkingdata-platformcost-optimizationegress

Data transfer cost between analytics services: the line nobody budgets for

Moving data between a lake, a warehouse, a stream, and a BI tool crosses availability zones, regions, and account boundaries. Each crossing has a price, and at analytics volumes they add up fast.

The C3X Team··8 min read

Quick answer

Analytics pipelines move terabytes, and every boundary crossing has a rate. Cross availability zone transfer is $0.01 per GB each direction, cross region within the US around $0.02 per GB, and internet egress $0.09 per GB for the first 10 TB. NAT gateway data processing adds $0.045 per GB on top of the $0.045 hourly charge. A pipeline shipping 100 TB a month through a NAT gateway to a service in another region can accrue $6,500 a month in transfer charges alone. The fixes are VPC endpoints, co locating services in one region and zone, and compressing before any boundary crossing.

Transfer charges are the most under modelled part of a data platform budget. Compute and storage appear in every architecture discussion; the network between them appears in none, right up until the invoice arrives with a data transfer line larger than the warehouse. At analytics volumes, where a single pipeline may move tens of terabytes a month, even a one cent per GB rate is material.

The rate card

PathApproximate rateCost of 100 TB
Within one availability zone, private IPFree$0
Cross AZ, same region$0.01/GB each direction$1,000 to $2,000
Cross region, US to US~$0.02/GB~$2,000
Internet egress, first 10 TB$0.09/GB~$8,700
NAT gateway data processing$0.045/GB~$4,500
Object storage to service, same regionFree$0
VPC gateway endpoint to object storageFree$0
VPC interface endpoint~$0.01/GB plus ~$0.01/hr per AZ~$1,000 plus ~$22

The two free rows are the important ones. Reading from object storage to a compute service in the same region costs nothing for transfer, which is why keeping the lake and the query engine in one region is the single biggest structural saving available.

The NAT gateway trap

A compute node in a private subnet reading from object storage without a gateway endpoint routes through the NAT gateway, and every byte is charged $0.045 for data processing. A Spark job reading 40 TB a month from a bucket in the same region pays $1,800 for traffic that should have been free. Creating a VPC gateway endpoint for object storage takes minutes, costs nothing, and eliminates the charge entirely. This is probably the most common avoidable cost in data platform networking, and it is invisible unless you break the bill down by usage type rather than by service.

Interface endpoints for other services cost about $0.01 per GB plus roughly $0.01 per hour per availability zone, around $22 a month for three zones. Still far cheaper than $0.045 per GB through NAT for any meaningful volume: the break even is around 630 GB a month per endpoint.

Cross zone charges inside a cluster

Distributed systems shuffle. A Spark job redistributing 20 TB during a join, or a Kafka cluster replicating across three zones, generates internal traffic that crosses zone boundaries at $0.01 per GB each way. A broker cluster ingesting 10 TB a month with replication factor 3 spread across zones moves roughly 20 TB across zones for replication, about $400 a month, plus consumer fetches from other zones. Rack aware consumer fetching, where consumers read from the replica in their own zone, eliminates a large share of that. For Spark, co locating executors in one zone removes shuffle transfer charges at the cost of some availability, which is usually the right trade for batch jobs that can simply be rerun.

Cross region replication

ScenarioMonthly volumeTransfer cost
Lake replicated to a DR region50 TB~$1,000
Warehouse in a different region from the lake30 TB scanned~$600
BI tool hosted outside the cloud5 TB extracted~$450 at internet rates
Stream consumers in a second region20 TB~$400

The warehouse row is the one worth scrutinizing. A query engine in us-east-1 reading a bucket in us-west-2 pays cross region transfer on every scan, forever. It is not a one time migration cost; it is a recurring tax proportional to query volume. Moving the compute to the data, or the data to the compute, eliminates it permanently.

Compress before crossing

Every transfer rate is per GB, so compression is a direct multiplier on the bill. Moving 50 TB a month across regions at $0.02 per GB costs $1,000 uncompressed and about $275 with ZSTD at typical ratios. Seecodec trade offs for the detail. Combined with columnar formats that reduce what needs moving in the first place, transfer volume frequently drops by 80 percent or more.

A checklist

Create gateway endpoints for object storage in every VPC. Audit NAT gateway data processing charges monthly and trace anything large. Keep lake, warehouse, and query compute in one region. Use zone aware replica fetching on streaming clusters. Compress everything crossing a boundary. Break the bill down by usage type so transfer is visible rather than buried. All of these are Terraform decisions, endpoints, subnets, regions, replication configuration, and C3X prices them from the resource catalog in the pull request, before the traffic starts flowing.

FAQ

How much does data transfer cost between analytics services?

Cross availability zone transfer is $0.01 per GB in each direction, cross region within the US around $0.02 per GB, internet egress $0.09 per GB for the first 10 TB, and NAT gateway data processing $0.045 per GB. Transfer within one zone on private IPs, and from object storage to compute in the same region, is free.

What is the NAT gateway trap in data pipelines?

Compute in a private subnet reading from object storage without a VPC gateway endpoint routes through the NAT gateway, paying $0.045 per GB for traffic that should be free. A job reading 40 TB a month pays $1,800 needlessly. Creating a gateway endpoint takes minutes, costs nothing, and removes the charge entirely.

Are VPC interface endpoints worth the cost?

For any meaningful volume, yes. They cost about $0.01 per GB plus roughly $0.01 per hour per availability zone, around $22 a month for three zones, against $0.045 per GB through a NAT gateway. The break even is roughly 630 GB a month per endpoint, which most analytics workloads exceed easily.

How much do cross zone charges cost inside a cluster?

More than expected. A broker cluster ingesting 10 TB a month with replication factor three spread across zones moves roughly 20 TB across zone boundaries, about $400 a month at $0.01 per GB, plus consumer fetch traffic. Zone aware replica fetching, where consumers read from their local replica, removes a large share of it.

Why is a warehouse in a different region from the lake expensive?

Because cross region transfer is charged on every scan, forever. A query engine in one region reading a bucket in another pays roughly $0.02 per GB continuously, about $600 a month on 30 TB of scanning. It is a recurring tax proportional to query volume, not a one time migration cost, so co locating them removes it permanently.

How does C3X help with transfer cost?

Endpoints, subnets, region placement, NAT gateways, and replication configuration are all Terraform. C3X prices them against a live catalog in the pull request, so a design that routes analytics traffic through a NAT gateway or places compute in a different region from its data is visible as a cost before the traffic ever flows.

What to do next

Catch expensive data paths at design time. C3X prices your Terraform resources in the pull request. Start with the quickstart.

Try C3X on your own Terraform

Free and open source. No API key required. One command to install, one command to estimate.