AI workload data transfer cost: the line item nobody forecasts

Quick answer

AI workloads move far more data than typical applications, and cloud networking charges per GB. Cross-AZ traffic costs about $0.01 per GB in each direction, cross-region about $0.02 per GB, and internet egress about $0.09 per GB for the first 10 TB. A distributed training job pushing 50 TB of gradient traffic across zones costs roughly $1,000 in transfer alone. Pulling a 20 TB dataset from another region costs about $400 per run. The fixes are keeping GPUs, data, and storage in one availability zone, caching datasets locally, and using VPC endpoints for object storage.

Data transfer is the classic invisible cloud cost, and AI workloads are the worst offenders because they move data by the terabyte as a matter of routine. Training reads the dataset repeatedly, distributed jobs exchange gradients on every step, checkpoints are written constantly, and inference services ship embeddings and documents between components. Every one of those hops may be metered.

The rates that apply

Path	Typical AWS rate	Notes
Within one AZ, private IP	free	The configuration to aim for
Cross-AZ, same region	about $0.01/GB each way	So about $0.02/GB round trip
Cross-region	about $0.02/GB	Varies by region pair
Internet egress	about $0.09/GB first 10 TB	Tiered down at volume
S3 to EC2, same region	free	With a gateway VPC endpoint
NAT gateway processing	about $0.045/GB	Plus about $0.045/hr per gateway

Full background on the AWS model is in AWS data transfer costs explained. GCP and Azure have similar structures with different numbers.

Where AI workloads generate volume

Dataset reads. A 20 TB dataset read once per epoch over 5 epochs is 100 TB of reads. If storage and compute are in the same region and you use a gateway endpoint, that is free. If the bucket is in another region, it is about $2,000. If the traffic routes through a NAT gateway instead of an endpoint, add $0.045 per GB, about $4,500 for the same 100 TB. The difference between the best and worst configuration here is thousands of dollars for identical work.

Distributed training gradients. All-reduce traffic scales with model size and step count. A 7B model in FP16 exchanges roughly 14 GB per all-reduce; at 20,000 steps across 8 workers that is a very large number, which is exactly why multi-GPU instances with NVLink and single-placement-group clusters exist. Keep distributed workers in one availability zone and inside a cluster placement group, and this traffic is free. Spread them across zones for availability and you pay $0.01 per GB each way on every step.

Checkpoint writes. A 70B model checkpoint near 800 GB, written every 30 minutes over a 48-hour run, is about 77 TB of writes. Same-region writes to object storage are free of transfer charges but do incur PUT request fees and storage. Cross-region replication of that data at $0.02 per GB would be about $1,540 per run.

Inference serving. Each response is small, but volume is enormous. Ten million responses a day at 4 KB each is 40 GB a day, about 1.2 TB a month, roughly $108 at internet egress rates. Streaming responses and retrieved documents can multiply that by ten.

A worked monthly bill

Component	Volume	Rate	Monthly
Cross-AZ gradient traffic	50 TB	$0.01/GB each way	about $1,000
Cross-region dataset pulls	30 TB	$0.02/GB	about $600
NAT-routed S3 reads	15 TB	$0.045/GB	about $675
Inference egress	3 TB	$0.09/GB	about $270
Total			about $2,545

Three of those four lines are avoidable with configuration changes alone, taking the bill to roughly $270.

The fixes, ranked

Add a gateway VPC endpoint for S3 and DynamoDB. It costs nothing and removes NAT processing charges on object storage traffic entirely, which is often the single biggest avoidable line. Keep training data in the same region as the GPUs, and copy once rather than reading cross-region repeatedly. Place distributed training workers in one availability zone and one cluster placement group. Cache the dataset on the instance's included local NVMe on the first epoch so subsequent epochs read locally at zero cost. Put a CDN in front of high-volume inference responses if they are cacheable. And compress payloads: gzip on JSON responses routinely cuts egress volume by 70 percent, which is a 70 percent cut on that line.

Multi-cloud and hybrid make it worse

Teams that train in one cloud and serve in another, or that pull data from an on-premises store, pay egress at the full internet rate on every byte crossing the boundary. Moving a 20 TB dataset out of a cloud at about $0.09 per GB costs roughly $1,843 per copy, and doing it monthly because the pipeline re-exports rather than syncing incrementally turns that into a $22,000 annual line for data that barely changed. Incremental sync, one-way replication with lifecycle cleanup, and doing the compute where the data already lives are all cheaper than moving terabytes repeatedly. If a workload genuinely must span providers, put the heavy data movement on a one-time migration and keep the ongoing traffic to metadata and results.

Design it in, do not discover it

Transfer cost is a property of topology, and topology is defined in Terraform. Subnet placement, endpoint configuration, bucket region, and NAT routing are all code, which means they can be reviewed and priced before they run. Price the network layout against the resource catalog so a bucket placed in the wrong region is caught in a pull request rather than in a quarterly bill review.

FAQ

Why is data transfer expensive for AI workloads?

Because AI moves data by the terabyte as routine work. Training reads the dataset once per epoch, distributed jobs exchange gradients every step, checkpoints are written constantly, and inference ships documents and embeddings between services. At about $0.01 per GB cross-AZ, $0.02 cross-region, and $0.09 for internet egress, terabyte-scale movement becomes thousands of dollars quickly.

How do I avoid cross-AZ charges in distributed training?

Place all distributed workers in a single availability zone and, on AWS, inside a cluster placement group. Traffic between instances in the same AZ over private IPs is free, while cross-AZ traffic costs about $0.01 per GB in each direction, which all-reduce gradient exchange incurs on every step. Single-AZ placement trades some availability for a large transfer saving on training jobs.

Does reading training data from S3 cost anything?

Within the same region and through a gateway VPC endpoint, no transfer charge applies, only GET request fees. If the traffic routes through a NAT gateway instead, you pay about $0.045 per GB of processing, which on 100 TB of reads is about $4,500. If the bucket is in another region, cross-region transfer at about $0.02 per GB adds roughly $2,000 for the same volume.

How much does inference egress cost?

Ten million responses a day at 4 KB each is about 40 GB a day, roughly 1.2 TB a month, or about $108 at the $0.09 per GB internet egress rate. Streaming responses, returned source documents in a retrieval system, or image outputs can multiply that tenfold. Gzip compression on JSON responses typically cuts the volume by around 70 percent.

What is the single biggest transfer saving for ML workloads?

Adding a gateway VPC endpoint for object storage. It costs nothing and eliminates NAT gateway processing charges of about $0.045 per GB on all S3 traffic, which for a training pipeline reading tens of terabytes is often the largest avoidable line on the bill. Second is keeping datasets and GPUs in the same region so no cross-region charge applies.

How does C3X help with AI data transfer cost?

Transfer cost is a property of topology, and topology is defined in Terraform: subnet placement, VPC endpoints, bucket regions, and NAT routing are all code. C3X prices that infrastructure against a live catalog before it deploys, so a bucket in the wrong region or a missing gateway endpoint is caught in the pull request rather than in a quarterly bill review.

What to do next

Catch transfer cost in the pull request. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.