Data lake storage tiering cost: moving 80 percent of your bytes for 20 percent of the price

Quick answer

Object storage tiers span more than a 20x price range: S3 Standard at $0.023 per GB per month, Standard-IA at $0.0125, Glacier Instant Retrieval at $0.004, Glacier Flexible at $0.0036, and Deep Archive at $0.00099. A 500 TB lake sitting entirely in Standard costs about $11,500 a month; a sensible lifecycle policy brings the same data to around $2,600. The catches are minimum storage durations, per object transition fees, and retrieval charges, all of which punish small files and frequently re read cold data.

Data lakes grow by accretion. Every pipeline writes, almost nothing deletes, and the default storage class is the most expensive one. Two years in, the majority of the bytes have not been read since the quarter they landed, and they are all sitting on the tier designed for data read many times a day. Fixing this is a lifecycle policy, not a migration project.

The price ladder

Tier	$/GB-month	Retrieval	Min duration	First byte latency
S3 Standard	$0.023	Free	None	Milliseconds
Standard-IA	$0.0125	$0.01/GB	30 days	Milliseconds
One Zone-IA	$0.01	$0.01/GB	30 days	Milliseconds
Glacier Instant Retrieval	$0.004	$0.03/GB	90 days	Milliseconds
Glacier Flexible Retrieval	$0.0036	$0.01/GB	90 days	Minutes to hours
Glacier Deep Archive	$0.00099	$0.02/GB	180 days	Up to 12 hours

Azure and GCP offer the same ladder with different names and similar ratios: hot, cool, cold, and archive tiers with escalating retrieval fees and minimum durations. The reasoning below transfers directly.

A 500 TB lake, two ways

Policy	Distribution	Monthly cost
Everything in Standard	500 TB hot	~$11,500
Tiered by age	50 TB Standard, 100 TB Standard-IA, 150 TB Glacier IR, 200 TB Deep Archive	~$2,620

That is a 77 percent reduction, roughly $107,000 a year, from a lifecycle configuration that takes an hour to write. The distribution above is not aggressive: it keeps 50 TB instantly available at full performance and another 250 TB available in milliseconds. Only the oldest 200 TB accepts a retrieval delay.

The three traps

Minimum storage durations. An object moved to Standard-IA and deleted after 10 days still bills 30 days. Moved to Deep Archive and deleted after a month, it bills 180 days. If your pipeline writes temporary intermediates that are cleaned up weekly, transitioning them to a colder tier costs more than leaving them in Standard. Exclude intermediate prefixes from lifecycle rules.

Transition request fees. Lifecycle transitions are charged per object, around $0.01 per 1,000 objects to IA tiers and roughly $0.05 per 1,000 to Glacier classes. A lake with 400 million small objects costs about $20,000 just to transition to Glacier once. This is the small file problem wearing a different hat and it is the main reason compaction should come before tiering.

Retrieval charges on data you still read. Glacier Instant Retrieval at $0.03 per GB means reading 10 TB of it costs $300. If a quarterly report scans two years of history, a tier that looks cheap on storage becomes expensive on access. Model annual reads, not just storage.

Compaction first, tiering second

Small files hurt twice: transition fees scale per object, and query engines pay per file overhead on every scan. A table stored as 2 million 5 MB files instead of 50,000 200 MB files costs 40 times more to transition and typically queries 3 to 10 times slower. Run compaction so files land between 128 MB and 1 GB, then apply lifecycle rules. The combination is far better than either alone. This connects directly tofile format choices, since columnar formats compress harder and reduce the byte count before any tiering happens.

When intelligent tiering earns its fee

Automatic tiering classes move objects between access tiers based on observed usage, for a monitoring fee of about $0.0025 per 1,000 objects per month. For 10 million objects that is $25 a month, trivial against a large bill, and it removes the need to predict access patterns. But it is a bad deal for objects under 128 KB, which are not eligible for auto tiering yet still incur monitoring, and unnecessary for data with a known, boring lifecycle such as daily partitions that are never read after 30 days. Use explicit rules where the pattern is predictable and automatic tiering where it genuinely is not.

A default policy worth copying

Age	Tier
0 to 30 days	Standard
30 to 90 days	Standard-IA
90 to 365 days	Glacier Instant Retrieval
Over 365 days	Deep Archive, or delete

Add an expiration rule so nothing lives forever by accident, and expire incomplete multipart uploads after 7 days since abandoned parts bill silently. Lifecycle rules belong in Terraform next to the bucket, and C3X prices buckets and their storage classes from the resource catalog so the tiering decision is visible in the pull request that creates the lake.

FAQ

How much can storage tiering save on a data lake?

Typically 70 to 80 percent. A 500 TB lake entirely in S3 Standard costs about $11,500 a month, while tiering it into 50 TB Standard, 100 TB Standard-IA, 150 TB Glacier Instant Retrieval, and 200 TB Deep Archive brings it to roughly $2,620. That is about $107,000 a year from a lifecycle configuration that takes an hour to write.

What are the S3 storage tier prices?

Standard is $0.023 per GB per month, Standard-IA $0.0125, One Zone-IA $0.01, Glacier Instant Retrieval $0.004, Glacier Flexible Retrieval $0.0036, and Glacier Deep Archive $0.00099. Retrieval fees and minimum storage durations rise as the storage price falls, so the cheapest tier is not always the cheapest total for data you still read.

What is the minimum storage duration trap?

Colder tiers bill a minimum period regardless of when you delete. Standard-IA bills 30 days, Glacier classes 90 days, and Deep Archive 180 days. Transitioning short lived pipeline intermediates to a cold tier therefore costs more than leaving them in Standard, so exclude temporary prefixes from lifecycle rules entirely.

Do lifecycle transitions cost money?

Yes, per object. Transitions run roughly $0.01 per 1,000 objects to infrequent access tiers and about $0.05 per 1,000 to Glacier classes. A lake containing 400 million small objects costs around $20,000 just to transition to Glacier once, which is why file compaction should always precede a tiering rollout.

Should I use intelligent tiering?

Use it where access patterns are genuinely unpredictable. The monitoring fee is about $0.0025 per 1,000 objects per month, around $25 for 10 million objects, which is trivial against a large bill. Avoid it for objects under 128 KB, which pay monitoring without being eligible for auto tiering, and for data with a known, boring lifecycle.

How does C3X help with data lake storage cost?

C3X prices buckets, their storage classes, and lifecycle configuration from Terraform before deployment, using a live catalog of real prices. That makes the difference between a bucket defaulting to Standard and one with a tiering policy visible as dollars in the pull request, at the moment the lake is designed rather than two years of accretion later.

What to do next

Price your storage tiers before the bytes pile up. C3X reads Terraform and costs your resources against a live catalog. Start with the quickstart.