Data retention policy cost: what keeping everything forever actually costs

Quick answer

Retention cost compounds because data accumulates while the rate stays flat. A pipeline landing 3 TB a month costs about $69 a month in hot object storage in month one and $828 a month by month twelve if nothing expires. Over three years that steady state reaches $2,484 a month for data that is mostly never read. The fix is a tiered retention policy with an explicit expiry: keep 30 days hot, 11 months in colder tiers, and delete or deep archive beyond, which typically cuts the bill by 75 to 90 percent while satisfying most compliance requirements.

Nobody sets retention to forever on purpose. It happens because the default is no expiry, because deleting data feels irreversible, and because the person who would benefit from deleting it is not the person who has to justify it if something turns out to have been needed. The result is that storage costs rise every month without anyone making a decision, which is the definition of an unmanaged expense.

The compounding arithmetic

Month	Accumulated data	Monthly cost, all hot	Cumulative spend
1	3 TB	$69	$69
6	18 TB	$414	$1,449
12	36 TB	$828	$5,382
24	72 TB	$1,656	$20,349
36	108 TB	$2,484	$44,850

At S3 Standard rates of $0.023 per GB per month. Three years of a single modest pipeline, never expired, is roughly $45,000. The same pipeline with a tiered policy and a 12 month expiry costs about $6,000 over the same period. And that is one pipeline; a platform typically runs dozens.

Retention windows have different prices by data type

Data type	Common retention	Price sensitivity
Application logs	7 to 30 days hot, 90 days archived	Very high, huge volume
Metrics	15 days raw, 13 months downsampled	High, cardinality driven
Raw event data	13 months, then aggregate	High
Aggregated business metrics	Indefinite	Low, tiny volume
Database backups	35 days, plus monthly archives	Medium
Audit and compliance records	7 years, deep archive	Low per GB, deep archive is cheap

The key insight is that the data with genuine long retention requirements, audit records and aggregated metrics, is almost always small. Audit logs at 50 GB a year in Deep Archive cost about $0.60 a month per year of history. Seven years of them is around $4.20 a month. Meanwhile the raw application logs nobody has read since the incident they were generated for occupy terabytes. Long retention is not the expensive part; long retention of high volume data in the hot tier is.

Downsampling beats deletion arguments

When a stakeholder insists on keeping years of history, offer aggregation rather than deletion. Raw event rows at one second granularity compressed to hourly aggregates typically shrink by 99 percent or more while preserving nearly every question anyone actually asks of old data. A metrics store keeping 15 days at full resolution, 90 days at five minute resolution, and 13 months at hourly resolution holds a small fraction of the bytes of a full resolution 13 month store, at a small fraction of the cost. This converts an unwinnable conversation about risk into an easy one about precision.

Where retention costs hide

Retention also interacts with per request charges. Lifecycle transitions and expirations are billed per object, so a bucket holding 200 million small files costs real money simply to expire. Compacting before applying an aggressive policy avoids paying a four figure one off fee to delete data you no longer want.

Beyond object storage, retention accrues in places people forget. Warehouse time travel and fail safe windows multiply table storage, and a 90 day time travel setting on a heavily updated table can hold several times the live data size. Snapshot and backup chains grow with every incremental. Log ingestion services often charge both for ingestion and for retention beyond a free window, so a 2 TB a month log volume kept 12 months carries an ongoing storage line long after ingestion was paid. Search cluster indices held on hot nodes are the most expensive per GB of all, which is whyindex lifecycle tiering matters so much.

Writing a policy that survives review

Tier	Window	Storage class	Cost per TB-month
Hot	0 to 30 days	Standard	~$23
Warm	30 to 90 days	Standard-IA	~$12.50
Cold	90 to 365 days	Glacier Instant Retrieval	~$4
Archive	1 to 7 years, compliance only	Deep Archive	~$1
Expired	Beyond policy	Deleted	$0

Every dataset gets an owner, a tier schedule, and an expiry date, written in the Terraform that creates the bucket rather than in a wiki page. Make the default a finite window so new pipelines inherit sanity, and require an explicit override with a named justification for anything indefinite. Review annually and check the actual access logs: data with zero reads in six months belongs a tier down, whatever anyone claimed when the policy was written. C3X prices buckets, storage classes, and lifecycle configuration from theresource catalog so the cost of a retention choice is visible at the moment it is made.

FAQ

How much does indefinite data retention cost?

It compounds. A pipeline landing 3 TB a month in hot object storage costs about $69 in month one, $828 by month twelve, and $2,484 by month thirty six, totalling roughly $45,000 over three years. The same pipeline with tiered storage and a twelve month expiry costs around $6,000 over the same period.

Is long retention always expensive?

No. Long retention of high volume data in the hot tier is expensive. The data with genuine multi year requirements, audit records and aggregated business metrics, is usually tiny: 50 GB of audit logs a year in Deep Archive costs about $0.60 a month, so seven years of history is roughly $4.20 a month.

What is downsampling and why does it help?

Downsampling replaces fine grained raw records with coarser aggregates as data ages, for example collapsing per second events into hourly rollups. It typically shrinks volume by 99 percent or more while preserving nearly every question asked of old data, which turns a difficult argument about deleting history into an easy one about precision.

Where does retention cost hide besides object storage?

Warehouse time travel and fail safe windows multiply table storage, sometimes several times over on heavily updated tables. Snapshot and backup chains grow with each incremental. Log services often charge for retention beyond a free window in addition to ingestion. Search cluster indices held on hot nodes are the most expensive per GB of all.

What is a sensible default retention policy?

Thirty days in the hot tier, thirty to ninety days in infrequent access, ninety days to a year in instant retrieval archive, one to seven years in deep archive for compliance only, and deletion beyond that. Make a finite window the default so new pipelines inherit it, and require a named justification for anything indefinite.

How does C3X help with retention cost?

C3X prices buckets, their storage classes, and lifecycle configuration from Terraform before deployment, against a live catalog of real prices. That means the difference between a bucket with no expiry and one with a tiered policy shows up as a dollar figure in the pull request, at the moment the choice is actually being made.

What to do next

Make retention choices with prices attached. C3X reads Terraform and costs your resources in review. Start with the quickstart.