Log volume reduction: twelve tactics ranked by savings per hour of work

Quick answer

The fastest wins are dropping health check and load balancer probe lines (often 20% to 40% of access log volume), raising the default log level from DEBUG to INFO in production (frequently 50% to 70% of application log bytes), and sampling successful requests while keeping all errors. At CloudWatch's $0.50 per GB, a service ingesting 100 GB/day costs $1,500 per month; a combined 60% reduction saves $900 per month for roughly two days of work. Drop at the agent, before ingestion, not with a retention policy afterwards, because ingestion is 16x the price of a month of storage.

Log volume reduction is where observability optimization pays best, because ingestion pricing dominates every other meter. At $0.50 per GB on CloudWatch, $2.76 per GB on Azure Monitor analytics logs, and $0.50 per GiB on Google Cloud Logging, every gigabyte you do not send saves real money immediately and forever. Here are the tactics, ordered by savings per hour of engineering effort.

The ranking

Tactic	Typical reduction	Effort
Drop health check / probe lines	20% to 40% of access logs	Hours
DEBUG to INFO in production	50% to 70% of app logs	Hours
Sample successful requests	30% to 60% of access logs	1 day
Drop noisy known-benign patterns	5% to 20%	1 day
Route verbose logs to a cheap tier	50% price cut on routed volume	1 day
Trim redundant fields at the agent	10% to 25% of bytes	2 days
Compress before forwarding	Varies by contract	2 days
Replace log-based metrics with real metrics	10% to 30%	1 week
Deduplicate stack traces	5% to 15%	1 week
Aggregate repeated lines at source	5% to 20%	1 week
Convert to structured logs, drop prose	10% to 30%	Weeks
Remove instrumented-but-unused log points	5% to 15%	Ongoing

The top three, in detail

Health check noise is almost always the single biggest free win. A load balancer probing a target every 10 seconds generates 8,640 requests per day per target, and with 60 targets that is 518,400 access log lines per day that describe nothing. At 400 bytes per line that is 207 MB/day, 6.2 GB/month, $3.11 on CloudWatch. Small. But the same pattern applies to Kubernetes liveness and readiness probes at 10-second intervals across every pod, service mesh sidecar health endpoints, and internal monitoring scrapes. On a 1,000-pod cluster with three probe types, the combined volume routinely reaches 15% to 30% of all access log bytes. Exclude the probe user agent or path at the log agent, not downstream.

Log level is the second. Shipping a service with DEBUG enabled in production is extremely common, usually because someone turned it on during an incident and never turned it back off. DEBUG output in a typical service is five to fifteen times the volume of INFO. If a service ingests 30 GB/day at DEBUG, it likely ingests 3 to 6 GB/day at INFO, a saving of roughly $360 to $405 per month on CloudWatch pricing for one service. Audit every service's effective level, and make the level a runtime configuration so raising it for an incident does not require a deploy and does not stay raised.

Request sampling is third because it takes slightly more care. The rule is asymmetric: keep 100% of non-2xx responses, 100% of requests over a latency threshold, and sample 2xx fast responses at 1% to 10%. On an access log where 99.5% of lines are uneventful 200s, 5% sampling on that portion cuts volume by about 94.5%. You keep every error, every slow request, and a statistically valid sample of the rest, which is sufficient for traffic analysis if you scale counts by the sampling rate.

Where to apply the filter

This matters more than which filter you apply. Dropping at the agent (Fluent Bit, Vector, the OpenTelemetry Collector) means the bytes never reach the ingestion meter. Dropping with a retention policy means you paid full ingestion price and then deleted the data, saving only the $0.03 per GB-month storage component. The price ratio between those two outcomes is 16 to 1 on CloudWatch.

In practice this means filter rules belong in the collector config that ships with your infrastructure, version controlled and reviewed. A Fluent Bit filter excluding paths matching /healthz, /readyz, and /metrics is four lines of config and it is the highest return on four lines of config in the whole observability domain. The same applies to CloudWatch-specific tuning.

The tactics that surprise people

Field trimming is undervalued. A structured log line carrying the full HTTP request headers, the complete environment, and a duplicated trace context can be 2 KB when the useful payload is 300 bytes. Dropping headers you never query, collapsing nested objects, and removing fields that duplicate data already in the log group name or stream name routinely cuts 15% to 25% of bytes with zero information loss. Measure your p50 log line size; if it is over 1 KB, there is fat.

Replacing log-based metrics with real metrics is slower but permanent. If a service logs a line purely so a metric filter can count it, you are paying $0.50 per GB to transport a number that a counter would express in a few bytes. Emitting a proper metric instead removes the log line entirely. The caveat is cardinality: do not turn a high-cardinality log into a high-cardinality metric, because you will trade a per-GB charge for a per-series charge that is worse.

Making it stick

Volume reduction regresses. New services ship with default configs, someone enables DEBUG during an incident, a library starts logging every request. Put a per-service ingestion metric on a dashboard, alert when any service's daily volume grows more than 50% week over week, and review the top five log producers monthly. Then price the log groups, filters, and forwarding infrastructure from Terraform against the resource catalog so new services arrive with a cost attached to their logging configuration.

FAQ

What is the fastest way to cut log volume?

Drop health check and probe lines at the log agent. Load balancer health checks, Kubernetes liveness and readiness probes, service mesh sidecar checks, and monitoring scrapes routinely make up 15% to 40% of access log volume on a busy cluster and describe nothing useful. A four-line Fluent Bit filter excluding paths like /healthz, /readyz, and /metrics is the highest-return configuration change available in observability.

How much does DEBUG logging cost in production?

DEBUG output is typically five to fifteen times the volume of INFO. A service ingesting 30 GB/day at DEBUG would ingest roughly 3 to 6 GB/day at INFO, which at CloudWatch's $0.50 per GB is a saving of about $360 to $405 per month for a single service. Debug levels are usually left on accidentally after an incident, so auditing every service's effective level is a quick, repeatable win.

Should I drop logs at the agent or use a shorter retention policy?

At the agent, always. Dropping at the agent means the bytes never hit the ingestion meter at $0.50 per GB. Shortening retention means you already paid full ingestion price and are only saving the $0.03 per GB-month storage component, a 16 to 1 difference on CloudWatch. Retention policies are for data you genuinely needed to ingest; agent filters are for data you never needed at all.

How should I sample access logs without losing signal?

Sample asymmetrically: keep 100% of non-2xx responses, 100% of requests exceeding a latency threshold, and sample fast 2xx responses at 1% to 10%. On a log where 99.5% of lines are uneventful successes, 5% sampling on that portion cuts total volume by roughly 94.5% while preserving every error and slow request. Scale counts by the sampling rate when doing traffic analysis and the statistics remain valid.

Does trimming log fields save meaningful money?

Yes, usually 10% to 25% of bytes. Structured log lines often carry full HTTP headers, complete environment blocks, and duplicated trace context, reaching 2 KB when the useful payload is 300 bytes. Dropping headers nobody queries, collapsing nested objects, and removing fields duplicated in the log group or stream name cuts volume with no information loss. If your median log line exceeds 1 KB, there is fat to remove.

How does C3X help keep log volume under control?

Log groups, agent configurations delivered through launch templates, subscription filters, and forwarding infrastructure are Terraform resources. C3X prices them from the plan, so a new service adding a log group with default settings gets a cost estimate in the pull request. That makes logging configuration a reviewed decision with a number attached, which is how volume reduction stops regressing every time a new service ships.

What to do next

Catch expensive logging configuration in review. C3X reads your Terraform and prices log groups and pipelines against a live catalog. Start with the quickstart.