Parquet vs CSV cost: the file format decision that changes your query bill by 10x

Quick answer

Parquet stores data by column with per column compression and embedded statistics. Against raw CSV it typically achieves 4x to 10x smaller files and lets query engines read only the columns and row groups they need. On scan priced engines at $5 per TB, a query selecting 3 columns from a 1 TB CSV table scans the full 1 TB for $5; the same query on Parquet might scan 15 GB for about $0.08. Storage falls too, from $23 per TB per month to roughly $3 to $6 for the same logical data. The conversion cost is one compaction job.

There are very few decisions in data engineering that reduce a bill by an order of magnitude and cost a single afternoon to implement. File format is one of them. Teams still land raw CSV or JSON into a lake and query it directly for months, paying for every byte of every row on every query, when a columnar format would charge them for the columns they actually asked for.

Why columnar changes the arithmetic

A row oriented file like CSV interleaves every column of every record. To read one column, the engine must read the whole file. A columnar file stores each column contiguously, compresses each one with a codec suited to its data, and records min and max statistics per row group. The engine reads only the column chunks the query references and skips row groups whose statistics rule them out.

Property	CSV	Parquet
Layout	Row oriented, text	Column oriented, binary
Typical compression ratio	1x raw, ~3x gzipped	4x to 10x
Column pruning	No	Yes
Predicate pushdown	No	Yes, via row group statistics
Splittable when compressed	Not with gzip	Yes
Schema in file	No	Yes, typed

The numbers on a real table

Take an events table with 40 columns and 2 billion rows, roughly 1 TB as raw CSV. Converted to Parquet with Snappy compression it lands at about 180 GB. A typical analytical query touches 3 of the 40 columns and filters to one month out of twelve.

Format	Stored size	Bytes scanned by the query	Cost at $5/TB
CSV, uncompressed	1,000 GB	1,000 GB	$5.00
CSV, gzipped	330 GB	330 GB	$1.65
Parquet, Snappy	180 GB	~13 GB	$0.07
Parquet, Snappy, partitioned by month	180 GB	~1.1 GB	$0.006

Run that query 5,000 times a month and the CSV version costs $25,000 while the partitioned Parquet version costs $30. This is not a marginal tuning gain; it is a different business. Storage follows the same direction: 1,000 GB at $0.023 per GB is $23 a month against 180 GB at about $4.14.

Compression codec choice

Codec	Relative size	Decompression speed	Use when
Snappy	Baseline	Fastest	Default for hot query data
ZSTD	~15 to 25% smaller	Fast	Best general trade off, warm and cold data
Gzip	~10 to 20% smaller than Snappy	Slow	Archive only
Uncompressed	3 to 5x larger	Instant	Almost never

Note that the codec is recorded per column chunk, so you can change it on new data without rewriting history. Run a comparison on one partition before committing: write the same day of data with two codecs, compare resulting bytes and query runtime, then pick on evidence. That experiment costs a few dollars of compute and settles a decision worth thousands a year.

ZSTD is usually the right default now: it compresses meaningfully better than Snappy at comparable decompression speed, which lowers both storage and scan bytes. On the 180 GB table above, ZSTD might bring it to 145 GB, cutting another 20 percent off both meters. Seecodec trade offs for the detailed comparison.

Row group and file sizing

Parquet only prunes effectively if row groups are large enough to carry useful statistics and files are large enough to avoid per file overhead. Aim for 128 MB to 1 GB files and 128 MB row groups. A table written as 500,000 tiny Parquet files performs worse than well sized CSV, because the engine spends its time opening files rather than reading data, and listing costs add up at $0.005 per 1,000 LIST requests plus $0.0004 per 1,000 GETs. Sort data on the column you filter most often before writing, so row group min and max values are narrow and pruning actually eliminates blocks.

What conversion costs

One practical warning: Parquet's benefits assume the query engine can push predicates down. Wrapping a filter column in a function, casting it, or comparing it against a differently typed literal can defeat row group skipping entirely and quietly return you to full scan pricing. Check bytes scanned after any query rewrite rather than assuming the format is doing its job.

A one off conversion of 1 TB of CSV to Parquet on a managed ETL service at around $0.44 per DPU hour, using 10 DPUs for 30 minutes, costs about $2.20. On a Spark cluster it is similar. Against the savings above, the payback period is measured in hours. Make it permanent by writing Parquet at ingestion rather than converting later, and keep the raw CSV only if you have a compliance reason, tiered to cold storage rather than sitting in the hot tier. Combine with partition pruning for the full effect. C3X prices the buckets, ETL compute, and network paths from Terraform against theresource catalog before the pipeline ships.

FAQ

How much cheaper is Parquet than CSV?

Typically 4x to 10x smaller on storage and far more on query scan volume. An events table of 1 TB as CSV lands around 180 GB as Snappy compressed Parquet, and a query touching 3 of 40 columns scans roughly 13 GB instead of the full terabyte. At $5 per TB that is $0.07 per query instead of $5.00.

Why does Parquet reduce query cost so much?

It stores data by column rather than by row, so an engine reads only the column chunks a query references. It also records min and max statistics per row group, letting the engine skip blocks that cannot match the filter. Combined with compression, a query touching a few columns of a filtered range reads a tiny fraction of the file.

Which compression codec should I use with Parquet?

ZSTD is usually the best default, giving roughly 15 to 25 percent smaller files than Snappy at comparable decompression speed, which lowers both storage and scan bytes. Snappy remains a reasonable choice for very latency sensitive hot data, and gzip is slow enough that it belongs only on archival data you rarely read.

How large should Parquet files be?

Aim for 128 MB to 1 GB files with roughly 128 MB row groups. Very small Parquet files perform worse than well sized CSV because the engine spends its time opening files rather than reading data, and per request charges accumulate at about $0.005 per 1,000 LIST and $0.0004 per 1,000 GET operations.

What does converting CSV to Parquet cost?

Very little. Converting 1 TB on a managed ETL service at about $0.44 per DPU hour using 10 DPUs for half an hour costs roughly $2.20, and a Spark cluster is comparable. Against savings that typically run into thousands of dollars a month on query and storage charges, payback is measured in hours.

Should I keep the original CSV files?

Only if a compliance or reprocessing requirement demands it, and then in a cold storage tier rather than the hot tier. Keeping a raw copy in Standard storage cancels a meaningful share of the savings. Better still, write Parquet at ingestion time so no conversion step or duplicate copy exists in the first place.

What to do next

Design pipelines with cost in view. C3X prices your Terraform resources before they ship. Start with the quickstart.