data-integrationcost-optimizationdata-platformelt

Managed data ingestion cost models: per row, per GB, per connector, per credit

Managed ingestion platforms price the same work four completely different ways, and the cheapest model depends entirely on the shape of your data. Here is how to compare them without guessing.

The C3X Team··8 min read

Quick answer

Managed ingestion is priced four ways: per active row changed, per GB moved, per connector per month, and per compute credit. Row based pricing punishes wide tables with frequent small updates; GB based pricing punishes large binary or text columns; connector based pricing punishes long tails of small sources. The same pipeline can vary by 5x or more across models. Self building on a managed ETL service at around $0.44 per DPU hour costs roughly $100 to $400 a month per pipeline in compute but adds engineering maintenance. Model your own row counts, byte volumes, and update frequency before comparing quotes.

Ingestion pricing is where cost modelling gets genuinely hard, because vendors meter fundamentally different things and the mapping between them depends on data you may not have measured. A team quoting three platforms can get numbers a factor of five apart for the same pipelines. The way through is to characterize your own data first, then run each pricing model against it.

The four models

ModelMeterCheap whenExpensive when
Per active rowDistinct rows inserted or changed per monthFew, wide rows changeMany narrow rows update often
Per GBBytes transferredNarrow rows, small payloadsWide rows, blobs, JSON documents
Per connectorNumber of configured sourcesFew large sourcesLong tail of small sources
Per credit or compute hourProcessing timeEfficient batch loadsFrequent small syncs

The same pipeline, four prices

Consider a pipeline replicating 12 tables from an operational database. Monthly: 40 million distinct rows change, the average row is 800 bytes, total transfer is 32 GB, and syncs run every 15 minutes.

ModelCalculationApprox. monthly cost
Per row at $0.0000340M rows~$1,200
Per GB at $1532 GB~$480
Per connector at $1001 source connector~$100
Self built on managed ETL0.5 DPU hours per sync, 2,880 syncs, $0.44/DPU-hr~$634

Now change one characteristic. If the rows average 8 KB instead of 800 bytes because they contain JSON payloads, transfer becomes 320 GB and the per GB model jumps to $4,800 while the per row model stays at $1,200. Conversely, if a status column updates on 400 million rows a month but the bytes barely move, the per row model becomes $12,000 while per GB stays low. Neither model is inherently better; they are bets on different data shapes.

Characterize before you compare

Measure four things per source: distinct rows changed per month, average row width in bytes, total sync frequency, and number of distinct sources. Most teams can get the first two from a week of production logs or a simple count over an updated at column. Without them, every quote is a guess.

Watch specifically for the high churn low value table. A sessions table where a last_seen timestamp updates on every request can generate more billable row changes than every business critical table combined. Excluding it, or syncing it hourly instead of every five minutes, often cuts a row priced bill by half. Similarly, deselect columns you do not use: on a GB priced model, dropping three large text columns from a 40 column table can cut transfer by 70 percent.

Sync frequency is the universal lever

Sync intervalSyncs/monthRelative row churn billedRelative compute
5 minutes8,640HighestHighest
15 minutes2,880~60% of 5 min~35%
1 hour720~35% of 5 min~12%
6 hours120~20% of 5 min~3%

Row based models often bill a row once per sync period in which it changed, so a row updated 30 times an hour counts once per hourly sync instead of twelve times under a five minute cadence. Frequency reduction therefore cuts cost on every model simultaneously. Set it per table from the real downstream requirement rather than globally at the fastest setting anyone asked for.

Build versus buy

One more variable worth pricing: historical backfills. Most models bill an initial full sync at the same rate as ongoing changes, so loading five years of history into a row priced platform can produce a first month invoice several times the steady state. Ask how backfills are metered before signing, and where possible load history once through a cheaper bulk path and let the managed connector handle only incremental changes.

Self building is cheaper on infrastructure and more expensive on people. A managed ETL job at $0.44 per DPU hour running 2 DPUs for 10 minutes every 15 minutes costs about $211 a month. Add orchestration, a small always on task for CDC, and storage, and a hand built pipeline runs $300 to $600 a month in infrastructure. Against a $1,200 vendor bill that looks like a win until you add schema drift handling, API pagination changes, retry logic, and monitoring, which realistically consume several days of engineering per quarter per source. The economics favour buying for standard SaaS sources with volatile APIs, and building for internal databases with stable schemas where CDC does most of the work.

Whichever you choose, the compute, storage, and network underneath are Terraform, and C3X prices them from theresource catalog so the infrastructure half of the decision is quantified in the pull request.

FAQ

Which ingestion pricing model is cheapest?

It depends entirely on your data shape. Per row pricing is cheap when few wide rows change and expensive when many narrow rows update often. Per GB is the reverse. Per connector favours a small number of large sources. The same 12 table pipeline can range from roughly $100 to $1,200 a month across models.

What should I measure before comparing ingestion quotes?

Four things per source: distinct rows changed per month, average row width in bytes, sync frequency, and the number of distinct sources. Most teams can derive the first two from a week of production logs or a count over an updated at column. Without those numbers every vendor quote is guesswork.

What is a high churn low value table?

A table where a low importance column updates constantly, such as a sessions table whose last_seen timestamp changes on every request. It can generate more billable row changes than every business critical table combined. Excluding it or syncing it hourly rather than every five minutes often halves a row priced ingestion bill.

How much does sync frequency affect cost?

Substantially, across every pricing model. Moving from five minute to hourly syncs cuts billed row churn to roughly 35 percent and compute to about 12 percent, because rows changing repeatedly within a window are typically billed once per window. Set frequency per table from real downstream needs rather than globally at the fastest requested setting.

Is building your own ingestion cheaper?

On infrastructure, usually. A managed ETL job running 2 DPUs for ten minutes every fifteen minutes at $0.44 per DPU hour is about $211 a month, and a full hand built pipeline lands around $300 to $600. The difference goes into engineering time for schema drift, API changes, retries, and monitoring, typically several days per quarter per source.

When is buying clearly the right call?

For standard SaaS sources with volatile APIs, where connector maintenance is the real cost and a vendor amortizes it across many customers. Building tends to win for internal databases with stable schemas, where change data capture does most of the work and the pipeline needs little ongoing attention once it is running correctly.

What to do next

Quantify the infrastructure half of build versus buy. C3X prices your Terraform resources before deploy. Start with the quickstart.

Try C3X on your own Terraform

Free and open source. No API key required. One command to install, one command to estimate.