Fine-tuning cost estimation: what a training run really costs

Quick answer

Estimate fine-tuning cost as GPU-hours times the hourly rate, where GPU-hours comes from tokens divided by achieved throughput. A LoRA fine-tune of a 7B model on 50 million tokens typically needs 6 to 12 GPU-hours on an A100 80GB, costing roughly $30 to $60 at about $5.12 per GPU-hour, or $10 to $20 on spot. A full-parameter fine-tune of the same model needs 8 GPUs and roughly 10x the compute, landing around $400 to $900. A 70B full fine-tune on billions of tokens runs into the tens of thousands. Budget for 5 to 15 runs, not one, because hyperparameter search and failed runs dominate real project cost.

Fine-tuning budgets go wrong in a predictable way: someone prices a single successful run, gets a reassuring number, and then spends ten times that on the experiments required to find the configuration that works. The estimate is not hard. It just needs to account for the whole campaign rather than the final run.

The four-number formula

GPU-hours equals training tokens divided by achieved tokens per second, divided by 3,600. Cost equals GPU-hours times the hourly rate times the number of GPUs times the number of runs. Everything else is refinement. The number people get wrong is achieved throughput, because published benchmarks assume ideal data pipelines and yours probably is not one.

Setup	Typical tokens/sec/GPU	Notes
7B LoRA on A100 80GB	about 3,000 to 6,000	Only adapter weights update
7B full fine-tune on A100 80GB	about 1,000 to 2,000	Optimizer states dominate memory
7B LoRA on H100 80GB	about 8,000 to 15,000	FP8 and better kernels
70B LoRA on 8x H100	about 1,500 to 3,000 per GPU	Sharded, interconnect matters

Worked examples

Case 1: 7B LoRA, 50 million tokens, 3 epochs. That is 150 million tokens of training at roughly 4,000 tokens per second, about 10.4 GPU-hours. On a single A100 80GB at about $5.12 per GPU-hour that is $53. On spot at roughly 60 percent off, about $21. Add a few dollars for storage and you have a sub-$60 experiment.

Case 2: 7B full fine-tune, same data. Throughput drops to about 1,500 tokens per second per GPU and you need 8 GPUs to hold optimizer states comfortably, so 150 million tokens takes about 3.5 hours of wall clock on 8 GPUs, roughly 28 GPU-hours. At $5.12 that is about $143, or on a p4de.24xlarge at $40.96 per hour for 3.5 hours, about $143. Consistent, because the per-GPU math is the same.

Case 3: 70B LoRA, 2 billion tokens. At about 2,000 tokens per second per GPU across 8 H100s, that is 16,000 tokens per second aggregate, so 2 billion tokens takes about 34.7 hours. A p5.48xlarge at $98.32 per hour gives roughly $3,412 for one run. Now multiply by the number of runs.

The multiplier nobody budgets

Activity	Typical runs	Effect on budget
Smoke tests on tiny subsets	10 to 30	Cheap, a few percent
Hyperparameter search	5 to 20	3x to 10x the single-run cost
Failed or diverged runs	2 to 5	Adds 20 to 50 percent
Final production run	1 to 2	The number everyone quotes

A realistic project multiplier is 5x to 15x the cost of the single successful run. The $3,412 job above is really a $20,000 to $50,000 program. Building that into the estimate up front is the difference between a forecast and a surprise, which is the same discipline described in ML training cost optimization.

Levers that cut the number

LoRA or other parameter-efficient methods instead of full fine-tuning cut compute by 3x to 10x and memory by more, often letting you drop from 8 GPUs to 1. Spot instances cut 50 to 70 percent when your training checkpoints every few hundred steps, covered in spot versus on-demand. Shorter sequence lengths help quadratically in attention. Deduplicating and filtering the dataset often removes 20 to 40 percent of tokens with no quality loss, which is a direct 20 to 40 percent cost cut. And running hyperparameter search on a smaller model or a 10 percent data subset before committing to the full run routinely saves more than every infrastructure optimization combined.

The supporting costs

Checkpoints are the sneaky one. A 7B model in FP32 with optimizer states is roughly 84 GB per checkpoint. Saving every 500 steps for a run with 40 checkpoints is 3.4 TB, about $78 a month on S3 Standard at $0.023 per GB-month, and far more if you never delete them across dozens of runs. Training data staged on FSx for Lustre runs around $0.145 per GB-month. Cross-region copies of data or checkpoints cost $0.02 per GB. None of these rival the GPU line, but together they are often 5 to 15 percent of the total.

Make the estimate before the approval

Run 200 steps on the target hardware, read off actual tokens per second, and extrapolate. That takes twenty minutes and turns a guess into a forecast accurate within about 15 percent. Then define the training cluster and its storage in Terraform and price it against the resource catalog, so the infrastructure side of the estimate is grounded in real rates rather than remembered ones.

FAQ

How do I estimate fine-tuning cost?

GPU-hours equals training tokens divided by achieved tokens per second per GPU divided by 3,600. Multiply by the hourly rate, the number of GPUs, and the number of runs. The number people get wrong is achieved throughput, so run 200 steps on the target hardware, read off the real figure, and extrapolate. That takes twenty minutes and gets the estimate within about 15 percent.

What does it cost to fine-tune a 7B model?

A LoRA fine-tune on 50 million tokens over three epochs takes about 10 GPU-hours at roughly 4,000 tokens per second, costing around $53 on an A100 80GB at about $5.12 per GPU-hour, or about $21 on spot. A full-parameter fine-tune of the same model on the same data runs roughly $143 because throughput drops and you need more GPUs to hold optimizer states.

Why do fine-tuning budgets get exceeded?

Because people price the single successful run and ignore the campaign around it. Hyperparameter search typically adds 3x to 10x, failed or diverged runs add 20 to 50 percent, and smoke tests add a few percent. A realistic project multiplier is 5x to 15x the single-run cost, so a $3,400 training job is really a $20,000 to $50,000 program.

How can I reduce fine-tuning cost?

Use LoRA or another parameter-efficient method, which cuts compute 3x to 10x and often lets you drop from eight GPUs to one. Use spot instances with frequent checkpointing for 50 to 70 percent off. Shorten sequence lengths, since attention cost grows quadratically. Deduplicate and filter the dataset, which commonly removes 20 to 40 percent of tokens with no quality loss, and run hyperparameter search on a data subset first.

How much does checkpoint storage add to a training run?

A 7B model in FP32 with optimizer states is roughly 84 GB per checkpoint. Saving every 500 steps across a run with 40 checkpoints is about 3.4 TB, which costs roughly $78 a month on S3 Standard at $0.023 per GB-month. That is small next to the GPU bill for one run but grows fast across dozens of experiments if old checkpoints are never deleted.

How does C3X help estimate fine-tuning cost?

C3X prices the GPU instances, storage volumes, and supporting infrastructure declared in your Terraform against a live catalog, so the hourly and monthly cost of a training cluster is known before it launches. Combining that with measured tokens per second gives a per-run figure, and multiplying by expected run count produces a defensible campaign budget rather than a hopeful one.

What to do next

Turn training budgets into forecasts. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.