GPU reserved capacity vs on-demand: committing to hardware that changes yearly

Quick answer

Commitment discounts on GPUs are real: roughly 25 to 40 percent for one year and 45 to 60 percent for three across the major clouds. The complication is that GPU generations turn over every 18 to 24 months, and a newer generation is often better value per unit of work, so a three-year commitment to current hardware can leave you paying a discounted rate for capacity that is no longer competitive. The usual answer is to commit one year rather than three on GPUs, cover only your measured floor of steady usage, prefer flexible savings plans over instance-specific reservations, and fill everything above the floor with spot.

Commitment discounts are the standard answer to steady cloud spend, and GPUs are the most expensive steady spend most AI teams have. But GPUs differ from general compute in one decisive way: the hardware improves fast enough that price-performance can shift by more than the discount you locked in. That changes the calculation in ways that a general reserved-instance playbook does not cover.

The discounts on offer

Mechanism	1 year	3 years	Flexibility
AWS Compute Savings Plan	about 25 to 30 percent	about 45 to 55 percent	High, across families
AWS EC2 Instance Savings Plan	about 30 to 40 percent	about 50 to 60 percent	Family and region locked
GCP committed use discount	about 37 percent	about 55 percent	Resource-based, moderate
Azure reservation	about 30 to 40 percent	about 55 to 60 percent	Size flexible within family
Spot / low-priority	50 to 70 percent	n/a	Interruptible

Note the last row: spot frequently beats even three-year commitments, with no lock-in, for any workload that tolerates interruption. Commitments are for the capacity that genuinely cannot be interrupted.

The generation risk, quantified

Suppose you commit three years to A100 capacity at about $5.12 per GPU-hour list, getting 55 percent off for an effective $2.30. Attractive. Two years later, newer hardware delivers three times the throughput at 2.4 times the price, so its cost per unit of work is about 20 percent lower than A100 list and comparable to your discounted A100 rate, while also being available on spot at 60 percent off. Your discount has been eroded not by price changes but by performance changes, and you still have a year to run.

Horizon	Discount	Generation risk	Usual verdict on GPUs
On-demand	0 percent	none	Default for uncertain workloads
1 year	25 to 40 percent	low	Best risk-adjusted choice
3 years	45 to 60 percent	high	Only for a proven stable floor
Spot	50 to 70 percent	none	Everything interruptible

Cover the floor, not the average

The standard error is committing to average usage. If your GPU fleet varies between 6 and 20 instances, a commitment covering 13 means you pay for 13 during the periods when you use 6. Commit to the floor, the level below which usage never falls, typically your p5 or p10 rather than your mean. Everything above the floor runs on-demand or spot. That is the same coverage-versus-utilization balance described in savings plan coverage versus utilization.

A worked decision

A team runs a steady inference fleet of 8 g6e.xlarge at about $1.861 per hour, roughly $10,868 a month at list, plus variable training that ranges from 0 to 12 A100-equivalent GPUs.

Strategy	Approx monthly	Notes
All on-demand	about $10,868 plus training	Maximum flexibility, highest cost
1-year plan on 8 serving nodes	about $7,500	About 31 percent saved, low risk
3-year plan on 8 serving nodes	about $5,000	About 54 percent saved, generation risk
1-year plan plus spot training	about $7,500 plus 60 percent off training	Usual best balance

The fourth row is typically the right answer: commit the serving floor for one year, run all training on spot, and leave burst serving on-demand. It captures most of the available saving while keeping the option to move to newer hardware within twelve months.

Capacity blocks and reserved GPU pools

Beyond discounts, some clouds sell reserved GPU capacity primarily for availability rather than price, letting you book a cluster of high-end GPUs for a defined window. For scarce hardware, guaranteed availability can be worth more than a discount: a training program that cannot start because no capacity is free costs more than a 20 percent premium. Price that as an availability purchase, not a savings one.

What happens when the commitment stops fitting

Commitments do not adapt when your architecture does. If you commit to a family of single-GPU inference nodes and then consolidate onto fewer large multi-GPU instances, an instance-specific reservation may no longer apply and you pay list on the new shape while still owing the old commitment. A flexible compute savings plan usually absorbs that change because it applies across families, which is why a slightly smaller discount on a flexible instrument often beats a larger one on a rigid instrument. Before signing, ask what happens if the model gets quantized and halves your GPU count, or if a new generation arrives next quarter. If the answer is that you keep paying regardless, size the commitment for the part of the fleet you would keep in either case.

Decide with numbers in review

Whatever you commit to, base it on measured usage over at least 60 days, re-examine quarterly, and prefer flexible instruments that survive an instance family change. Model the fleet in Terraform and price it against the resource catalog so the on-demand baseline is known before you negotiate against it, and so a change in instance family is immediately visible as a change in what your commitment actually covers.

FAQ

How much do GPU commitments save?

Roughly 25 to 40 percent for one year and 45 to 60 percent for three across the major clouds, depending on the instrument. AWS Compute Savings Plans run about 25 to 30 percent for one year, GCP committed use discounts about 37 percent, and Azure reservations about 30 to 40 percent. Spot capacity at 50 to 70 percent off frequently beats even three-year commitments for interruptible work.

Should I commit to GPU capacity for three years?

Usually not. GPU generations turn over every 18 to 24 months and newer hardware often delivers better price-performance, so a three-year lock can leave you paying a discounted rate for capacity that is no longer competitive. One year is generally the best risk-adjusted horizon for GPUs, reserving three-year commitments for a proven, stable, long-lived serving floor.

How much capacity should I commit to?

Only the floor, meaning the level below which your usage never falls, typically the fifth or tenth percentile rather than the mean. If a fleet varies between 6 and 20 instances, committing to 13 means paying for 13 during periods when you use 6. Everything above the floor should run on-demand or on spot, which preserves flexibility while still capturing most of the discount.

Is spot better than a commitment for GPUs?

For anything interruptible, usually yes. Spot discounts of 50 to 70 percent often exceed even three-year commitment rates, and they carry no lock-in or generation risk. Commitments are for capacity that genuinely cannot tolerate interruption, such as production inference. The common pattern is a one-year commitment on the serving floor with all training running on spot.

What are GPU capacity reservations for?

Availability rather than price. Some clouds let you book a block of scarce high-end GPUs for a defined window so the capacity is guaranteed when your training program starts. For hardware that is hard to obtain, guaranteed availability can be worth more than a discount, since a program that cannot start because nothing is free costs more than a modest premium.

How does C3X help with GPU commitment decisions?

C3X prices the GPU instances in your Terraform against a live catalog, so the on-demand baseline of a fleet is a known number before you commit against it. It also makes instance family changes visible in review, which matters because a change of family can leave an instance-specific commitment covering capacity you no longer run.

What to do next

Know your on-demand baseline before you commit. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.