GPU spot interruption economics: when a 70 percent discount is worth the risk

Quick answer

GPU spot capacity typically costs 50 to 70 percent less than on-demand, so an A100 at about $5.12 per GPU-hour becomes roughly $1.50 to $2.50. The cost of an interruption is the work done since the last checkpoint plus restart overhead. With 15-minute checkpoints you lose at most 15 minutes per interruption, so even four interruptions a day waste only about 4 percent of runtime, far less than the 60 percent you save. Spot stops making sense when checkpointing is expensive or impossible, when capacity is so scarce that the job never completes, or for latency-critical serving where a lost replica means dropped requests.

The spot discount on GPUs is the largest single lever in AI infrastructure cost, and it is routinely left unused because interruption feels scary in a way that a 60 percent overspend does not. The right way to settle it is not intuition but expected value: the discount is known, and the expected loss from interruptions is computable once you decide your checkpoint interval.

The discount, roughly

GPU class	On-demand per GPU-hour	Typical spot range	Discount
T4 (g4dn)	about $0.53	about $0.16 to $0.25	55 to 70 percent
L4 (g6)	about $0.80	about $0.28 to $0.40	50 to 65 percent
A10G (g5)	about $1.01	about $0.35 to $0.50	50 to 65 percent
A100 80GB (p4de)	about $5.12	about $1.50 to $2.50	50 to 70 percent
H100 (p5)	about $12.29	about $4.00 to $8.00	35 to 65 percent

Spot rates move continuously with supply and demand, and high-end GPU spot is scarcer and less discounted than commodity CPU spot. The background mechanics are in spot versus on-demand savings.

The cost of an interruption

When a spot GPU is reclaimed you lose everything computed since the last checkpoint, plus the time to reacquire capacity, pull the container image, reload the checkpoint, and warm up. Call the second part restart overhead, typically 3 to 10 minutes for a large model.

Checkpoint interval	Avg work lost per interruption	Plus restart	Waste at 4 interruptions/day
60 minutes	about 30 min	about 37 min	about 10 percent of runtime
30 minutes	about 15 min	about 22 min	about 6 percent
15 minutes	about 7.5 min	about 14 min	about 4 percent
5 minutes	about 2.5 min	about 9 min	about 2.5 percent

Average work lost is half the interval, since interruptions arrive at random points. Notice that even the worst row wastes 10 percent, against a 60 percent saving. The expected value is overwhelmingly positive for anything restartable.

The break-even, stated plainly

Spot is worth it whenever the discount exceeds the interruption waste. At a 60 percent discount, you would need to lose more than 60 percent of your runtime to interruptions before on-demand wins. That requires something like an interruption every 20 minutes combined with hourly checkpoints. If you see that, the problem is the checkpoint interval, not spot.

But checkpointing is not free either

A 70B model checkpoint with optimizer states can be 800 GB. Writing that to object storage takes minutes and stalls training while it happens. Checkpointing every 5 minutes would spend more time writing than training. The practical answer is asynchronous or sharded checkpointing, where each rank writes its own shard in parallel and training resumes immediately, plus keeping the most recent checkpoint on local NVMe (included free on p4d, p5, and g6e instances) with periodic promotion to S3 at $0.023 per GB-month. Get that right and a 15-minute interval costs under 2 percent of throughput.

Where spot is the wrong answer

Workload	Spot suitable?	Why
Pretraining and fine-tuning	Yes	Checkpointable, deadline-flexible
Batch inference and embedding jobs	Yes	Idempotent, resumable per shard
Hyperparameter sweeps	Yes	Individual trials are disposable
Real-time serving, single replica	No	Interruption drops live requests
Real-time serving, mixed fleet	Partly	On-demand baseline plus spot burst
Tight-deadline production run	Careful	Capacity may not be available

For serving, the standard pattern is an on-demand or reserved baseline that covers your floor of traffic, with spot capacity handling the burst above it. If spot vanishes, you degrade to higher latency rather than failing, and you still capture the discount on the majority of capacity. That mirrors the mixed-fleet approach in model serving cost.

What the saving looks like on a real budget

Workload	On-demand monthly	Spot at 60 percent off	Saved
4x A100 training, 300 hrs/mo	about $6,144	about $2,458	about $3,686
8x A100 continuous	about $29,901	about $11,960	about $17,941
10x L4 batch scoring, 200 hrs/mo	about $1,610	about $644	about $966

Even after subtracting 5 to 10 percent for interruption waste, the middle row saves over $16,000 a month. That is the scale of what is left on the table when a training fleet runs entirely on-demand out of caution rather than out of a measured requirement.

Practical setup

Diversify across instance types and availability zones, since capacity pools are independent and a request spanning six pools is far less likely to be fully starved than one pinned to a single type. Handle the two-minute interruption notice by flushing a checkpoint immediately. Set a maximum price at the on-demand rate so you never pay more than you would have anyway. And model the mix in Terraform so the blended rate is visible: price the node groups against the resource catalog and the spot share becomes an explicit design decision rather than an afterthought.

FAQ

How much do spot GPUs save?

Typically 50 to 70 percent off on-demand, though high-end GPUs are scarcer and less discounted. An A100 80GB at about $5.12 per GPU-hour on-demand commonly runs $1.50 to $2.50 on spot. A T4 at about $0.53 drops to roughly $0.16 to $0.25. H100 spot discounts are more variable, often 35 to 65 percent, because demand for that capacity is high.

How much does a spot interruption actually cost?

The work done since your last checkpoint, which averages half the checkpoint interval, plus restart overhead of roughly 3 to 10 minutes for a large model. With 15-minute checkpoints, four interruptions a day waste about 4 percent of runtime. Against a 60 percent discount, that is an overwhelmingly positive trade for anything restartable.

What checkpoint interval should I use on spot GPUs?

Short enough that lost work is small, long enough that checkpointing does not dominate. Fifteen minutes is a good default for most training runs. Use asynchronous or sharded checkpointing so training resumes immediately, keep the latest checkpoint on the instance's included local NVMe, and promote periodically to object storage at $0.023 per GB-month rather than writing every checkpoint remotely.

When should I not use spot GPUs?

For single-replica real-time serving, where an interruption drops live requests; for tight-deadline production runs where capacity may simply not be available when you need it; and for workloads that cannot checkpoint cheaply. For serving generally, use an on-demand or reserved baseline covering your traffic floor with spot handling burst above it, so losing spot degrades latency instead of causing failures.

How do I reduce spot interruption frequency?

Diversify across instance types and availability zones, because capacity pools are independent and a request spanning six pools is far less likely to be starved than one pinned to a single type. Set your maximum price at the on-demand rate so you never overpay, handle the two-minute interruption notice by flushing a checkpoint immediately, and prefer older GPU generations where contention is lower.

How does C3X help with spot GPU planning?

C3X prices the instance types and node groups in your Terraform against a live catalog, so the blended cost of a mixed on-demand and spot fleet is visible before deployment. Seeing the on-demand equivalent next to your planned spot share makes the discount concrete and shows exactly how much an all-on-demand fallback would cost if capacity dried up.

What to do next

Make the spot decision with numbers. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.