Azure ML compute cluster cost: sizing, scaling, and low-priority nodes

Quick answer

Azure Machine Learning does not add a surcharge on compute: you pay the underlying VM rate, so an NC4as_T4_v3 with one T4 costs about $0.526 per hour and an NC24ads_A100_v4 with one A100 80GB about $3.67. That means the cost is decided entirely by cluster configuration. A compute cluster with minimum nodes set to 1 instead of 0 on A100 hardware costs about $2,679 a month doing nothing. Set minimum nodes to 0, set the idle scale-down timeout to 120 to 300 seconds, and use low-priority nodes at up to 90 percent off for interruptible training.

Azure ML has an unusual and welcome property among managed ML platforms: it charges the standard VM rate for compute with no platform markup. The corollary is that every dollar of the bill is a consequence of a configuration choice you made, and the two choices that matter most, minimum node count and idle timeout, take about thirty seconds to set correctly and are routinely left at expensive defaults.

GPU VM rates that matter

VM size	GPU	Approx per hour	Monthly at 24x7
NC4as_T4_v3	1x T4 16GB	about $0.526	about $384
NC8as_T4_v3	1x T4 16GB	about $0.752	about $549
NC24ads_A100_v4	1x A100 80GB	about $3.67	about $2,679
ND96asr_v4	8x A100 40GB	about $27.20	about $19,856
ND96isr_H100_v5	8x H100 80GB	about $98	about $71,540

Rates vary by region and these are US list figures. The same VMs are available outside Azure ML, which is precisely why the platform can price at parity.

The two settings that decide the bill

Minimum nodes. A compute cluster with minimum nodes set to zero costs nothing when no jobs are queued. Set it to one for faster job start and you pay the full VM rate continuously, 730 hours a month whether or not anyone submits a job. On A100 hardware that is about $2,679 a month for a warm start that saves perhaps three minutes per job. Unless you submit jobs constantly, this is the worst trade in the product.

Idle scale-down timeout. The default is generous, and clusters commonly sit idle for many minutes after each job. A cluster running 40 jobs a day with a 30-minute idle timeout wastes up to 20 hours a day of node time if jobs are spread out. Dropping to 120 to 300 seconds typically recovers 30 to 60 percent of a training cluster's bill with no practical impact.

Config	Effective billed hours/month	Cost on NC24ads_A100_v4
Min nodes 1, 30 min idle	730	about $2,679
Min nodes 0, 30 min idle	about 400	about $1,468
Min nodes 0, 5 min idle	about 180	about $661
Min nodes 0, 5 min idle, low-priority	about 180	about $130

Low-priority nodes

Azure low-priority VMs use surplus capacity and can be preempted with 30 seconds of notice, in exchange for discounts commonly reaching 60 to 90 percent. For training that checkpoints regularly, this is close to free money: a job that takes 10 percent longer because of occasional preemption but costs 80 percent less is an overwhelming win. Azure ML supports low-priority compute clusters directly and will requeue preempted jobs. The reasoning matches the interruption economics used for spot capacity generally.

Reservations for the steady part

If some GPU capacity genuinely runs continuously, for example a production inference endpoint, Azure reservations cut roughly 30 to 40 percent for one year and up to about 60 percent for three, as described in Azure reservations explained. Reserve only the floor you are certain about and leave burst capacity on-demand or low-priority, because a three-year GPU commitment on a generation that turns over in two is an expensive way to be right about utilization.

The other lines

Managed online endpoints bill the VM rate per instance continuously, so the same minimum-instance discipline applies: an endpoint serving a retired model is pure waste. The workspace requires a storage account, a key vault, and a container registry, which together typically cost $20 to $200 a month depending on artifact volume, and the registry can grow quickly if old model images are never pruned. Data egress from the region follows standard Azure rates. Broader tactics are in Azure ML cost optimization.

Right-size the VM to the job

Azure GPU VM families bundle very different CPU and memory allocations, and picking the largest available size out of habit is expensive. NC8as_T4_v3 at about $0.752 per hour gives the same single T4 as NC4as_T4_v3 at about $0.526, with double the vCPUs. If your data pipeline is GPU-bound, that extra $0.226 per hour buys nothing, about $165 a month per node wasted. If your pipeline is CPU-bound on augmentation or decoding, the larger size may raise GPU utilization enough to pay for itself several times over. The way to tell is to measure GPU utilization during a real job: below 60 percent with idle GPU waits points at a CPU bottleneck, and above 90 percent means the smaller size is the right call.

Encode the settings, do not remember them

Minimum node count, maximum node count, idle timeout, VM size, and priority tier are all Terraform attributes. Put a module in place that defaults minimum nodes to zero and idle timeout to a few minutes, and price the cluster against the resource catalog in review. That turns the two most expensive mistakes in Azure ML into things the codebase prevents rather than things a person has to remember.

FAQ

Does Azure ML add a markup on compute?

No. Azure Machine Learning charges the standard underlying VM rate with no platform surcharge, so an NC4as_T4_v3 with one T4 costs about $0.526 per hour and an NC24ads_A100_v4 with one A100 80GB about $3.67, the same as running those VMs directly. That means the entire bill is determined by how you configure clusters, not by a platform fee.

What is the most expensive Azure ML misconfiguration?

Setting minimum nodes to one instead of zero on a compute cluster. That keeps a node running continuously, 730 hours a month, whether or not any job is queued. On NC24ads_A100_v4 hardware that is about $2,679 a month to save roughly three minutes of job start time. Unless jobs arrive constantly, minimum nodes should be zero.

What idle timeout should an Azure ML cluster use?

Typically 120 to 300 seconds. Long idle timeouts mean nodes sit billing after each job completes, and a cluster running spread-out jobs with a 30-minute timeout can waste most of the day. Dropping to a few minutes commonly recovers 30 to 60 percent of a training cluster's cost with no practical impact on throughput.

How much do Azure low-priority nodes save?

Commonly 60 to 90 percent off the standard VM rate, in exchange for possible preemption with about 30 seconds of notice. For training that checkpoints regularly this is near free money: a job taking 10 percent longer due to occasional preemption but costing 80 percent less is a clear win, and Azure ML can requeue preempted jobs automatically.

Should I buy Azure reservations for GPU capacity?

Only for capacity that genuinely runs continuously, such as a production inference endpoint. Reservations cut roughly 30 to 40 percent for one year and up to about 60 percent for three years. Reserve just the floor you are confident about and leave burst on-demand or low-priority, since a three-year commitment on a GPU generation that turns over in two carries real risk.

How does C3X help with Azure ML cost?

Minimum and maximum node counts, idle timeouts, VM sizes, and priority tiers are all Terraform attributes, so C3X prices a compute cluster against a live catalog before it is created. A module that defaults minimum nodes to zero and idle timeout to a few minutes turns the two most expensive Azure ML mistakes into things the codebase prevents automatically.

What to do next

Set Azure ML cluster defaults in code. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.