GCP Vertex AI cost: training, deployed endpoints, and idle prediction nodes

Quick answer

Vertex AI bills training per node-hour (by machine type and accelerators), online prediction per node-hour for the endpoint you keep deployed (billed even when idle), plus AutoML and model API (token) usage. The classic surprise is a deployed endpoint sitting idle around the clock, so undeploy unused endpoints and use batch prediction or autoscaling to zero where possible.

Vertex AI is Google Cloud managed ML platform, and its cost spans training, prediction, and managed model APIs. The line that catches teams is online prediction: a deployed endpoint bills for its nodes continuously, so an experiment left deployed quietly runs up cost long after anyone used it.

Training

Training jobs bill per node-hour based on the machine type and any GPUs or TPUs attached, for the duration of the job. Accelerators dominate when used, so right-size them, use them only when the model needs them, and prefer shorter, well-tuned jobs over long, oversized ones.

Online prediction: the idle-endpoint trap

Mode	Bills for	Idle cost
Online prediction endpoint	Node-hours deployed	Yes, continuously
Batch prediction	Resources for the job	None

A deployed online endpoint bills for its minimum nodes whether or not it serves requests, so idle or forgotten endpoints are pure waste. Undeploy endpoints you are not using, set autoscaling minimums low (to zero where supported), and use batch prediction for workloads that do not need real-time responses, since batch bills only for the job.

Model APIs and AutoML

Managed model APIs (including Gemini) bill per token or per character of input and output, so prompt and response size drive cost, the same token discipline as any LLM usage. AutoML training and prediction have their own per-hour and per-node rates. Match the approach to the need rather than reaching for the heaviest option.

Controlling Vertex AI cost

Undeploy unused endpoints, use batch prediction where real-time is not required, right-size accelerators for training, set low autoscaling minimums, and control token volume for model APIs. The endpoint-idle trap is the single most common Vertex AI overspend, the same idle-resource lesson as the reduce-your-cloud-bill checklist.

FAQ

How is Vertex AI priced?

Training bills per node-hour by machine type and accelerators, online prediction bills per node-hour for the deployed endpoint (even when idle), and managed model APIs bill per token or character. AutoML has its own per-hour and per-node rates. The deployed-endpoint line is the common surprise.

Why is my Vertex AI endpoint so expensive?

Because a deployed online prediction endpoint bills for its minimum nodes continuously, whether or not it serves requests. An idle or forgotten endpoint runs up cost around the clock. Undeploy unused endpoints and set autoscaling minimums low, to zero where supported.

How do I reduce Vertex AI cost?

Undeploy unused endpoints, use batch prediction where real-time responses are not needed (it bills only for the job), right-size accelerators for training, set low autoscaling minimums on endpoints, and control token volume for managed model APIs.

Online prediction or batch prediction for cost?

Use batch prediction when you do not need real-time responses, because it bills only for the job with no idle cost, unlike an online endpoint that bills for deployed nodes continuously. Reserve online endpoints for genuine low-latency, always-available serving.

How is Vertex AI training billed?

Per node-hour based on the machine type and any GPUs or TPUs attached, for the duration of the training job. Accelerators dominate cost when used, so right-size them, attach them only when the model needs them, and prefer shorter, well-tuned jobs.

Does C3X estimate Vertex AI cost?

Vertex AI cost is driven by node-hours, deployed endpoints, and token usage, which are usage inputs. C3X prices the surrounding infrastructure, and you model training, endpoint, and prediction usage to estimate the Vertex AI charges.

What to do next

Estimate the infrastructure around your ML platform before you deploy. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.