machine learning
29 articles on machine learning — what drives the cost, how it is priced, and where the savings actually are.
Azure Machine Learning cost: compute is the bill, not the workspace
Azure ML's workspace is low-cost; the bill is the compute it runs, training clusters, compute instances, and managed online endpoints, plus associated storage. Idle compute instances and always-on endpoints are the classic waste. Here is the model.
GCP Cloud TPU cost: per chip-hour, with Spot and commitments
Cloud TPU bills per chip-hour by TPU version and configuration, with on-demand, Spot (preemptible), and committed-use pricing. Idle TPUs and full-price on-demand for interruptible training are the expensive mistakes. Here is how to cut cost.
AWS Inferentia and Trainium cost: cheaper ML accelerators than GPUs
Inf2 (Inferentia) and Trn1 (Trainium) instances are AWS custom ML accelerators priced per instance-hour, often far cheaper per inference or training step than GPU instances. The catch is model compatibility. Here is when they cut cost.
GCP Vertex AI cost: training, deployed endpoints, and idle prediction nodes
Vertex AI bills training per node-hour, online prediction per node-hour for the endpoint you keep deployed, plus AutoML and model API usage. Idle deployed endpoints are the classic surprise. Here is where the money goes and how to cut it.
SageMaker cost optimization: kill the idle inference endpoint
The classic SageMaker overspend is a real-time endpoint running 24/7 for a model called rarely. Here's how to cut it with Serverless Inference, multi-model endpoints, Managed Spot Training, and auto-stopping notebooks.