Cloud Cost Engineering and Open Source FinOps
Notes from the team building C3X. Cost estimation for Terraform, the economics of cloud infrastructure, and how to ship FinOps tooling without a SaaS gate.
MLOps pipeline cost: budgeting the whole ML lifecycle
MLOps cost spans data prep, training, serving, feature stores, registries, monitoring, and orchestration, each a piece of the bill. Serving and training compute usually dominate, but the hidden pieces add up. Here is how to budget the whole lifecycle.
AWS ParallelCluster cost: the HPC cluster is the compute
ParallelCluster is a free tool that provisions HPC clusters; the cost is the underlying EC2 compute, storage, and networking it launches. Right-sizing nodes, using Spot, and scaling to zero when idle drive the bill. Here is the model.
Azure Batch cost: the service is free, the VMs are not
Azure Batch is free to use; you pay for the underlying VM compute in the pools it manages, plus storage and networking. Low-priority (Spot) VMs, right-sized pools, and autoscaling to zero drive the bill. Here is the model.
HPC in the cloud: what high-performance computing really costs
Cloud HPC trades a capital cluster purchase for pay-as-you-go compute, so cost is driven by node type, job runtime, and utilization. Spot, scale-to-zero, and efficient jobs make it economical. Here is how the cost works.
Spot instance cost strategy: big discounts for interruptible work
Spot instances (spare capacity at up to 90% off) can be reclaimed with little notice, so they suit fault-tolerant, interruptible, or checkpointed work. Diversifying pools and handling interruption make them safe. Here is the strategy.
GPU instance cost compared: choosing the right accelerator
GPU instances vary enormously in cost, from older inference GPUs to top training accelerators costing many dollars per hour. Matching GPU class to the workload, and using Spot, drives the bill. This compares the tradeoffs.
ML training cost optimization: getting models trained for less
Training cost is GPU-hours: accelerator class times count times training time. Spot with checkpointing, right-sized GPUs, efficient data pipelines, and stopping idle instances cut it sharply. Here is how.
ML inference cost optimization: serving models efficiently
Inference cost is serving capacity times uptime: the accelerators or CPUs kept ready to respond. Autoscaling, right-sized hardware, batching, and scaling to zero when idle cut it. Here is how.
GPU vs CPU for inference: which is cheaper to serve on
GPUs accelerate inference but cost far more per hour; CPUs are cheaper but slower per request. For small models, low throughput, or latency-tolerant workloads, CPU is often cheaper per inference. Here is how to decide.
Model serving cost: managed endpoints vs self-hosted
Serving a model can use a managed endpoint (SageMaker, Vertex, Azure ML) that bundles ops at a premium, or self-hosted serving on your own compute that is cheaper per hour but adds operational burden. Here is the tradeoff.
Batch vs real-time inference: the cost of immediacy
Real-time inference keeps serving capacity always ready to respond instantly; batch inference processes predictions in bulk on transient compute. When predictions can wait, batch is far cheaper. Here is the tradeoff.
LLM API cost optimization: paying less per token
LLM API cost is tokens: input plus output tokens times the per-token rate, which varies by model tier. Choosing the right model per task, trimming prompts, caching, and capping output cut the bill. Here is how.
RAG infrastructure cost: the pieces behind retrieval-augmented generation
A RAG system's cost spans a vector database, embedding generation, the LLM API for generation, and the retrieval compute. Each scales differently, and the LLM and vector store usually dominate. Here is how to budget it.
AWS Verified Access cost: per app connection per hour plus data
Verified Access bills per application connected per hour plus data processed, giving zero-trust access to internal apps without a VPN. Many connected apps and high traffic drive the bill. Here is the model.
AWS Clean Rooms cost: per compute-unit of collaboration queries
AWS Clean Rooms bills for the compute (CRPU) that runs privacy-preserving queries across partners' data without sharing raw data. Query volume and complexity drive the bill. Here is the model.
Azure Firewall Manager cost: policy management plus the firewalls
Firewall Manager centrally manages Azure Firewall policies and security across many networks; the meaningful cost is the Azure Firewalls themselves plus per-policy charges beyond the first. Here is the model.
Azure DDoS Protection cost: a flat monthly plan plus overage
Azure DDoS Protection (Network or IP Protection) bills a flat monthly fee covering a set of protected resources, with per-resource charges beyond the included count. It protects against volumetric attacks. Here is the model.
AWS GuardDuty cost explained: per GB and per event analyzed
GuardDuty bills by the volume of logs and events it analyzes, CloudTrail events, VPC flow logs, DNS logs, and per feature (S3, EKS, malware, RDS protection). High-traffic accounts and enabled features drive the bill. Here is the model.
Secrets management cost compared: AWS vs Azure vs GCP vs Vault
AWS Secrets Manager charges per secret per month plus API calls; Azure Key Vault and GCP Secret Manager charge mostly per operation with cheap or free storage; self-hosted Vault trades license or ops cost. This compares the models.
WAF cost compared: AWS WAF vs Azure vs Cloud Armor
AWS WAF charges per web ACL, per rule, and per million requests; Azure WAF and GCP Cloud Armor bundle differently (policy plus per-rule plus requests). The cheapest depends on rule count and traffic. This compares the models.
DDoS protection cost compared: AWS Shield vs Azure vs Cloud Armor
AWS Shield Standard is free with Advanced a flat monthly commitment; Azure DDoS Protection is a flat plan or per-IP; GCP relies on always-on network protection plus Cloud Armor. The models differ sharply. This compares them.
Cloud security tooling cost: budgeting for the security stack
Security tools, threat detection, posture management, WAF, secrets, DDoS, add up across a cloud estate. Most are usage- or resource-based, so the stack cost scales with your footprint. Here is how to budget for it sensibly.
SIEM cost compared: Sentinel vs Security Lake vs Chronicle
SIEM cost is dominated by data ingestion: Microsoft Sentinel bills per GB ingested (with commitment tiers), AWS Security Lake centralizes data in S3 with usage-based analytics, and Google Chronicle prices differently. Ingestion volume drives all. This compares them.
Data egress cost optimization: the tax on moving data out
Egress, data leaving a cloud to the internet or across regions, is billed per GB and is one of the most underestimated cloud costs. Architecture, caching, and keeping traffic in-region cut it. Here is how.