Self-hosted LLM vs hosted API: where the cost crossover actually is

Quick answer

Self-hosting has a fixed floor and a near-zero marginal cost. One g6e.12xlarge (4x L40S) runs about $10.49 per hour, roughly $7,658 a month at 24x7. A hosted API charges per token with no floor. If a hosted model costs about $0.50 per million input tokens and $1.50 per million output tokens, that same $7,658 buys roughly 5 to 8 billion tokens a month. So the crossover for a mid-size open model sits in the region of 2 to 10 billion tokens per month, depending on your achieved GPU utilization. Below that, the API is cheaper and carries no ops burden. Above it, self-hosting wins, but only if you keep utilization above about 50 percent.

The self-hosting decision is usually argued on control, privacy, and latency. Those are real, but the argument that actually moves budgets is the cost curve, and the two options have fundamentally different shapes. A hosted API is a straight line through the origin: no volume, no cost. Self-hosting is a flat floor with a gentle slope: you pay the full GPU bill at one request per day and almost the same at ten million. The crossover is where those lines meet, and it is further out than most people assume.

What self-hosting actually costs

Setup	Hardware	Per hour	Per month at 24x7
Small model, single GPU	g6.xlarge, 1x L4 24GB	about $0.805	about $588
Mid model, one L40S	g6e.xlarge, 1x L40S 48GB	about $1.861	about $1,359
Mid model, four L40S	g6e.12xlarge, 4x L40S	about $10.49	about $7,658
Large model, eight H100	p5.48xlarge, 8x H100	about $98.32	about $71,774

And that is only the GPU line. Add a load balancer at about $16 a month plus LCU charges, model artifact storage on S3 at $0.023 per GB-month, a second replica for availability which doubles the GPU bill, and the engineering time to run it. A realistic all-in figure for a production self-hosted endpoint with two replicas of a mid-size model is $3,000 to $16,000 a month before anyone is paged at 3 a.m.

What the API side costs

Hosted inference is priced per million tokens, typically in the range of $0.15 to $3 for input and $0.60 to $15 for output depending on model class. Take a mid-tier point of $0.50 input and $1.50 output. A workload sending 1,000 input tokens and receiving 300 output tokens per request costs $0.0005 plus $0.00045, about $0.00095 per request, or $0.95 per thousand requests. Token pricing mechanics are covered in LLM token cost explained.

The crossover, calculated

Monthly requests	API cost at $0.95/1K	Self-hosted (2x g6e.xlarge)	Cheaper
100,000	about $95	about $2,718	API by 28x
1,000,000	about $950	about $2,718	API by 2.9x
3,000,000	about $2,850	about $2,718	Roughly even
10,000,000	about $9,500	about $2,718 to $7,658	Self-hosted
50,000,000	about $47,500	about $15,000 to $30,000	Self-hosted by 2x

The catch in the right-hand column is capacity. Two L40S GPUs will not serve 50 million requests a month at acceptable latency, so as volume grows you add GPUs and the self-hosted line stops being flat. It grows roughly linearly once you are saturated, just with a much lower slope than the API. That is why the crossover is a band, not a point.

Utilization is the whole game

The self-hosted cost per token is the hourly rate divided by tokens actually produced in that hour. If your GPU produces 2,500 output tokens per second when saturated, one L40S at $1.861 per hour generates 9 million tokens per hour at full tilt, a cost of about $0.21 per million output tokens. That is far below any hosted rate. At 20 percent utilization, the same GPU costs about $1.03 per million, and the advantage mostly evaporates. Traffic that peaks for four hours a day and idles the rest runs at maybe 25 percent utilization unless you scale down aggressively, a problem examined in GPU instance cost optimization.

The hybrid that usually wins

Most mature setups do not pick one. They route bulk, latency-tolerant, high-volume work to a self-hosted open model where the per-token cost is low, and send hard prompts or spiky overflow to a hosted API where there is no idle cost. Batch jobs go to spot GPUs at 50 to 70 percent off, since interruptions are cheap when work is restartable. That split lets you run the self-hosted fleet at high utilization, which is the only condition under which self-hosting is genuinely cheap, while the API absorbs the peaks you would otherwise have to over-provision for. Related routing tactics appear in LLM API cost optimization.

Before you commit

Measure three numbers: monthly token volume, achieved tokens per second per GPU under your real prompt distribution, and the utilization you can realistically sustain. Then compute self-hosted cost per million tokens and compare it directly with the API rate. Define the GPU fleet in Terraform and price it against the resource catalog so the fixed floor is visible before it is committed, not after two replicas have been running idle for a quarter.

FAQ

At what volume does self-hosting an LLM become cheaper than an API?

Typically somewhere between 2 and 10 billion tokens a month for a mid-size open model, though the exact point depends on achieved GPU utilization. Two L40S replicas cost roughly $2,718 a month and a hosted API at about $0.95 per thousand requests reaches that around 3 million requests. Below the crossover the API is cheaper and has no ops burden; above it, self-hosting wins if utilization stays high.

What is the fixed cost of self-hosting a model?

The GPU instances run whether or not requests arrive. A single L40S on g6e.xlarge is about $1.861 per hour, roughly $1,359 a month at 24x7, and production usually needs two replicas for availability. Add a load balancer at about $16 a month plus LCU charges, artifact storage at $0.023 per GB-month on S3, and engineering time. A realistic production floor is $3,000 to $16,000 a month.

Why does GPU utilization matter so much for self-hosting?

Because you pay per hour regardless of throughput, so cost per token is the hourly rate divided by tokens actually produced. An L40S at $1.861 per hour producing 2,500 output tokens per second when saturated costs about $0.21 per million output tokens. At 20 percent utilization that becomes about $1.03 per million, which erases most of the advantage over a hosted API.

Should I run a hybrid of self-hosted and hosted inference?

Usually yes. Route high-volume, latency-tolerant work to a self-hosted open model where per-token cost is low, and send spiky overflow or hard prompts to a hosted API that has no idle cost. Run batch jobs on spot GPUs at 50 to 70 percent off. This keeps the self-hosted fleet at high utilization, the only condition where self-hosting is genuinely cheap, while the API absorbs peaks.

What costs do people forget when self-hosting?

The second replica for availability, which doubles the GPU bill; load balancer and data transfer charges; model artifact and checkpoint storage; the cost of idle time during off-peak hours; and engineering time for upgrades, incident response, and capacity planning. Hosted APIs bundle all of that into the per-token rate, which is why the API often wins below the crossover even when the raw GPU math looks favorable.

How does C3X help with this decision?

C3X prices the GPU instances, load balancers, and storage in your Terraform against a live catalog, so the fixed monthly floor of a self-hosted endpoint is visible before it is deployed. Comparing that floor with your measured token volume and API rate turns a debate into arithmetic, and catching an oversized or duplicated GPU node group in the pull request avoids months of idle spend.

What to do next

Know the fixed cost of self-hosting before you commit. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.