LLM token cost explained: what you actually pay per call

Quick answer

LLM APIs bill per token, where a token is roughly three-quarters of a word, and split the price between input tokens (your prompt and context, cheaper) and output tokens (the model's response, usually several times more expensive). Cost per call equals input tokens times the input rate plus output tokens times the output rate. It grows fast when you send long prompts, large retrieved context, or full conversation history on every call. Control it by trimming context, caching, capping output length, and picking a smaller model where quality allows.

Every call to a large language model API has a price you can compute exactly, if you understand tokens. Because the model bills per token in and per token out, and those counts balloon with long prompts and conversation history, LLM cost can go from trivial to alarming as usage scales. Knowing the model is the first step to keeping the bill in hand.

How token pricing works

Component	What it is	Relative price
Input tokens	Your prompt, system message, and context	Cheaper
Output tokens	The model's generated response	Several times pricier

A token is roughly three-quarters of an English word, so a page of text is around 500 tokens. Providers charge a per-token rate for input and a higher per-token rate for output, often a few times higher, because generating tokens is more expensive than reading them. The cost of a single call is input tokens times the input rate plus output tokens times the output rate, and the totals add up across every request.

What makes token bills grow

The usual culprits are large context and repeated history. Retrieval-augmented generation stuffs retrieved documents into every prompt, and chat apps resend the whole conversation on each turn, so input tokens grow with usage. Long, verbose outputs multiply the pricier output charge. A system prompt sent on every call adds up too. The result is that naive designs pay for the same context thousands of times, the exact problem a RAG pipeline cost analysis unpacks in detail.

Levers to control token cost

Trim context to what the model actually needs, since every extra token in the prompt is billed. Cap the maximum output length so responses cannot run away. Use prompt caching where the provider offers it, so a repeated system prompt or context is charged at a reduced rate. Summarize or truncate long conversation history instead of resending it verbatim. Each lever cuts token counts directly, and together they can reduce cost by a large factor without hurting quality.

Model choice and the bigger picture

The largest lever is often model selection: a smaller or cheaper model can be many times less expensive per token, and for many tasks its quality is sufficient. Route easy requests to a small model and reserve the expensive model for hard ones. Weigh this against the cost of hosting your own model if volume is very high, and remember that token spend is only the inference side of the training versus inference split. Price your AI infrastructure choices against the resource catalog before you scale usage.

FAQ

What is a token in LLM pricing?

A token is a chunk of text the model processes, roughly three-quarters of an English word on average, so about 500 tokens per page of text. LLM APIs bill per token rather than per word or per call, counting both the tokens you send (input) and the tokens the model generates (output). Understanding tokens is the basis for computing and controlling LLM cost.

Why do output tokens cost more than input tokens?

Generating tokens is more computationally expensive than reading them, so providers charge a higher per-token rate for output than for input, often several times higher. This means verbose responses are disproportionately expensive, and capping output length is one of the more effective cost levers. Cost per call is input tokens times the input rate plus output tokens times the output rate.

How do I calculate the cost of an LLM API call?

Multiply the number of input tokens (your prompt, system message, and any context) by the input rate, multiply the number of output tokens (the model's response) by the higher output rate, and add the two. Across many calls, the total scales with how much context you send each time and how long the responses are, which is why long prompts and history get expensive.

What makes LLM token bills grow unexpectedly?

Large context and repeated history: retrieval-augmented generation stuffs retrieved documents into every prompt, chat apps resend the whole conversation on each turn, and a system prompt is sent on every call. These make input tokens grow with usage, so naive designs pay for the same context thousands of times. Verbose outputs multiply the pricier output charge on top.

How can I reduce LLM token cost?

Trim context to what the model actually needs, cap maximum output length, use prompt caching so repeated context is charged at a reduced rate, summarize or truncate long conversation history instead of resending it verbatim, and route easy requests to a smaller, cheaper model. Model choice is often the largest lever, since a smaller model can be many times cheaper per token.

How does C3X relate to LLM token cost?

C3X prices the cloud infrastructure around your AI workloads, the compute, hosting, and supporting services, before you deploy, so the surrounding cost is known at design time. While token spend is billed by the model provider, the infrastructure that serves, retrieves, and stores context for your LLM app is exactly what C3X prices against a live catalog in the pull request.

What to do next

Price the infrastructure behind your AI workloads before you scale. C3X reads your Terraform and prices your resources against a live catalog. Start with the quickstart.