How Much Do LLM API Calls Actually Cost? A Practical Guide
By ToolZoneX Team
•
August 2026
If you've ever been surprised by an OpenAI, Anthropic, or Google Gemini bill, it's usually because of one detail teams overlook until it's too late: input tokens and output tokens are priced separately, and output is almost always the more expensive half.
The two numbers that matter
Every major provider prices per million tokens, split into:
- Input tokens — your prompt, system instructions, conversation history, and any retrieved context (RAG chunks, tool results) you send with the request.
- Output tokens — what the model generates back. This is typically 3-5x more expensive per token than input, since generation is the computationally heavier half of the request.
That asymmetry is why a chatbot that echoes short answers is cheap, while a coding assistant or long-form writing tool that generates thousands of output tokens per response can get expensive fast — even if the input prompt is identical.
Roughly what things cost today
Pricing changes often, but as a snapshot (checked against provider documentation as of August 2026), cost per million tokens spans a wide range depending on model tier:
- Small/fast models (GPT-4o mini, Gemini 2.5 Flash-Lite) — a few cents per million input tokens, well under a dollar for output. Good for classification, extraction, and high-volume simple tasks.
- Mid-tier models (Gemini 2.5 Flash, Claude Haiku) — roughly $0.30-$1 per million input, $2-5 per million output. The default choice for most production chat and agent workloads.
- Frontier models (GPT-4.1, GPT-4o, Claude Sonnet, Gemini 2.5 Pro) — $2-3 per million input, $8-15 per million output. Reserved for tasks that genuinely need the strongest reasoning.
- Top-tier/flagship models (Claude Opus, Claude Fable) — $5-10 per million input, $25-50 per million output — the most capable tier, priced accordingly.
These bands shift constantly as providers release new models and cut prices on older ones, so always check current rates before budgeting — the LLM Cost Calculator keeps a running set of current per-model rates you can plug your own volume into.
The lever most teams miss: prompt caching
If your requests repeat the same system prompt, few-shot examples, or long context (a codebase, a document, a big tool schema) across many calls, prompt caching can cut costs substantially — providers discount cached input tokens because they don't need to reprocess content they've already seen recently. For any app sending the same large context repeatedly, this is usually the single biggest cost lever available, often bigger than switching to a cheaper model.
A worked example
Say you're running a support chatbot that handles 50,000 conversations a month, each averaging 800 input tokens (system prompt + history) and 300 output tokens (the reply). At a mid-tier model's rates (~$0.30/1M input, ~$2.50/1M output), that's roughly:
- Input: 50,000 × 800 = 40M tokens → ~$12
- Output: 50,000 × 300 = 15M tokens → ~$37.50
- Total: ~$50/month
Swap in a frontier model at 10x the per-token rate and the same volume jumps to roughly $500/month — which is exactly why picking the right model tier for the task, not just the most capable one, is the first cost decision worth making.
API calls vs. self-hosting
Paying per token only makes sense up to a point. At very high, steady volume, self-hosting an open-weight model on rented GPU hardware can be cheaper than API calls — but it trades a predictable per-token bill for infrastructure to manage (uptime, scaling, batching) and upfront GPU cost. If you're evaluating that tradeoff, the GPU Cost Calculator breaks down current hourly rental rates across GPU tiers, from budget inference cards to multi-GPU hyperscaler instances.
Before you ship
Run your expected monthly volume through the LLM Cost Calculator before committing to a model in production. It's a five-minute check that catches the two most common surprises: underestimating output token volume, and picking a frontier-tier model for a task a cheaper one would handle just as well.