Skip to content

Search across models, tools, services, companies, providers, APIs, plans, articles and news.

Costs & pricingIntermediate 15 min

Cutting LLM API costs — the four main levers

Prompt caching, smart routing, batch APIs and context hygiene — combined correctly, a 50–80% reduction is realistic, not marketing.

Cutting LLM API costs — the four main levers

Measure before you cut

You cannot optimize what you cannot see. From day one, log token usage per feature (input, output, reasoning tokens). Typically two or three hot paths make up 80% of the bill — optimization starts there, not with averages.

  • Mind the price table: input tokens are usually several times cheaper than output; reasoning tokens also cost.
  • Review each provider’s usage dashboard weekly.

Lever one: prompt caching

If your prompts carry a long static part (system prompt, reference documents, few-shot examples), prompt caching makes that part 50–90% cheaper on the major providers. Only mind the order: static content first, variable content (the user’s question) last.

  • Caches usually live a few minutes; batch repeated traffic close together.
  • Keep the static block minimal and separate from the variable part to avoid cache invalidation.

Levers two and three: routing and batching

Not every request needs a frontier model. Put a simple router in front that sends light work to a small model and escalates the hard cases — reported savings run 40–70%. Non-urgent work (archive summarization, classification, scheduled content) belongs on batch APIs, typically at half price.

  • To split easy/hard, ask the model itself to rate difficulty first, or use input length as a proxy.
  • Measure the small-model route against your golden set so quality drop becomes visible.

Lever four: context hygiene

The biggest hidden cost is endless chat history and bloated system prompts. Smart summarization of old turns, trimming tool logs and deduplication typically cut 20–40% of input tokens with no quality loss. Semantic caching, meanwhile, returns stored answers directly for repeated intents (FAQs).

  • Every 10 tokens removed from a system prompt is real money at thousands of requests.
  • Put an evaluation gate before trading quality for savings: the cheaper model stays only if tests pass.