Most cost-cutting in AI comes with a catch. Switch to a cheaper model and you might lose quality. Turn down effort and you risk a worse answer on a hard task. There’s judgement involved, trade-offs to weigh.

Prompt caching has no catch. It’s a discount of roughly 90% on repeated input that changes nothing about the output — same model, same answer, a fraction of the bill. And most builders never turn it on, because it’s a setting on a pricing page nobody reads rather than a button in the chat window.

This is the closest thing to free money in the whole token economy. Let’s collect it.

The evergreen idea: you keep paying for the same tokens

Here’s what actually happens on a coding session. Every turn, the model needs context: your system prompt, the files it’s working on, the docs, the conversation so far. That context gets re-sent on every single call. Ask ten follow-up questions about the same file and you’ve sent that file ten times — and paid for it ten times.

On most real calls, input dwarfs output. You might send 50,000 tokens of context to get back 500 tokens of code. So your bill is dominated not by what the model writes, but by the same context you keep re-feeding it.

Prompt caching breaks that loop. The provider stores the stable prefix of your request — the part that doesn’t change between calls — and on the next call, instead of charging full price to re-read it, charges roughly a tenth. The first call “writes” the cache at normal price (or slightly above); every call after “reads” it at a deep discount. Reuse the same context across a session and the savings compound fast.

The mental model: stable context is an asset you pay to store once, not a cost you pay to re-send every time.

Cost of five repeated calls with the same context, with and without prompt caching Without caching — full input every call = 5.0 units With caching — full once, then ~10% = 1.4 units Illustrative: input-heavy calls that reuse a stable prefix (system prompt, repo, docs). Cached input reads bill roughly -90% on the majors. Batch API takes another -50%. OpenAI note: cache writes ~1.25x standard input, ~30-min cache life. Keep the prefix stable.

The current state (as of 19 July 2026)

Caching is now standard across the majors and most challengers. The exact terms differ, so confirm against current docs before you build around them.

As of 2026-07-19:

  • Claude (Fable 5, Opus 4.8, Sonnet 5, Haiku 4.5): cached input reads bill roughly −90%. Combined with the batch API (−50% on non-interactive jobs), input-heavy workloads get dramatically cheaper.
  • OpenAI GPT-5.6 (Sol / Terra / Luna): cached input reads −90%, but cache writes cost about 1.25× standard input, and the cache lives ~30 minutes. So caching pays off when you reuse a prefix soon and often; a one-off call gets no benefit and a small write penalty.
  • Challengers: Kimi’s context caching drops repeated input to roughly $0.10–0.16 / 1M (from ~$3 uncached); GLM lists cached input as low as $0.03–0.26 depending on model; DeepSeek is already so cheap that caching is a bonus rather than a necessity.

Because caching acts on input, and input dominates most calls, it is usually the single biggest lever on a raw-token bill — bigger than switching one model tier down.

Why builders miss it

Two reasons. First, it’s invisible: nothing in a chat UI says “you just re-sent 50k tokens.” The waste doesn’t show up until the invoice does. Second, it takes a little structure — caching only works on a stable prefix, so if you shuffle your context every call (reordering files, editing the system prompt, injecting a fresh timestamp near the top), the cache never gets a hit. The discount is real but it rewards discipline: put the unchanging stuff first and leave it alone.

That’s the difference between builders who see caching “working” and those who swear it does nothing. It’s not the feature — it’s whether their prompt is shaped to let the cache catch.

The playbook

  • Put stable content first, volatile content last. System prompt, repo files, reference docs at the top (cacheable); the specific question at the bottom. Never inject a changing value (timestamp, random ID) into the cacheable prefix.
  • Reuse within the cache window. On OpenAI especially, the ~30-minute life means bursts of related calls cache well; sporadic one-offs don’t. Batch your related work together in time.
  • Batch anything non-interactive. Overnight refactors, bulk transforms, test generation — the batch API’s −50% stacks on top of caching.
  • Mind the write penalty. A prefix you cache but only call once costs slightly more, not less (the ~1.25× write). Cache things you’ll reuse, not everything.
  • Measure input vs output. If your calls are input-heavy (most coding is), caching is your top priority. If they’re output-heavy (long generation), model and effort choice matter more.
  • Re-check the terms. Discount rates, write costs, and cache lifetimes get retuned; confirm current numbers before relying on them.

The Khula Take

Caching is filed under “neat optimisation,” a tip you apply after the real work. We think that’s exactly the wrong frame. The lesson caching teaches isn’t “toggle this setting” — it’s that you should be paying for context once and reusing it, everywhere in your stack, not just in the API.

Follow that idea past the pricing page and it changes how you build. The most expensive thing in most AI workflows isn’t the model — it’s re-deriving the same context over and over: re-reading the same documents, re-explaining the same rules, re-answering the same customer question. Caching is that principle applied to one API call. The bigger win for a South African business is applying it to the whole operation: build systems where knowledge is captured once and served cheaply forever — a cached answer, a stored result, a workflow that doesn’t recompute what it already knows. That’s Applied AI as leverage. The API discount is the demo; designing your business so it never pays twice for the same thinking is the product.

Where this is heading

Getting caching right by hand — shaping every prompt so the prefix stays stable, batching work into the cache window — is fiddly, and it’s exactly the kind of housekeeping the Token Economy harness will handle: structuring calls so the cache always catches, and reporting your real post-cache cost rather than the sticker one. The estimator, too, will price a build assuming disciplined caching, because that’s the honest number.

Until then, the money is sitting there: switch caching on, put your stable context first, and stop paying full price to re-read the same file ten times. Confirm the current cache terms against your vendor’s docs first — in this market, even the discounts get repriced.

Sources (July 2026): - Anthropic caching & batch — platform.claude.com/docs, finout - OpenAI GPT-5.6 cache terms (read −90%, write 1.25×, ~30-min) — OpenAI Help Center - Kimi / GLM cache rates — OpenRouter Kimi, docs.z.ai

Cache discounts, write costs, and lifetimes change frequently. Confirm against the vendor docs before relying on any figure here.