Prompt caching vs raising effort: which lever actually saves more money

By Charles · 2 October 2026 · 5 min read

Caching is free quality-wise but small on output-heavy traffic, while effort is the larger lever. To save $1,000 a month you need either about 9,300 cacheable tokens or a single effort step.

If your bill is too high, the larger lever is almost always effort, not caching. On the reference shape — 4,000 in, 900 out, Claude Sonnet 5.5 — caching the stable prefix saves roughly $324 a month at high, while dropping from high to low saves roughly $3,780. But caching costs nothing in quality, so the order is not a choice: cache first, then move effort.

The two levers move different sides of the bill, which is why the comparison is shape-dependent rather than absolute.

What each lever actually moves

Caching moves the input side. Effort moves the output side. On the reference shape the input side is only 10% of the bill at high, so caching has a low ceiling there. Flip the shape to a long-context retrieval task and the ranking reverses, because the input portion becomes most of the bill.

The ceiling is the thing to estimate before you optimise. If input is 10% of your bill, no amount of caching saves more than 10%. If output is 90%, effort is where the money is.

What prompt caching saves

On Claude Sonnet 5.5 the three relevant rates are $2 per million for fresh input, $0.20 per million for a cache read, and $2.50 per million for a cache write. A read is 90% cheaper than fresh input. A write is 25% more expensive than fresh input.

Take the reference shape with a stable 3,000-token prefix reused on every call, and 1,000 tokens of genuinely new input:

• Without caching: 4,000 x $2 per million = $0.0080 per call.
• With a cache hit: 3,000 x $0.20 per million = $0.0006, plus 1,000 x $2 per million = $0.0020, so $0.0026 per call.

That is a saving of $0.0054 per call, or about 67% of the input cost. At 2,000 calls a day over 30 days, 60,000 calls, the monthly saving is about $324.

Notice what caching does not do: it does not change the answer. The model sees the same tokens, rendered the same way, at a lower price. That makes it the safest lever on the list, and the first one to pull.

What lowering effort saves

Now run the same arithmetic on the output side, holding the prompt fixed. From the reference shape at Claude Sonnet 5.5 rates:

Dropping from high to low saves $3,780 a month on the same 60,000 calls. That is about 11.7x the caching saving on the identical shape, and it comes from one string in the request body.

The catch is real, though. Caching is free quality-wise and effort is not. Lowering effort changes the answer, so it has to be validated against your own evals before it goes to production. That is the whole trade: a larger saving for a cost that is not measured in dollars.

Saving $1,000 a month: two routes

Suppose you need to take $1,000 a month out of this workload. Over 60,000 calls that is $0.01667 per call. Here is what each route has to do to get there.

The cache figure comes from the read discount: each cached token saves $1.80 per million, so $1,000 over 60,000 calls needs about 9,300 cached tokens. That is a large prefix. A single effort step does the same job with no prompt engineering at all, provided the quality holds.

If you have both a large prefix and slack in the effort level, you do not have to choose. Take the caching saving, then take the effort saving on top.

Cache write is a premium, not a discount

It is easy to treat caching as uniformly cheaper. It is not. The write costs more than fresh input.

• Two uncached calls on a million tokens of prefix: 2 x $2.00 = $4.00.
• One write plus one read: $2.50 + $0.20 = $2.70.

Caching therefore pays from the second reuse onward. If the prefix changes on every request, every call is a write and you pay 25% more than you would have paid without caching at all. A prefix that is rebuilt per request is the one case where caching is a straight loss.

Where the two levers interfere

The interference is a single mechanism: effort shapes the rendered prompt. Change it at the top level and the cached prefix no longer matches, so the next request is a cache miss and you pay a write again.

If you vary effort per request at the top level, you turn a workload with a 90% hit rate into a workload with a 0% hit rate. You then pay the write premium on every call, and the caching lever does nothing.

The fix is per-message effort. Setting effort on a message rather than on the request keeps everything before that message identical, so the prefix stays cached and only the new work is affected.

When caching is the bigger lever

The ranking reverses on input-heavy traffic, which is why the shape matters more than the lever. Take a retrieval task with 100,000 input tokens and a 500-token answer. Input at $2 per million is $0.20 per call. Output at high, with the modelled 8x multiplier, is 4,000 tokens at $10 per million, or $0.04. Input is 83% of the bill, and effort can only reach the other 17%.

Caching the 90,000-token prefix cuts it from $0.18 to $0.018 and leaves $0.02 of fresh input, so the input side falls from $0.20 to $0.038. The effort lever, on the same call, can only touch the $0.04 output portion. Caching is roughly 4.6x the saving here, the mirror image of the reference shape.

The rule that falls out is simple: look at the input share of your bill before you decide which lever to pull. Above roughly 50%, cache. Below it, move effort.

The decision rule

1. Cache any prefix that is reused two or more times. It is free quality-wise, so there is no reason not to.
2. Do not cache a prefix that changes on every request. You would be paying the write premium for nothing.
3. Once caching is in place, move effort. That is where the large multiples live.
4. Never vary top-level effort on a cached prefix. Use per-message effort if you need to vary it at all.
5. Re-check the ranking whenever the shape changes. Input-heavy traffic and output-heavy traffic have opposite answers.

The short version

• Caching moves the input side; effort moves the output side. Effort is bigger on output-heavy traffic.
• On the reference shape, caching saves about $324 a month and dropping to low saves about $3,780.
• A cache read is 90% cheaper than fresh input on Sonnet 5.5; a cache write is 25% more expensive.
• Caching pays off from the second reuse onward. A prefix rebuilt every call is a loss.
• To save $1,000 a month here you need roughly 9,300 cacheable tokens, but only one effort step.
• Top-level effort changes invalidate the cache. Use per-message effort instead.

Frequently asked