What "effort" and thinking budgets actually control when calling an LLM API
Two knobs get confused constantly when calling a reasoning-capable model API: how much the model thinks before answering, and how much it's allowed to produce in total. They're not the same control, and mixing them up produces either wasted spend or truncated output.
max_tokens is a hard ceiling, not a target
max_tokens caps the total output a single response can contain — thinking plus the final
answer combined, where thinking is exposed. Hit the cap mid-thought and the response is cut
off there, not gracefully wrapped up. It's an enforced limit the model isn't even aware of
while generating — setting it too low doesn't make the model more concise, it just risks
truncating a response that needed more room.
"Effort" controls how much reasoning depth to spend, not a token count
Effort (commonly exposed as low / medium / high / and higher tiers depending on the
provider) is a relative dial on how thoroughly the model reasons and how many
tool-call round-trips it takes before answering — not a specific token budget you set in
advance. Lower effort means faster, cheaper, more consolidated responses with less
exploration; higher effort means more thorough reasoning, more tool calls, and
correspondingly higher token spend. The right setting depends on the task: a simple
classification or lookup rarely benefits from high effort, while a genuinely hard,
multi-step problem often does — defaulting to the highest setting for everything just
burns tokens on tasks that didn't need it.
Adaptive thinking replaced fixed thinking budgets
Older API designs asked callers to specify a fixed thinking token budget up front — guess a number, and the model reasons up to that ceiling regardless of whether the task actually needed it:
# old pattern: guess a token budget up front
thinking={"type": "enabled", "budget_tokens": 10000}
# current pattern: let the model decide, control depth via effort instead
thinking={"type": "adaptive"}
output_config={"effort": "high"}
The adaptive form removes the need to guess a number that's either wastefully high or riskily low. This matters practically, not just conceptually: code written against the older fixed-budget pattern needs updating when migrating to a current model, since on several current model families that parameter is rejected outright (a 400 error) rather than silently ignored — it's a breaking change, not a soft deprecation.
Why this distinction matters for cost and latency
Treating effort and max_tokens as the same lever leads to two common mistakes: setting
max_tokens low to "save money" (which risks truncation, not savings — the model still
reasons as much as effort dictates, it just gets cut off before finishing) and setting
effort high by default for every request (which reliably increases cost and latency
whether or not the task needed the extra reasoning depth). The two knobs answer different
questions — max_tokens answers "what's the hard ceiling on output length," effort
answers "how hard should the model try" — and tuning only one while ignoring the other
usually means paying for depth a simple task didn't need, or truncating a hard task that
needed more room than it got.
Sources: Anthropic's Claude API documentation on extended/adaptive thinking and the effort parameter, including the deprecation of fixed thinking-token budgets in favor of adaptive thinking on current models.