Reserve Before Dispatch: Budgets That Fire Before Inference

Most LLM budget tooling is a notification system. The request runs, the provider bills, the dashboard updates, and at some threshold an alert informs you that money has been spent. By the time a human reads it, the spend is a historical fact. Alerts are accounting. They are not control.

The alternative is positional: enforce the budget before inference, in the request path, while refusal is still free. nRouter's documentation of LLM budget management describes the mechanics in three verbs — reserve, dispatch, settle — and the ordering is the entire point.

Reserve, dispatch, settle

Before a request goes to a provider, the gateway reserves an estimated amount against a budget. If the reservation would exceed the ceiling, the request is refused before any provider spend is authorized — a hard refusal, not a warning. If it fits, the request dispatches, and afterward the actual usage settles against the reservation, so the ledger reflects what was really spent rather than the estimate.

This is the same pattern payment systems have used for decades — authorization holds before capture — applied to inference. It converts the budget from a reporting threshold into an invariant the system maintains on every call.

Scopes: organization, team, key, request

Ceilings are set at four scopes: organization, team, key, and request. The scopes answer different failure modes. Organization ceilings bound total exposure. Team ceilings make a budget somebody's property — the precondition for anyone optimizing it. Key ceilings contain a leaked or misused key's blast radius to its allocation. Request ceilings stop a single pathological call — a huge context, a runaway generation — from consuming a team's budget in one shot.

The agent case

Agent loops are where post-hoc budgets fail hardest, because the loop's spend rate is machine-speed. A stuck agent can burn a meaningful budget between two dashboard refreshes. The documented protections are per-run: maximum steps and token budgets per run, alongside the scope ceilings — the loop hits its own wall before it reaches the team's.

What to demand from your current stack

Ask three questions of whatever routes your traffic today. Can a request be refused before provider spend exists? Is the refusal based on a reservation, or on a counter that updates after the fact? And can a single key, team, or run be bounded independently? If the answers are no, after-the-fact, and no, then what you have is spend visibility. Visibility is worth having — the per-request ledger (model, tokens, cost, latency, retries, outcome, attributed to team and key) is how cost reviews get honest. But visibility plus enforcement is the difference between watching a budget and having one.

Disclosure: I work with the nRouter team. Mechanics described are as documented on the linked page.