Token Intelligence

Routing runs on every request, whether you sent "auto" or named a model. What changes is the ceiling: a named model caps how high the router may reach, and auto lets it reach as high as your plan allows.

The algorithm is versioned. This page describes ti-1.0.0, and the version changes when a signal, its test, its points or a tier threshold changes. It is deterministic for a given input on a given version: the same text and the same conversation length always produce the same score. The model preference lists are not part of the version — they move with the catalogue — so a decision is reproducible against a version and the catalogue of the day, not against the version alone.

The scorer is a heuristic over the last user message, plus the number of messages in the conversation. It is not a model call, it costs nothing, and it is deterministic — the same text always scores the same number. Points accumulate and the total is capped at 100.

The two banded signals behave differently. Length picks exactly one band. Depth is cumulative: a conversation of more than twenty messages scores both rows, for 20 points. Every signal at its top band would total 145, which is why the cap at 100 exists and why a maximal request and a merely very large one both score 100.

Two consequences are worth knowing. Only the last user message is scored, so a long conversation whose latest turn is "yes, do that" scores as a short message with a depth bonus. And the word tests are word tests: "write a haiku" picks up the creative points, and so does "create a table".

x-lobstack-complexity carries the score on every response. If a score surprises you, the fastest way to see why is to send the same text to the preview endpoint below.

It is worth being blunt about the ceiling of a heuristic this small, because the value on offer is that you can audit it rather than that it is clever.

Each tier has a maximum complexity it will accept. The router walks from nano upward and stops at the first tier whose threshold covers the score. The lead model is the first key in the tier's preference list that exists in the registry, and it is what an auto request lands on.

The cost multiplier is a relative weight, not a price. It is what x-lobstack-savings-pct is computed from: one minus the selected tier's multiplier over the ceiling tier's, floored at zero. That number is an estimate about tiers. The measured dollar figure is a different header and a different rule, described on Metering & cost.

Within a tier the order is a quality preference, not a price sort. The router takes the first model in the list, which is frequently not the cheapest one available at that tier. That is the default; the ordering can be changed, and the next section says how and what it would cost you differently.

The standard tier leads with Claude Sonnet 5 at $2.00 in and $10.00 out per million tokens, while DeepSeek V4 Flash sits in the same tier at $0.27 and $1.10. Routing standard traffic to the cheaper one would cut the bill several times over and change what answers a customer, which is a product decision rather than a cost decision.

The same reasoning shapes the flagship tier, where Opus 5 leads rather than Fable 5.1. Fable is the more capable model, and it costs twice as much. Doubling the price of every hard request without the caller asking is not a decision to make on their behalf. Name Fable when you want it.

That preference order can be replaced with a cheapest-first walk, and the switch is GATEWAY_PRICE_AWARE_ROUTING. It is off by default. Turning it on changes which model answers a paying customer, which is a decision about the product rather than a routing detail, so it is not on because it tested well.

It changes the order and nothing else. The candidate set, the availability predicate and the BYOK provider lock are untouched, so the two modes can only ever disagree about which of the same eligible models is picked — never about whether a model is eligible at all. Ties keep list order, so a decision stays reproducible.

Cheapest-first is not price alone. Sorting purely on price routed traffic to de-listed previous-generation keys that the registry keeps for callers who name them explicitly, and cheapest is not a reason to serve somebody last year’s model. Current generation first, then price.

Both columns are produced by the router’s own ordering function, called once in each mode, so this table cannot describe a sort the Gateway does not perform. Prices are blended at 3:1 input to output, and the comparison is across every managed model in the registry — every one of the 5 tiers leads with a different model under the two orderings.

A real deployment sees fewer changes than that, and the reason is worth stating rather than leaving as a footnote: a cheaper model on a provider whose key is not configured is not eligible in either mode, so it cannot be picked and cannot appear as a saving. The table above is the ceiling on what the flag can do, not a forecast of what it would do for you. What it would do for the deployment actually serving you is a question for route-preview, which narrows to the providers this deployment can reach.

A plan caps how high the router may reach, regardless of the score.

A priced plan carries its ceiling as data, so there is one ceiling mechanism rather than two tables that can disagree. Subscriptions on the older messages-per-month tiers have no priced plan and keep their own map: starter reaches standard, and pro, performance and enterprise reach flagship.

When a score exceeds the ceiling nothing fails: the request is served at the ceiling tier. A complexity-100 request on Free is answered by the standard tier, and x-lobstack-savings-pct reads 0 because the selected tier is the ceiling tier.

Two defaults are worth stating plainly. An org-level API key with no active subscription resolves to the legacy starter tier, because a fallback that is generous here spends real money. A tier that is neither a priced plan nor one of the four legacy names falls back to flagship, which is the opposite direction — it exists for deployments that are not selling priced plans at all.

Every fallback inside the selector is bounded by the ceiling too. A tier whose providers are all unkeyed does not silently fall through to something more expensive; it falls to the best keyed model at or below the ceiling.

Naming a model sets a ceiling. The effective ceiling is the lower of your model's tier and your plan's tier, and the router still walks up from nano and stops at the first tier that clears the score. You get the exact model you named only when the selected tier is that model's tier and its provider has a key configured. Otherwise you get that tier's lead model.

x-lobstack-routed is true whenever you sent auto, even when the model chosen happens to be the one you would have picked. It means "the Gateway chose this", not "the Gateway substituted".

In BYOK mode routing is additionally locked to your key's provider, because a single-vendor key cannot call another vendor. A Claude key routes among Claude models only.

This endpoint answers "where would this go?" without spending a token. It runs the same selectModel and the same price registry the real endpoint uses, so the answer is the decision rather than a simulation of it. It is unauthenticated, runs no inference, calls no provider and writes nothing.

The body accepts prompt (required, up to 8,000 characters), requested_model (default auto), plan_tier (default pro), conversation_length and expected_output_tokens. An unknown model is a 400.

plan_tier still takes the four legacy tier names rather than the plan ids in the table above. starter reproduces a standard ceiling and the other three reproduce flagship; anything else, including "free", is ignored and answered as pro. Use starter to preview what a ceiling-capped plan would do.

Token counts here are estimates at roughly four characters per token, and the response says so in token_estimate.estimated. Output length is unknowable before generation, so it defaults to half the prompt unless you supply it. The billed figure always comes from the provider's own usage block on the real request.

baseline here is the named case only: it is null when you send auto. The live endpoint does more than this one — on auto it reports a plan_ceiling baseline, labelled as such — so treat a null here as "the preview has nothing to compare" rather than as what the real request will report. The preview also quotes provider list prices in cost_usd, where the live receipt quotes the Lobstack rate. See Metering & cost.

The preview reports managed_key_configured because it runs against the whole registry while the live path narrows to providers whose key is actually set on the deployment.

The Token Intelligence page runs the same selector in your browser and shows the per-signal breakdown alongside it, so you can watch a decision decompose without writing any code.

Every routing decision is a pure function of five things, and all five are either on the response or on this page. Given a response you can re-derive the model it was served by:

The one input not carried on the response is the conversation length, because the Gateway is stateless and counts what you sent it. If a re-derivation disagrees with a header, the fastest check is to post the same message to /api/gateway/v1/route-preview and compare — it runs the same selector with no inference and no state, so a difference is a difference in the input.

Tier membership and the model preference order live in MODEL_TIERS, and the signals in COMPLEXITY_SIGNALS. Both are exported data, and the table on this page is rendered from them rather than transcribed, so this document cannot drift from the code it describes. A build check re-scores a corpus against the previous implementation and fails if any total moves.