Jev intent routing: send each request to the right handler

Jev intent routing saves money mostly by not calling a model at all. In TypeSafe's own routing example one of the four customer-service intents, order_status, goes to a database lookup with no language model in the path; the expensive handlers are only reached after a cheap typed classifier has decided they are worth paying for. This post builds that router end to end.

What follows is the question design, the thresholds, the three fallbacks most teams forget, and the arithmetic that tells you whether the router pays for itself. Every figure comes from TypeSafe's documentation or from a third-party benchmark, both linked on the number.

What jev intent routing does, and what it replaces

Jev is TypeSafe's decision model. You send it a state and a set of typed questions, and it returns a choice, a score or a probability with a confidence value attached, rather than prose your code has to parse. TypeSafe describes three question types: Choice, Score and Noul, all of which can be mixed in one call and are evaluated in parallel against the same state.

A router built on that is a single call at the front of your request path. It answers "which handler should take this?" and "how sure are you?", and your own code does the branching. Nothing about the model chooses the next step: TypeSafe is explicit that code owns the control flow and the model only supplies narrow judgments.

What it replaces is the classifier prompt: a general-purpose model asked to read a ticket and return JSON naming the intent. That works until the JSON does not parse, or the model invents a fifth category, or you need to know how confident it was and it tells you "high". We compared the two model classes properly in our post on when a decision model beats an LLM; the short version is that a constrained answer space is the feature, not the limitation.

The worked router: four intents, three kinds of handler

Take the customer-service case from TypeSafe's intent routing pattern. Four intents come in. They do not all deserve the same handler.

Intent	Handler	What decides
order_status	Deterministic code, no model	Intent confidence alone
product_question	Product specialist LLM	Intent confidence alone
return_exchange	Returns specialist LLM	Intent confidence alone
complaint	Complaint LLM or a person	A second question: complexity

That table is the whole design. One branch needs no AI. Two need different context loaded into different prompts, which is the real reason to split them. One needs a second signal before you dare automate it.

Question design: two questions, not one

The instinct is to ask "what should we do with this ticket?" and let the model work it out. TypeSafe's build guide calls decomposition the most important idea in the guide, on the grounds that broad questions hide several judgments behind one answer. So the router asks two:

{
  "intent": {
    "type": "choice",
    "instructions": "The primary intent of this customer message",
    "criteria": {
      "order_status": "Asking about an existing order",
      "product_question": "Asking about a product before buying",
      "return_exchange": "Wants to return or exchange something",
      "complaint": "Unhappy with experience, wants resolution"
    }
  },
  "complexity": {
    "type": "score",
    "instructions": "How complex is this request to resolve",
    "criteria": [
      "Simple lookup or standard procedure",
      "Requires some judgment or multi-step process",
      "Unusual situation, edge case, or escalation needed"
    ]
  }
}

Two things about the criteria are worth copying. The Choice options are written as what the customer is doing, not as department names, because the model is matching a message against a description. And the Score levels are ordered descriptions rather than numbers, so the rubric is legible to the person tuning it six months later.

When two options keep getting confused, TypeSafe's advanced guidance is to make each option an object carrying what it covers, not_for what belongs to a neighbouring option, and a couple of examples, using the same field names across options so the model can compare them directly. In the systems we build, that structure fixes more misroutes than any amount of rewriting the top-level instruction.

Thresholds: one floor, then one per action

There is no single correct confidence threshold, and TypeSafe's own examples do not use one. The intent routing pattern sends anything under 0.5 intent confidence to a human. The confidence-gated routing pattern uses a 0.6 floor for a voice banking interface, then demands above 0.85 before approving a transfer without asking the customer to confirm. The build guide's worked triage gates topic confidence at 0.75.

The pattern behind the numbers: a floor that catches genuine uncertainty, then a higher bar for each action in proportion to what it costs to get wrong. Showing the wrong screen is recoverable. Moving money is not.

One detail that trips people up. Confidence is derived from the probability distribution across the options, and TypeSafe notes that Noul answers do not carry one. A Noul returns a probability between 0 and 1 that a statement is true. So when the docs' triage code tests refund_requested.noul >= 0.7, that is a probability threshold, not a confidence threshold, and it means something different: the model is fairly sure the statement is true, not that the model is sure of itself. Mixing the two up in the same if block is a bug you will not see in a test.

Fallbacks: there are three, and most routers have one

Low confidence sends the request to a person. That is the fallback everybody writes. Two more matter as much.

1. A confident answer you still should not act on. In the intent routing example, complaint does not go straight to the complaint LLM. It checks the complexity score, and escalates to a human if complexity is above 1 or if the complexity answer's own confidence is below 0.5. An uncertain complexity reading is treated exactly like a complex case.
2. The call that does not return. TypeSafe publishes rate limits of 250,000 tokens per second and 1,200 requests per minute, and returns 429 over either; its models page warns those limits can change without notice while it works through demand. The SDKs retry with backoff, but you need a decision for the request that still fails: a default route, usually to the handler with the widest competence, not a stack trace in front of a customer.
3. The model moving under you. jev-latest is an alias that currently resolves to jev-1.13.0 and will move when a release ships. If you tuned thresholds against a version, pin the versioned ID and migrate deliberately. A threshold calibrated on one set of weights is not calibrated on another.

Where the cost saving actually comes from

Three separate savings get bundled together under "routing is cheaper", and they behave differently.

The requests that reach no model. This is the largest and the least discussed. If a third of your inbound volume is order status, that third becomes a SQL query and a template. No token is billed after the classifier.

One call instead of several. TypeSafe's parallel questions cookbook runs 13 questions over the Wikipedia article on the GDPR, about 54,000 characters. Batched into one request the run averages $0.000497 and 0.27 seconds; as 13 single-question calls it averages $0.006090 and 2.71 seconds, which the cookbook reports as 12.2x cheaper and 10.0x faster with no change to the answers. The document dominates every request, so sending it once instead of thirteen times is where that goes.

The price gap between the classifier and the handler. Jev lists at $0.042 per million input tokens with output free. For a like-for-like comparison, Benchmark Heaven's JevBench v1.3.0, scored 21 September 2026 across 534 decisions, publishes a measured cost per 1,000 decisions on the same workload: $0.040 for Jev 1.13.0 against $0.242 for GPT-5.6 Luna (low), $0.264 for Gemini 3.1 Flash-Lite and $0.594 for DeepSeek V4.1 Flash. Those are the benchmark operator's own measurements on its own tasks, not vendor list prices, and they describe the configurations it tested.

Here is the part that decides whether your router is worth building. It adds a call to every request, including the ones that go to the expensive handler anyway. Using the JevBench figures, the classifier costs about 17% of what one frontier-model decision costs on the same task ($0.040 against $0.242). So the router breaks even only once it deflects roughly one request in six away from the expensive path. Below that, you have added latency and a dependency for nothing.

Measure the deflection rate before you promise anyone a number. The same arithmetic governs the broader question of which model runs which traffic, which we looked at in our breakdown of how open-weight models take most of the tokens and a fraction of the spend.

Ask the speculative questions too

The instinct from LLM work is to ask the minimum. With a model that ingests the state once and evaluates every question against it in parallel, that instinct is wrong. TypeSafe's speculative fan-out pattern recommends putting every question the system might need into one request and letting code discard the irrelevant answers.

Its smart-home demo makes the point concrete: for "turn off all of the lights in the house" the system asks what action should be taken on the lights before it knows the request concerns lights at all. Asking it later would mean a second round trip for a question that cost almost nothing to ask early.

The budget is the real constraint, not the question count. TypeSafe documents 64,000 tokens per request covering the state plus all questions combined, and 32,000 for the state plus the single longest question. Jev takes text only; images, audio and video have to be turned into text or structured fields before they become state.

Route the branches that need prose to a language model and keep the typed decision in front of it. That split, rather than a choice between the two, is what most production systems end up with, and it is the same separation of duties we argue for in routing model traffic through a gateway instead of handing out API keys.

When a router is the wrong thing to build

Give the counter-argument its due, because it wins more often than vendor documentation suggests.

If the classification is a rule, write the rule. TypeSafe's own build guide opens with an invoice that goes to collections when it is more than 30 days overdue, in four lines of Python, and says to keep deterministic work in code because it is reliable and cheap. A router in front of a rule is a liability with a monthly bill.

If there are two intents and a keyword match separates them, the keyword match is faster, free and auditable. If every branch ends at the same handler with a different system prompt, you have a prompt-selection problem, not a routing problem. And if your traffic is not in English, test first: TypeSafe states that English is its primary training language and where accuracy is currently best, that other languages including CJK scripts are handled but not equally well, and that you should pay close attention to confidence when routing them.

One more limit worth stating plainly. A typed classifier makes a fast judgment; it does not reason through multi-step arithmetic. A question that requires chaining several inferences should be split into several questions and combined in your code, or handed to a reasoning model. Asking a decision model to do arithmetic and then trusting its confidence is how routers acquire a reputation for being confidently wrong.

What to do first

Do not tune thresholds on your intuition. Log the chosen intent, the full probability distribution, the handler that ran and whether the outcome needed rework, for two weeks of real traffic. TypeSafe's guidance is to test thresholds by plotting confidence against accuracy on your own data, and the plot usually shows that one intent is carrying all the misroutes.

Then fix that intent's criteria rather than lowering the floor for everything. If you want a second pair of eyes on the design before it ships, that kind of review is a normal part of our AI engineering work, and the interesting argument is almost always about which branch should never reach a model at all.