A taxonomy is a graph. A classification call is one Choice. Hierarchical routing is the discipline of turning the graph into a sequence of bounded choices and then deciding which path is good enough to act on. Done well, it beats a flat classifier because every decision is small enough to explain and every branch can carry its own policy.
The failure mode is equally clear: a greedy walk commits to the first plausible label and cannot recover from an early wrong turn. The patterns below are mostly about keeping alternatives alive until the evidence narrows.
Classify at each level, not at once
The hierarchical classification cookbook turns every sibling set into one Choice question, keyed c0, c1, and so on. A node with a single option short-circuits to certainty without a call. The full probability distribution over children is what makes the search possible, because it supplies the edge probabilities a path score is built from.
That is different from asking one question with every leaf as an option. The model never has to compare a patent subclass against a retail category against a medical heading. It compares siblings, where the criteria can be narrow and the answer is a genuine choice rather than a guess about the shape of the whole tree. This is hierarchical classification as a control flow, not a single label.
Beam search keeps an early mistake recoverable
The cookbook keeps K=3 paths alive and scores each by the geometric mean of its edge probabilities, product(p) ** (1 / decisions). Length normalization matters: without it, shallow leaves would always beat deep ones. On four labeled hierarchies, beam search matched all four expected leaves while greedy matched two, recovering the correct CPC patent and Shopify product that greedy lost.
The taxonomies are real and dated: CPC 2026.05 patents, Shopify's 2026-02 product taxonomy, MeSH 2026 expanded from a DAG into tree paths, and a frozen CookSafe snapshot. Defaults are jev-1.12, beam width 3, max depth 12, and a retry policy of five attempts with exponential backoff. For hierarchies deeper than about ten levels, the cookbook recommends computing the score as exp(mean(log(probs))) to avoid precision errors.
The caveat is the reason beam search exists: greedy cannot recover an early wrong turn. The second caveat is that exploring more paths barely adds latency, because every path in the beam runs as parallel questions. The cost is input tokens, not wall-clock time. That trade is the subject of one request, many questions.
Ambiguity is a routing outcome
Not every taxonomy decision should end in a leaf. The intent routing pattern asks a Choice over four intents and a Score for complexity in the same request. Intent confidence below 0.5 goes to a human before the action is considered. order_status goes to deterministic code with no LLM involved, while complaints escalate when complexity exceeds level 1 or its confidence falls below 0.5.
The pattern's caveat is the important one: route on the pair of answer and confidence, not either alone. A confident misroute is worse than an uncertain one because it never reaches review. This is intent routing with an explicit abstain, and it is the same reason the hierarchical cookbook records a separation metric (the top path's score over its nearest rival) even though it does not prune on it.
Tool search is taxonomy routing over a catalog
jev-tool-search treats tool selection as a retrieval and ranking problem over 525 real MCP tools and 94 annotated tasks. BM25 found the right tool first 32% of the time at no model cost; embeddings reached 46% for $0.01 per 1,000 searches; two-stage Jev search reached 56% for $1.03; Voyage rerank-2.5 over the whole catalog reached 60% for $2.23; and embeddings plus the experimental Jev engine reached 59% with 92% in the top five for $0.26.
Those rows come from different runs, measured in Frankfurt through the Vercel AI Gateway on 25–26 September 2026. The README warns that with 94 tasks, differences under about ten points are noise, and a later interleaved run scored the FastMCP-style search at 54% rather than 56%. The honest conclusion is not that Jev wins tool search. It is that a cheap decision layer can sit between an embedding shortlist and an expensive reranker, and the cost/latency position may matter more than a few points of top-1.
Batch the levels when the tree is shallow
For shallow trees, separate requests per level are wasteful. The speculative fan-out pattern sends a category Choice plus four speculative Score and Noul judgments in one request, then ignores the answers the taken branch does not use. Bugs escalate at severity above 1.5 with reproducibility above 0.6; refunds flag above 0.7; frustration flags above 1.5 regardless of category.
Every speculative question adds input tokens on every call, so fan out on questions several branches can reuse. This is the practical version of the beam search trade: you pay for alternatives in tokens, not necessarily in latency.
The TypeSafe dojo shows the same arithmetic in a live demo. Its router asks three typed questions per task (model tier, ambiguity, and blast radius) and climbs from tool to haiku to sonnet to opus to fable. Triage asks about ten questions per ticket in one batched call at roughly 450 ms; the queue scene fans the same panel over eight tickets and about 80 questions in roughly 600 ms. Those latency and savings figures are the project's own live-demo readouts, not independent benchmarks, and the whole dojo is advisory: low-confidence cases are escalated rather than acted on.
Verify the rung you land on
A taxonomy route is a hypothesis about where a record belongs. The SDE cascade shows how to check the result without redoing the classification: a cheap extractor produces a record, a battery of per-field Nouls verifies it in one request, and any field above 0.7 escalates to a reasoning model. A fabricated description read 0.95 and 0.85 on two failure questions while correct fields sat between 0.02 and 0.14.
The same idea applies to a taxonomy: after the path is chosen, ask whether the leaf's definition actually fits the document, and whether a competing sibling fits better. The advanced structure page supports passing a subtree as each option's value, so the model sees what lives under a branch before committing. It also warns that subtrees grow fast; trim a branch to its direct children plus a sample of leaves to keep the request affordable.
Keep the taxonomy in code
The jaggedness list is a useful checklist for this work: counting is unreliable, so do not ask how many children match; the model reads literally, so write the boundary between siblings; and structural invariants do not hold, so do not assume probabilities across separate calls compose cleanly.
A durable design puts the taxonomy, the beam width, the thresholds, and the fallback route in code. The model supplies edge probabilities and leaf-fit judgments; code decides how far to search, when to stop, and when to hand the record to a person. That keeps the policy reviewable when the taxonomy changes, which is the only part of the system that changes often. The routing article covers the model-and-tool version of the same decision, and designing around jagged edges is the companion checklist for the failure modes.
