CommunityJev Practices & Patterns
JevCascade: Readout Stability and Inference-Time Compute Allocation for Prefill-Only Decision Models (arXiv 2610.07716)
JevCascade allocates inference-time compute for prefill-only decision models using readout stability: when only the candidate menu changes, post-intervention accuracy is already determined by the cached first-pass distribution. Menu-only interventions are predicted to within 4.2 points across six model families.
- Format
- Article
- Published by
- Community
- Last verified
- Oct 2026
Highlights
- Restricts pass-1 probabilities to the menu, renormalizes, and reads the argmax with no labels and no second pass.
- Menu-only interventions predicted to within 4.2 points across six model families, ten core datasets, and two task types.
- A probability-level variant of the estimator errs by 21.0 points, showing the property is ordinal rather than probabilistic.
- A 0.8B model on a curated 5-candidate menu reaches 95.4% on CLINC150 versus 80.0% for a 4B model on 150 labels.
- Uniform extra passes buy calibration but almost no accuracy at matched cost, per the paper.
Watch out
arXiv preprint dated October 6, 2026 (cs.CL) by Ran Li and Lei Chen, updated October 10, 2026. The 4.2-point bound is an average across the tested families and datasets; the property is ordinal and class-specific, and generative models still err by 1.6 to 15.8 points.
Questions about this entry
Short answers drawn from the linked project or page.
It scores every candidate in a menu in a single forward pass and never decodes, which makes one call one to two orders of magnitude cheaper than a same-scale generative language model.
Readout stability: when an intervention changes only the menu and leaves the input text fixed, the post-intervention accuracy is already determined by the cached first-pass distribution.
Across six model families, ten core datasets (sixteen total), and two task types, menu-only interventions are predicted to within 4.2 points, and exactly for one family.
It errs by 21.0 points, which the authors use to show the property is ordinal, not probabilistic, and it is also class-specific since same-scale generative models err by 1.6 to 15.8 points.
Uniform extra passes buy calibration but almost no accuracy, while at matched cost a confidence cascade beats every scheme that re-asks the same model, according to the paper.
Curating the menu wins in the reported comparison: a 0.8B model on a curated 5-candidate menu reaches 95.4% on CLINC150 against 80.0% for a 4B model on the full 150-label menu.
The authors say the property makes inference-time compute decidable before deployment, since the restriction predictor needs no labels and no second pass to forecast accuracy under menu-only changes.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
Facebook: Intake Router keeps diagnosis out of the answer space at the front desk.
FacebookClinicnotes
YouTube: Jev explained for routing and judging, with a comparison against LLMs.
Hacker News: Vercel on using Jev for defined decisions and GPT-6 Astra when the job also has to write or use tools.
flashbrew4 pointsTypeSafe AI Jev vs. GPT-6 AstraRead the thread on Hacker NewsHacker News: ASTGL replay chose local Laya routing over hosted Jev on one Mac Studio setup.
Jmeg8r15 pointsDoes an open-weight decision model beat a hosted one? Jev vs. LayaRead the thread on Hacker NewsRelated terms
Glossary definitions related to this entry.
More Jev resources like this
Koa-decision: Fast and Consistent Structured Decision Making with Generative LLMs (arXiv 2609.36115)
Koa-decision reduces single-step decisions such as classification, endpointing, Boolean checks, and scoring to constrained generation with single-token outputs. On a production intent-routing benchmark it reaches 85.5% accuracy in about half a second, up to roughly 7.5x faster than frontier models.
Jev Practices & PatternsArticle#community#research#arxiv
Community
An analysis of open-weight typed decision models that finds their probabilities mostly follow option labels rather than written definitions. Deleting every definition leaves laya-td accuracy unchanged (0.8559 versus 0.8487), while renaming options to A and B raises accuracy by 0.1511.
Jev Practices & PatternsArticle#community#research#arxiv
Community
A student-guided distillation pipeline for routing across 60 LLM task categories: a ModernBERT student retrieves top-k candidates and a DeBERTa-v3 zero-shot NLI teacher reranks only those. The best checkpoint reaches 77.5% teacher agreement, with Coverage@16 of 91-100%.
Jev Practices & PatternsArticle#community#research#arxiv
Community
LLM-as-Jev asks how far off-the-shelf LLMs already go as Jev-style decision models and when fine-tuning is needed. Frozen Qwen3.5-4B and Qwen3-0.6B match or exceed existing training-free readouts on the same 4B backbone, and Thinking Jev adds budgeted test-time reasoning.
Jev Practices & PatternsArticle#community#research#arxiv
Community
4GitHub stars
An analysis of ordinal-scale bias in JEV 1.13 and three open KEV models: across 36 ordinal datasets, final decisions use only 67-76% of the effective gold support versus 87-102% on four nominal tasks. BA-LoRA post-training raises utilization from about 47% to 86%.
Jev Practices & PatternsArticle#community#research#arxiv
Community
Glax147/jev_ordinal_scale_biaPython
Glax147/jev_ordinal_scale_biaPython3GitHub stars
PACT turns contrastive training pairs into four loss terms for single-token typed-decision models. On a frozen 324-item holdout it matches the published recipe's accuracy (84.6% versus 85.2%) while cutting answer flips under relabelling from 13.8% to 9.8%.
Jev Practices & PatternsArticle#community#research#arxiv
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
More than eighty preprints now evaluate, attack, and reproduce the typed-decision class. The broadest board, the coherence gap, the rejection bottleneck, and the attacks that still work.
Jev Practices & Patterns#research#evaluation#calibration
Read article
TypeSafe publishes evals, but the numbers that matter are the ones measured on your task. A tour of the independent benchmarks, field reports, and the recurring finding that question design dominates.
Jev Practices & Patterns#benchmarks#evaluation#calibration
Read article
From the Jev community
Posts from builders shipping with Jev right now.