CommunityJev Practices & Patterns
Labels Override Definitions in Jev-Style Typed Decision Models (arXiv 2610.02586)
An analysis of open-weight typed decision models that finds their probabilities mostly follow option labels rather than written definitions. Deleting every definition leaves laya-td accuracy unchanged (0.8559 versus 0.8487), while renaming options to A and B raises accuracy by 0.1511.
- Format
- Article
- Published by
- Community
- Last verified
- Oct 2026
Highlights
- Across four open-weight typed decision models, eleven classification tasks, and a synthetic routing suite, the answer is mostly the labels.
- laya-td scores 0.8559 with definitions deleted versus 0.8487 with them, though definitions alone support 0.7971.
- Renaming options to A and B raises accuracy by +0.1511 with a 95% interval of [+0.1377, +0.1646].
- Von is unaffected because it writes only the definition; laya writes each option as label plus definition.
- Changing that expression with no weight change makes laya checkpoints exactly invariant and drops von from 0.8511 to 0.2281.
Watch out
arXiv preprint dated October 1, 2026 (cs.AI) by Seyedarmin Azizi, Erfan Baghaei Potraghloo, and Massoud Pedram. The finding comes from open implementations on a Qwen2.5 backbone and the authors' own PolicyBench suite, and the abstract reports no evaluation on proprietary Jev itself.
Questions about this entry
Short answers drawn from the linked project or page.
It is the paper's term for a typed decision model preferring the option's label over its written definition when returning probabilities, which the authors observe across four open-weight models.
Accuracy stays essentially unchanged: laya-td scores 0.8559 with definitions deleted against 0.8487 with them, even though those definitions support 0.7971 on their own.
Renaming the options to A and B raised accuracy by +0.1511, with a 95% interval of [+0.1377, +0.1646], further evidence that the label is what moves the decision.
The two code bases differ in one expression: laya writes each option as label and definition together, while von writes only the definition, so von's prompts carry no competing label text.
No. Changing the option expression in both directions with no weight changed made all three laya checkpoints exactly invariant and created the effect in von, whose accuracy fell from 0.8511 to 0.2281 when a label contradicted its definition.
Earlier work attributed it to the constrained decision head these models use instead of a text decoder; these results locate the cause in the prompt rendering of label and definition.
The paper provides a two-call test that tells a practitioner which case applies to their model and measures what four mitigations are worth, according to the abstract.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
Facebook: Intake Router keeps diagnosis out of the answer space at the front desk.
FacebookClinicnotes
YouTube: Jev explained for routing and judging, with a comparison against LLMs.
Hacker News: Vercel on using Jev for defined decisions and GPT-6 Astra when the job also has to write or use tools.
flashbrew4 pointsTypeSafe AI Jev vs. GPT-6 AstraRead the thread on Hacker NewsHacker News: ASTGL replay chose local Laya routing over hosted Jev on one Mac Studio setup.
Jmeg8r15 pointsDoes an open-weight decision model beat a hosted one? Jev vs. LayaRead the thread on Hacker NewsMore Jev resources like this
JevCascade allocates inference-time compute for prefill-only decision models using readout stability: when only the candidate menu changes, post-intervention accuracy is already determined by the cached first-pass distribution. Menu-only interventions are predicted to within 4.2 points across six model families.
Jev Practices & PatternsArticle#community#research#arxiv
Community
A student-guided distillation pipeline for routing across 60 LLM task categories: a ModernBERT student retrieves top-k candidates and a DeBERTa-v3 zero-shot NLI teacher reranks only those. The best checkpoint reaches 77.5% teacher agreement, with Coverage@16 of 91-100%.
Jev Practices & PatternsArticle#community#research#arxiv
Community
Koa-decision: Fast and Consistent Structured Decision Making with Generative LLMs (arXiv 2609.36115)
Koa-decision reduces single-step decisions such as classification, endpointing, Boolean checks, and scoring to constrained generation with single-token outputs. On a production intent-routing benchmark it reaches 85.5% accuracy in about half a second, up to roughly 7.5x faster than frontier models.
Jev Practices & PatternsArticle#community#research#arxiv
Community
LLM-as-Jev asks how far off-the-shelf LLMs already go as Jev-style decision models and when fine-tuning is needed. Frozen Qwen3.5-4B and Qwen3-0.6B match or exceed existing training-free readouts on the same 4B backbone, and Thinking Jev adds budgeted test-time reasoning.
Jev Practices & PatternsArticle#community#research#arxiv
Community
4GitHub stars
An analysis of ordinal-scale bias in JEV 1.13 and three open KEV models: across 36 ordinal datasets, final decisions use only 67-76% of the effective gold support versus 87-102% on four nominal tasks. BA-LoRA post-training raises utilization from about 47% to 86%.
Jev Practices & PatternsArticle#community#research#arxiv
Community
Glax147/jev_ordinal_scale_biaPython
Glax147/jev_ordinal_scale_biaPythonAn evaluation of Jev and two open-weight typed decision models when option names and definitions conflict. With yes/no names, decision-flip rates are up to 70.4 percentage points higher than the 0/1 control, and mean AUC drops from 93.8% to a below-chance 23.2%.
Jev Practices & PatternsArticle#community#research#arxiv
Community
From the Jev guides
Original write-ups on what this entry is for.
More than eighty preprints now evaluate, attack, and reproduce the typed-decision class. The broadest board, the coherence gap, the rejection bottleneck, and the attacks that still work.
Jev Practices & Patterns#research#evaluation#calibration
Read article
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
A Choice can only pick from its options. How rejection options, candidate coverage, and escalation bands keep a narrow taxonomy honest under inputs it never anticipated.
Jev Practices & Patterns#rejection#confidence#patterns
Read article
From the Jev community
Posts from builders shipping with Jev right now.