CommunityJev Practices & Patterns
Student-Guided Teacher Distillation for Efficient LLM Task Routing: Positioning Against Jev-Style System-1 Classifiers (arXiv 2610.02516)
A student-guided distillation pipeline for routing across 60 LLM task categories: a ModernBERT student retrieves top-k candidates and a DeBERTa-v3 zero-shot NLI teacher reranks only those. The best checkpoint reaches 77.5% teacher agreement, with Coverage@16 of 91-100%.
- Format
- Article
- Published by
- Community
- Last verified
- Oct 2026
Highlights
- A zero-shot NLI router scores one premise-hypothesis pair per label, so cost scales linearly with taxonomy size.
- ModernBERT predicts the full category distribution in one pass and retrieves a top-k set that DeBERTa-v3 reranks.
- Best student checkpoint reaches 77.5% teacher agreement on a 200-example evaluation set.
- Preliminary coverage measurements show Coverage@16 of 91-100%.
- Truncated top-k teacher scores should not be treated as full 60-class soft targets, because zeroing classes biases KL distillation.
Watch out
arXiv preprint dated October 1, 2026 (cs.LG, cs.AI), 13 pages, by Haifeng Wu and co-authors. Coverage numbers are preliminary, and the abstract states that a complete evaluation, including coverage at multiple k and a larger human-reviewed test set, remains in progress.
Questions about this entry
Short answers drawn from the linked project or page.
Routing user requests to specialized LLM tasks with zero-shot classifiers is expensive because a zero-shot NLI classifier must evaluate one premise-hypothesis pair per label, so cost scales linearly with taxonomy size.
A compact ModernBERT classifier predicts the full distribution over 60 task categories in one forward pass and retrieves a top-k candidate set, which a larger DeBERTa-v3 zero-shot NLI classifier reranks instead of scoring all 60 labels.
Teacher labels iteratively improve the student, which then produces sharper candidates for the next round; the candidate generator is trained end-to-end on the target taxonomy and is the same model serving production traffic.
The best student checkpoint reaches 77.5% teacher agreement on a 200-example evaluation set, and preliminary coverage measurements show Coverage@16 of 91-100%.
Truncated top-k teacher scores should not be treated as full 60-class soft targets for KL distillation: zeroing untruncated classes destroys the dark knowledge soft-label distillation depends on, introducing systematic bias rather than a harmless sparse approximation.
The candidate generator is trained end-to-end on the target taxonomy, unlike generic embedding retrieval or clustering-derived shortlists, and the paper distinguishes itself from LLM-routing work that routes between candidate models.
A complete evaluation remains in progress, including coverage at multiple k on a held-out set, an embedding-retrieval baseline, and a larger human-reviewed test set, according to the abstract.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
Facebook: Intake Router keeps diagnosis out of the answer space at the front desk.
FacebookClinicnotes
YouTube: Jev explained for routing and judging, with a comparison against LLMs.
Hacker News: Vercel on using Jev for defined decisions and GPT-6 Astra when the job also has to write or use tools.
flashbrew4 pointsTypeSafe AI Jev vs. GPT-6 AstraRead the thread on Hacker NewsHacker News: ASTGL replay chose local Laya routing over hosted Jev on one Mac Studio setup.
Jmeg8r15 pointsDoes an open-weight decision model beat a hosted one? Jev vs. LayaRead the thread on Hacker NewsMore Jev resources like this
JevCascade allocates inference-time compute for prefill-only decision models using readout stability: when only the candidate menu changes, post-intervention accuracy is already determined by the cached first-pass distribution. Menu-only interventions are predicted to within 4.2 points across six model families.
Jev Practices & PatternsArticle#community#research#arxiv
Community
An analysis of open-weight typed decision models that finds their probabilities mostly follow option labels rather than written definitions. Deleting every definition leaves laya-td accuracy unchanged (0.8559 versus 0.8487), while renaming options to A and B raises accuracy by 0.1511.
Jev Practices & PatternsArticle#community#research#arxiv
Community
Koa-decision: Fast and Consistent Structured Decision Making with Generative LLMs (arXiv 2609.36115)
Koa-decision reduces single-step decisions such as classification, endpointing, Boolean checks, and scoring to constrained generation with single-token outputs. On a production intent-routing benchmark it reaches 85.5% accuracy in about half a second, up to roughly 7.5x faster than frontier models.
Jev Practices & PatternsArticle#community#research#arxiv
Community
LLM-as-Jev asks how far off-the-shelf LLMs already go as Jev-style decision models and when fine-tuning is needed. Frozen Qwen3.5-4B and Qwen3-0.6B match or exceed existing training-free readouts on the same 4B backbone, and Thinking Jev adds budgeted test-time reasoning.
Jev Practices & PatternsArticle#community#research#arxiv
Community
A study of option-order sensitivity in System One models and a training-free fix called Runoff. It scores each option alone to keep a few candidates, then compares them with each candidate in every position, removing order dependence in two forward passes.
Jev Practices & PatternsArticle#community#research#arxiv
Community
Documents a failure mode in letter-bound remote sensing classification: 97.71% OA on UCM in vocabulary but 7.35%–96.40% cross-dataset accuracy and inverted confidence (AUROC 0.38). RSAJEV replaces the letter readout with training-free prototype retrieval, reaching 89.43% on the UCM cross-dataset gate.
Jev Practices & PatternsArticle#community#research#arxiv
Community
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
More than eighty preprints now evaluate, attack, and reproduce the typed-decision class. The broadest board, the coherence gap, the rejection bottleneck, and the attacks that still work.
Jev Practices & Patterns#research#evaluation#calibration
Read article
A Choice can only pick from its options. How rejection options, candidate coverage, and escalation bands keep a narrow taxonomy honest under inputs it never anticipated.
Jev Practices & Patterns#rejection#confidence#patterns
Read article
From the Jev community
Posts from builders shipping with Jev right now.