CommunityJev Practices & Patterns
Koa-decision: Fast and Consistent Structured Decision Making with Generative LLMs (arXiv 2609.36115)
Koa-decision reduces single-step decisions such as classification, endpointing, Boolean checks, and scoring to constrained generation with single-token outputs. On a production intent-routing benchmark it reaches 85.5% accuracy in about half a second, up to roughly 7.5x faster than frontier models.
- Format
- Article
- Published by
- Community
- Last verified
- Oct 2026
Highlights
- Atomic label tokens plus supervised fine-tuning reduce classification to a deterministic one-step decoding problem.
- Reaches 85.5% accuracy on a production intent-routing benchmark, competitive with Claude-4.8-Opus and Gemini-Pro-3.1.
- Answers in about half a second, several-fold faster than every frontier model, up to ~7.5x at the median.
- Against Jev/TypeSafe it is competitive on accuracy and faster at the median, per the abstract.
- Handles multimodal inputs and multi-label outputs that single-label text systems do not.
Watch out
arXiv preprint dated September 28, 2026 (cs.LG) by Shenghong Dai and co-authors; v2 renamed Koa-action to Koa-decision. The headline numbers come from the authors' production intent-routing benchmark under their serving conditions, and the abstract announces no open release.
Questions about this entry
Short answers drawn from the linked project or page.
Low-latency atomic actions: fast, single-step decisions such as classification, semantic endpointing, Boolean checks, and scoring, formulated as constrained generation with single-token outputs.
It introduces atomic label tokens and applies supervised fine-tuning, reducing classification to a deterministic one-step decoding problem instead of the verbose multi-token outputs that prompting and constrained decoding produce.
On a production intent-routing benchmark, Koa-decision reaches 85.5% accuracy, described as competitive with Claude-4.8-Opus and Gemini-Pro-3.1 and ahead of GPT-5 and Gemini-2.5-Pro.
It answers in about half a second, several-fold faster than every frontier model under identical serving conditions, up to roughly 7.5x at the median, according to the abstract.
Against the dedicated single-token system Jev/TypeSafe, Koa-decision is competitive on accuracy and faster at the median, while also handling multimodal inputs and multi-label outputs that single-label text systems do not.
Encoder-based models achieve faster inference but sacrifice task flexibility, while prompting and constrained decoding keep flexibility but produce verbose outputs that require expensive token-by-token generation.
The abstract describes the framework and its results but does not announce a code release, so availability of the implementation is not established by this paper alone.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
Facebook: Intake Router keeps diagnosis out of the answer space at the front desk.
FacebookClinicnotes
YouTube: Jev explained for routing and judging, with a comparison against LLMs.
Hacker News: Vercel on using Jev for defined decisions and GPT-6 Astra when the job also has to write or use tools.
flashbrew4 pointsTypeSafe AI Jev vs. GPT-6 AstraRead the thread on Hacker NewsHacker News: ASTGL replay chose local Laya routing over hosted Jev on one Mac Studio setup.
Jmeg8r15 pointsDoes an open-weight decision model beat a hosted one? Jev vs. LayaRead the thread on Hacker NewsMore Jev resources like this
JevCascade allocates inference-time compute for prefill-only decision models using readout stability: when only the candidate menu changes, post-intervention accuracy is already determined by the cached first-pass distribution. Menu-only interventions are predicted to within 4.2 points across six model families.
Jev Practices & PatternsArticle#community#research#arxiv
Community
An analysis of open-weight typed decision models that finds their probabilities mostly follow option labels rather than written definitions. Deleting every definition leaves laya-td accuracy unchanged (0.8559 versus 0.8487), while renaming options to A and B raises accuracy by 0.1511.
Jev Practices & PatternsArticle#community#research#arxiv
Community
A student-guided distillation pipeline for routing across 60 LLM task categories: a ModernBERT student retrieves top-k candidates and a DeBERTa-v3 zero-shot NLI teacher reranks only those. The best checkpoint reaches 77.5% teacher agreement, with Coverage@16 of 91-100%.
Jev Practices & PatternsArticle#community#research#arxiv
Community
LLM-as-Jev asks how far off-the-shelf LLMs already go as Jev-style decision models and when fine-tuning is needed. Frozen Qwen3.5-4B and Qwen3-0.6B match or exceed existing training-free readouts on the same 4B backbone, and Thinking Jev adds budgeted test-time reasoning.
Jev Practices & PatternsArticle#community#research#arxiv
Community
4GitHub stars
An analysis of ordinal-scale bias in JEV 1.13 and three open KEV models: across 36 ordinal datasets, final decisions use only 67-76% of the effective gold support versus 87-102% on four nominal tasks. BA-LoRA post-training raises utilization from about 47% to 86%.
Jev Practices & PatternsArticle#community#research#arxiv
Community
Glax147/jev_ordinal_scale_biaPython
Glax147/jev_ordinal_scale_biaPython3GitHub stars
PACT turns contrastive training pairs into four loss terms for single-token typed-decision models. On a frozen 324-item holdout it matches the published recipe's accuracy (84.6% versus 85.2%) while cutting answer flips under relabelling from 13.8% to 9.8%.
Jev Practices & PatternsArticle#community#research#arxiv
From the Jev guides
Original write-ups on what this entry is for.
More than eighty preprints now evaluate, attack, and reproduce the typed-decision class. The broadest board, the coherence gap, the rejection bottleneck, and the attacks that still work.
Jev Practices & Patterns#research#evaluation#calibration
Read article
A Choice can only pick from its options. How rejection options, candidate coverage, and escalation bands keep a narrow taxonomy honest under inputs it never anticipated.
Jev Practices & Patterns#rejection#confidence#patterns
Read article
Most Jev accuracy problems are state problems. What belongs in a state, what is a pre-made judgment to leave out, and why naming the slice you ask about matters more than the wording.
Jev Practices & Patterns#state#design#primitives
Read article
From the Jev community
Posts from builders shipping with Jev right now.