CommunityJev Models & Reimplementations
Laya Typed-Decisions
The fine-tuned Laya checkpoint for four typed-decision workflows: 0.766 accuracy on 2,000 held-out decisions, above Jev 1.13.0's published 0.727 and the teacher self-agreement ceiling it trained past.
- Published by
- Community
- Author
- Convai Innovations
- Last verified
- Oct 2026
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- The only Laya build fine-tuned on decision-format data: 1,200 training cases, 6,000 decisions across four synthetic workflows.
- Its 0.766 clears TypeSafe Jev 1.13.0's published 0.727 and the 0.735 teacher self-agreement ceiling on the same 2,000 decisions.
- Brier 0.062 against Jev's published 0.148, but ECE 0.213 versus 0.144 — sharper, and still over-confident.
- Router will not select it silently; auto_task_detection=True is required because it is specialised, not general.
Watch out
A specialist for invoice processing, security incidents, customer service, and agent-trace observability. Outside those four workflows the card says to expect base-model behaviour or worse, and calibrated temperatures were fitted on training data, so treat confidence as uncalibrated until you refit on your own split.
Questions about this entry
Short answers drawn from the linked project or page.
Only on this benchmark, and the card is explicit that the Jev 0.727 is a third-party published figure rather than a run made here, with different prompts and sample sizes. Laya also trails Jev where it matters for distributions: soft accuracy 0.471 against 0.580, so check the metric you care about.
The card's notebook runs the whole loop on Kaggle's free 2xT4 GPUs in roughly four to five hours: build the dataset, train, fit calibration temperatures, evaluate, then push the result back to the Hub. The training split is 1,200 cases and 6,000 decisions.
It will not select the checkpoint automatically unless you construct Router with auto_task_detection=True, because it is specialized to four synthetic workflows. You can also route explicitly with router.predict(state, questions, model='typed-decisions').
noul 0.857 (ECE 0.192, n=600), choice 0.733 (ECE 0.255, n=600) and score 0.723 (ECE 0.199, n=800) on the official test split.
Invoice processing leads at 0.804, then security incidents 0.766, customer service 0.764 and agent-trace observability 0.730.
It uses RLCD: the policy reports a distribution, exploration adds zero-mean Gaussian noise to the logits, and the reward is a strictly proper scoring rule (log plus spherical, plus a ranked probability score for ordinal questions), so expected reward is maximized only by honest probabilities. Updates are REINFORCE with a group-mean baseline plus soft cross-entropy against the teacher's distributions.
agent.predict_batch(states, questions) scores many states in shared forward passes with answers identical to one call per state, which suits workflow triage over a queue.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsRelated terms
Glossary definitions related to this entry.
More Jev resources like this
8GitHub stars
Open recipe for calibrated decision models: an LLM teacher writes training data from a one-sentence task description, a LoRA plus decision head on Qwen3-4B-Base answers Choice, Score and Noul questions with calibrated probabilities, evaluated next to Jev on identical items.
Jev Models & Reimplementations#community#calibration#training
Community
scienthoon/lucePython
scienthoon/lucePythonAn untrained readout: stock Gemma-4-31B weights plus a decider config that reads option letters at the answer slot, scored #2 of 70 on Decision Index v0.2.1 at 57.33 against Jev 1.13.0's 57.91.
Jev Models & Reimplementations#community#open-models#local
Community
877GitHub stars
A local server that reads next-token probabilities from a fine-tuned Spark model and returns Choice, Score, and Noul answers with zero generated tokens.
Jev Models & Reimplementations#community#python#open-models
Community
Rizzo-AI-Academy/rizzo-flowPython
Rizzo-AI-Academy/rizzo-flowPython58GitHub stars
A local /v1/systemone server that reads label probabilities from a GGUF chat model, with prompt repetition and a confidence shrink toward uniform.
Jev Models & Reimplementations#community#python#open-models
Community
Mushroom-Systems/lichenPython
Mushroom-Systems/lichenPythonSupersonic Labs' 144.3M-parameter decision model on mmBERT-small: choice, score, and noul answers from supplied options in one pass, 73.15% on 2,000 typed decisions, Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
Liquid AI's 3.12B decision model built on LFM2.5-VL: state plus typed questions in one forward pass with zero output tokens, 48.57 on Decision Index 0.2.1 and 8 ms per decision on an RTX 4090.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
Base decision checkpoints sit near chance on purpose. The typed-decisions loop, the corpora behind the students, and the temperature fit that turns probabilities into thresholds.
Jev Practices & Patterns#fine-tuning#calibration#open-models
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
From the Jev community
Posts from builders shipping with Jev right now.