CommunityJev Models & Reimplementations
Darwin-27B-ZTC
FINAL-Bench's zero-token classifier: one forward pass plus a fitted temperature produce a full distribution over options, scoring 0.743 on the 2,000-decision typed-decisions test split.
- Published by
- Community
- Author
- FINAL-Bench
- Last verified
- Oct 2026
Used inJev for Software & DevTools
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- 0.743 accuracy, Brier 0.097, and KL 0.204 on the typed-decisions test split, with noul reaching 0.845.
- Inference code is vendored in the repository under autojev/, with a patch for the text-only backbone.
- The headline accuracy comes from the second run — the one with saved predictions, the card says.
- The scorer reproduced the benchmark's Uniform reference (KL 0.444, Brier 0.238) as its validation.
Watch out
About 54 GB of BF16 weights and a Transformers build with Qwen3.5 support are required, and no inference provider deploys it. Two identical-settings runs disagreed on 14 of 2,000 decisions, so near-ties can flip with request batching.
Questions about this entry
Short answers drawn from the linked project or page.
No. It reads a state plus typed questions and returns a full probability distribution per question in one forward pass, with zero generated tokens; the repository's readout head and a fitted scalar temperature produce the scores.
Roughly 54 GB free for BF16 weights, plus torch, transformers, safetensors, and pillow. It takes the POST /v1/systemone request shape with noul, choice, and score questions, but it is not deployed by any inference provider.
On the LocalLLaMA/typed-decisions test split (400 cases, 2,000 decisions) it scores zero-shot 0.743 accuracy, KL 0.204 from gold, and Brier 0.097, with noul at 0.845, choice 0.732, and score 0.675.
Only within noise: the card ran the test set twice with the same settings and got 0.741 and 0.743. The two runs disagreed on 14 of 2,000 decisions, which the card attributes to server batching of concurrent requests flipping near-ties.
Options are listed in the prompt with short codes. A readout head maps the final hidden state of one forward pass to one score per option, a softmax over the scores gives the distribution, and a fitted scalar temperature calibrates it.
Apache-2.0 for the model. The inference code vendored in the repository under autojev/ is MIT, with its copyright notice in autojev/LICENSE, and support for a text-only backbone was added on top.
No. It is zero-shot on the test split, never seeing its four workflows or question schemas, and training used the public training and development splits of public decision benchmarks only, with no benchmark test split.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsMore Jev resources like this
877GitHub stars
A local server that reads next-token probabilities from a fine-tuned Spark model and returns Choice, Score, and Noul answers with zero generated tokens.
Jev Models & Reimplementations#community#python#open-models
Community
Rizzo-AI-Academy/rizzo-flowPython
Rizzo-AI-Academy/rizzo-flowPython58GitHub stars
An Apache-2.0 decision engine with a TypeSafe-compatible endpoint, document answers with source passages, and a ViZDoom demo that picks actions from structured game state.
Jev Models & Reimplementations#community#python#local
Community
deepanwadhwa/OpenDecisionPython
deepanwadhwa/OpenDecisionPython58GitHub stars
A local /v1/systemone server that reads label probabilities from a GGUF chat model, with prompt repetition and a confidence shrink toward uniform.
Jev Models & Reimplementations#community#python#open-models
Community
Mushroom-Systems/lichenPython
Mushroom-Systems/lichenPythonThe fine-tuned Laya checkpoint for four typed-decision workflows: 0.766 accuracy on 2,000 held-out decisions, above Jev 1.13.0's published 0.727 and the teacher self-agreement ceiling it trained past.
Jev Models & Reimplementations#community#open-models#local
Community
Supersonic Labs' 144.3M-parameter decision model on mmBERT-small: choice, score, and noul answers from supplied options in one pass, 73.15% on 2,000 typed decisions, Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
The vision sibling: the same text-trained System 1 head on Qwen3.8-27B's image encoder, taking images zero-shot, ranked first of 20 vision decision models on the community Jev Decision Index 0.3.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Hundreds of repositories carry the system-one tag, but the ecosystem is a dozen ideas in different sizes. A tour of the families, the licenses, and what a download does not include.
Jev Models & Reimplementations#open-models#hugging-face#local
Read article
From the Jev community
Posts from builders shipping with Jev right now.