CommunityJev Models & Reimplementations
AutoTrust GEV-26B-Decide
AutoTrust's Gemma-based decision model with adaptive thinking: System 1 answers below 0.8 confidence hand off to the full Gemma 4 26B and mix the two by half, behind POST /v1/decide.
- Published by
- Community
- Author
- AutoTrust AI Lab
- Last verified
- Oct 2026
Used inJev for Software & DevTools
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- Adaptive thinking picks System 2 when System 1 confidence drops below 0.8 and returns p = ½p₁ + ½p₂.
- Thinking lifts GPQA Diamond 42.9 to 78.6 and BBH 75.0 to 92.0; HLE barely moves, 8.4 to 17.8.
- Its 62.48 is AutoTrust's own run of the 0.2.1 scoring kit — not a leaderboard entry, because adaptive mode misses the index latency cap.
- Two calibration files: calibration.json for the index, calibration_gold.json with a noul temperature of 1.214 for gating real actions.
Watch out
The LoRA-on-Gemma-4 path needs a bundled vLLM patch and one 80 GB GPU. Thinking can take a median 13.4 seconds on knowledge tasks, it hurts classification (BANKING77 falls 88.0 to 85.0), and the model trained on some public training splits whose test splits the index uses.
Questions about this entry
Short answers drawn from the linked project or page.
No. The card shows gains on GPQA, MMLU-Pro, BBH, and CRUXEval, no benefit on chess, and a regression on classification, where BANKING77 falls from 88.0 to 85.0. Adaptive mode also cannot meet the Decision Index latency limit.
calibration.json is the default used for the index, but the card points at calibration_gold.json — calibrated against ground truth — when you gate automatic actions. Above the decision card it also recommends top_k 0 and top_p 1.0 when calling completions directly.
hf download the repo, then bash serve.sh runs a vLLM server on :8000 with a POST /v1/decide route. It needs one GPU with 80 GB or more and a vLLM build with Gemma-4 support plus the bundled lm_head LoRA patch.
System 1 decides in one forward pass at a median 45 ms on a B200, with 257 decisions per second across 64 concurrent clients. Adaptive thinking adds about 1 second per 250 thinking tokens when it fires.
It takes kind (noul, score, or choice), a state, one question, options for choice (2-256 strings), and thinking set to off, auto, or on. The response returns options, probabilities, choice, choice_index, and usage.
For visual control it is used as System 1: 95% of 60 computer-use tasks at about 85 ms per click, but only when given element text. With numbered boxes alone it completes 15% and declares the task complete too early.
The backbone's native context is 262,144 tokens, tested to 128K. Choice takes 2-256 options, read 16 at a time in a tournament by default, so every option is scored and none is pruned.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsMore Jev resources like this
An open distillation of Jev 1.13 onto Qwen3.8-27B: a 108.9M-parameter decision block answers typed questions in one prefill pass and adds a POST /v1/decide route beside a full System 2.
Jev Models & Reimplementations#community#open-models#local
Community
The 9B member of AutoTrust's Jev students: a 40.2M-parameter decision block on Qwen3.5-9B with 0.019 mean KL to Jev 1.13 and 70.7% HumanEval left untouched, small enough for one modest GPU.
Jev Models & Reimplementations#community#open-models#local
Community
GGUF builds of the Jev-Style v3 decision model on Qwen3.5-0.8B: a 0.53 GB Q4_K_M file the card says is decision-identical to FP32 on 240/240 parity rows, served through a pinned llama.cpp scorer.
Jev Models & Reimplementations#community#open-models#local
Community
A 400M-parameter Jev-compatible decision model with a browser build: 0.796 on typed decisions against Laya's 0.766, 17 ms per question on an L40S, and an ONNX file that runs in a tab.
Jev Models & Reimplementations#community#open-models#local
Community
1.3kGitHub stars
A local yes/no decision model that speaks Jev's wire format on a laptop CPU, refusing Choice and Score until those question types ship.
Jev Models & Reimplementations#community#python#open-models
Community
feder-cr/jevC++
feder-cr/jevC++8GitHub stars
A Python server in front of llama-server that reads one-token label probabilities and exposes them as POST /v1/systemone.
Jev Models & Reimplementations#community#python#open-models
Community
khimaros/verdictPython
khimaros/verdictPythonFrom the Jev guides
Original write-ups on what this entry is for.
Base decision checkpoints sit near chance on purpose. The typed-decisions loop, the corpora behind the students, and the temperature fit that turns probabilities into thresholds.
Jev Practices & Patterns#fine-tuning#calibration#open-models
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
From the Jev community
Posts from builders shipping with Jev right now.