CommunityJev Models & Reimplementations
AutoTrust JEV-Gemma4-26B-A4B
AutoTrust's combined System 1 and System 2 bundle on Gemma-4-26B-A4B-it: Decision Index 0.2.1 of 58.05, a 24-slot head with noul, choice and score, and the base chat model left bit-identical.
- Published by
- Community
- Author
- AutoTrust AI Lab
- Last verified
- Oct 2026
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- Serves two systems from one set of weights: bit-identical Gemma-4-26B-A4B-it generation plus a LoRA and 24-slot head for typed decisions.
- Decision Index 0.2.1 balanced skill 58.05 against TypeSafe Jev 1.13's 57.91 on the board and JEV-27B's 53.30.
- 150,317 of 150,759 suite-0.2 requests scored with zero errors; nothing is generated for System 1 decisions.
- A separate calibration_gold.json (noul 1.214, choice 1.098, score 1.000) targets confidence-gated automatic actions.
- Wider choice questions are read in groups of 16 options plus a final read, so no option is pruned.
Watch out
The 58.05 is a complete Decision Index 0.2.1 run, but the card discloses that System 1 trained on teacher distributions and the public training splits of some Decision Index datasets, and that MMMU and MMMU-Pro are not valid evaluations for it. It is weaker than JEV-27B on long structured inputs, sits below chance on HLE, and is English-centric.
Questions about this entry
Short answers drawn from the linked project or page.
System 2 is the unmodified google/gemma-4-26B-A4B-it, text and image input, bit-identical to the base model. System 1 merges a LoRA adapter at load and reads calibrated probabilities for noul, choice and score from a 24-slot head in one prefill pass.
A complete run of suite 0.2 scored with the kit's score --edition 0.2.1: 150,317 of 150,759 requests scored with zero errors, balanced skill 58.05, balanced raw 67.35 and breadth skill 56.98.
Choice reads 2-16 options directly; wider questions are split into contiguous groups of 16 plus a final 16-option read, and every option is read rather than pruned before scoring.
calibration.json holds the defaults used in the Decision Index run (noul 1.003, choice 1.017, score 0.999). Use calibration_gold.json, fitted against ground-truth answers (noul 1.214, choice 1.098, score 1.000), when you gate automatic actions on confidence.
The card lists teacher errors propagating into System 1, weaker long-structured-input performance than JEV-27B on Home appliance simulator and POP909, below-chance HLE, noul and score accepting only canonical options, and an English-centric scope not meant for high-stakes decisions without gating.
Download the snapshot, load Gemma4ForConditionalGeneration.from_pretrained with bfloat16 on CUDA, merge adapter/ through PeftModel, then apply head.safetensors to the last hidden state with slot masks and per-kind temperatures. The card prints a full decide() example.
The card says System 1 trained on teacher distributions and ground-truth data including public training splits of some datasets whose test splits the Decision Index uses; no test split was used, suite items were excluded, and the list went to the Decision Index maintainers.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsRelated terms
Glossary definitions related to this entry.
More Jev resources like this
alibiserikbay's Apache-2.0 Jev alternative on Qwen3.5-4B with a merged rank-16 LoRA: one forward pass returns a probability per option, 0.784 on JevBench's public hard tier at about 13 ms per decision.
Jev Models & Reimplementations#community#open-models#local
Community
The 9B JevK5: the same v0.3 data on Qwen3.5-9B, retrained for 1.5 epochs by a pre-registered sweep, reaching index proxy 0.794 and a 0.859 hand-written hard set, Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
Mithril's open, Jev-shaped typed-decision model on DeBERTa-v3-large: choice up to 255 options, score and noul from one forward pass, 0.854 in-domain with ECE 0.022, trained on public gold labels only.
Jev Models & Reimplementations#community#open-models#local
Community
chaoliangUNSW's v1 Jev-Style decision model in GGUF from 1.3 GB: calibrated choice, bool and score answers on Qwen3.5-2B-Base, 82.3% on five held-out task families with ECE 0.017.
Jev Models & Reimplementations#community#open-models#local
Community
An Apple Silicon MLX bf16 build of the v1 Jev-Style decision model: 77 ms decisions on an M1 Max with calibrated choice, bool and score probabilities, Apache-2.0 weights.
Jev Models & Reimplementations#community#open-models#local
Community
chenz53's standalone decision model derived from Google Gemma 4 26B A4B IT: full merged BF16 weights, a fitted temperature of 1.736507, and 79.50% equal-type accuracy over 1,100 final questions.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
From the Jev community
Posts from builders shipping with Jev right now.