CommunityJev Models & Reimplementations
System One Zeroshot
A 70.8M-parameter DeBERTa-v3-xsmall cross-encoder that answers typed decision questions in one batched pass with zero generated tokens, shipped with a calibration.json that the engine requires.
- Published by
- Community
- Author
- shreyanbr
- Last verified
- Oct 2026
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- No supervision: the base NLI checkpoint is served as is, with no fine-tuning on decision data.
- Accuracy ranges from intent 0.508 and tool 0.384 to needs_tool 0.792, type 0.560, and is_incident 0.754 on 500-item splits.
- Two of six questions sit at or below the majority floor for every system measured, Claude Haiku 4.5 included.
- calibration.json is required: raw margins are not probabilities, and the engine reads a per-question temperature and a Platt scaler.
Watch out
Apache-2.0. The base checkpoint was trained on banking77, so that row is contaminated, and banking77's published label-error rate near 14% caps achievable accuracy around 0.86. The tickets dataset is CC-BY-NC-4.0, so triage supervision is non-commercial, and all intervals come from a single seed and run.
Questions about this entry
Short answers drawn from the linked project or page.
No. The card says supervision for this checkpoint is none: the base NLI checkpoint is served as is, and it implements the TypeSafe Jev POST /v1/systemone schema.
The raw margins are not probabilities; calibration.json ships alongside the weights and the engine reads a per-question temperature and a Platt scaler from it.
Next to the majority floor, not on its own: intent 0.508 against a 0.010 floor and tool 0.384 against 0.306 look different from priority 0.464, which equals its 0.464 floor.
The base checkpoint was trained on banking77, so that row is contaminated; banking77 also has a published label-error rate near 14%, which caps achievable accuracy around 0.86.
The weights are Apache-2.0, but the tickets dataset is CC-BY-NC-4.0, so the triage supervision is not for commercial use, and triage.priority is close to unlearnable on this data.
Build a SystemOneEngine('shreyanbr/system-one-zeroshot'), validate a SystemOneRequest with your state and questions, and call engine.answer(request), then read the answers dict.
The card says single seed, single run, with intervals bootstrapped over test items rather than over training seeds, so treat the figures as one draw rather than a stable average.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsRelated terms
Glossary definitions related to this entry.
More Jev resources like this
AutoTrust's combined System 1 and System 2 bundle on Gemma-4-26B-A4B-it: Decision Index 0.2.1 of 58.05, a 24-slot head with noul, choice and score, and the base chat model left bit-identical.
Jev Models & Reimplementations#community#open-models#local
Community
The v3 Jev-Style decision model at 2B in GGUF: F16 3.78 GB and Q8_0 2.01 GB match PyTorch FP32 on 100% and 99.7% of parity rows, 73.6% on JevBench v1.4.1 public items.
Jev Models & Reimplementations#community#open-models#local
Community
An Open Classification Foundation smoke checkpoint: a 164M-parameter ModernBERT-base with a cross-entropy decision head, 0.751 accuracy and 0.044 ECE on 2,000 held-out typed decisions.
Jev Models & Reimplementations#community#open-models#local
Community
A decision model that scores its own clarifying questions by value of information: at AUC .799 against an oracle .797, it asks only when asking is expected to change the answer, then decides or hands off.
Jev Models & Reimplementations#community#open-models#local
Community
alibiserikbay's Apache-2.0 Jev alternative on Qwen3.5-4B with a merged rank-16 LoRA: one forward pass returns a probability per option, 0.784 on JevBench's public hard tier at about 13 ms per decision.
Jev Models & Reimplementations#community#open-models#local
Community
The 9B JevK5: the same v0.3 data on Qwen3.5-9B, retrained for 1.5 epochs by a pre-registered sweep, reaching index proxy 0.794 and a 0.859 hand-written hard set, Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
From the Jev community
Posts from builders shipping with Jev right now.