CommunityJev Models & Reimplementations
systemone-lite 0.5B
A local System One-compatible decision model on Qwen2.5-0.5B-Instruct: 50.7% accuracy and 0.245 ECE on JevBench public with a 13.1 ms median, built for demos rather than production.
- Published by
- Community
- Author
- dwidlee
- Last verified
- Oct 2026
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- Phase2 held-out accuracy is 61.6% on 4,700 test cases; the first-800 protocol scores 63.9%, and JevBench public 50.7% at T=1.0.
- JevBench latency p50 is 13.1 ms, with short payloads typically 10–30 ms on a consumer GPU in-process.
- Serving is option-restricted next-token scoring over closed criteria, yes–no, and score questions; JSON option keys are mapped after scoring.
- Trained with the action-v2-qwen run: 10,000 steps, 1e-5 cosine learning rate, on 240,800 training rows with 0% overlap against 4,700 test cases.
- Per-gym accuracy on full test runs from game2048 at 34.6% and sokoban 38.2% up to alloc and ticket at 98% or more.
Watch out
Apache-2.0 and not affiliated with TypeSafe AI or Jev. The card calls 0.5B demos-and-experiments territory, says calibration is mediocre so probabilities should not be trusted as confidence, and notes uniform-random on JevBench is about 32% rather than 50%. The stable repo name is overwritten by new training runs.
Questions about this entry
Short answers drawn from the linked project or page.
No. The card states it is not affiliated with TypeSafe AI or Jev and has different weights, a different scoring path, and a different confidence formula.
No. The card says calibration is mediocre and warns not to trust probabilities as calibrated confidence. On JevBench public the model scores 0.245 ECE at T=1.0.
61.6% on the Phase2 held-out test of 4,700 cases, 63.9% on the first-800 protocol, and 50.7% accuracy on JevBench public's 231 tasks, where uniform random is about 32%.
From the systemone-lite repository, pip install -e '.[dev]', then systemone-lite --model dwidlee/systemone-lite-0.5b --port 8000. Python callers use SystemOneClient and the choice, noul, and score helpers.
Spatial planning is weakest: game2048 34.6%, sokoban 38.2%, chess 42.0%, gridworld 45.6%, and connect4 47.0% on the full test split, while alloc and ticket sit at 98% or higher.
dwidlee/systemone-lite-phase2 with 240,800 training rows and 4,700 test rows at 0% train-test overlap, via the action-v2-qwen run of 10,000 steps with a 1e-5 cosine learning rate and stratification.
Closed-option scoring ranks the supplied symbols, usually one vocabulary id each, and JSON option keys are mapped back after scoring. There is no autoregressive generation of the answer.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsMore Jev resources like this
TokenRhythm's 4B prefill-only decision model built on NeoHorse-1-4B: given a state and application-defined questions it returns Choice, Noul, and Score probabilities without generating text, reporting a 77.70 six-group aggregate under Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
58GitHub stars
An Apache-2.0 decision engine with a TypeSafe-compatible endpoint, document answers with source passages, and a ViZDoom demo that picks actions from structured game state.
Jev Models & Reimplementations#community#python#local
Community
deepanwadhwa/OpenDecisionPython
deepanwadhwa/OpenDecisionPythonAn unofficial cross-encoder fine-tune of ModernBERT-ja-310m for Jev Local: it reads a question-plus-state and one candidate per sequence and hits 92.62% on the JGLUE JNLI test, with CC BY-SA 4.0 weights.
Jev Models & Reimplementations#community#open-models#local
Community
The 322M mmBERT-base build of Laya for 100+ languages: state plus typed questions return typed answers with probabilities in one forward pass, with up to 8,192 tokens for long documents.
Jev Models & Reimplementations#community#open-models#local
Community
An Open Classification Foundation smoke checkpoint: a 164M-parameter ModernBERT-base with a cross-entropy decision head, 0.751 accuracy and 0.044 ECE on 2,000 held-out typed decisions.
Jev Models & Reimplementations#community#open-models#local
Community
An open Thai + English System One decision model at 0.8B, Apache-2.0: choice with up to 255 options, fractional score, and noul answers in one forward pass over a Qwen3.5-0.8B text tower.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
From the Jev community
Posts from builders shipping with Jev right now.