CommunityJev Models & Reimplementations
OCF Typed-Decisions Smoke
An Open Classification Foundation smoke checkpoint: a 164M-parameter ModernBERT-base with a cross-entropy decision head, 0.751 accuracy and 0.044 ECE on 2,000 held-out typed decisions.
- Published by
- Community
- Author
- MSGEncrypted
- Last verified
- Oct 2026
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- ~164M parameters on answerdotai/ModernBERT-base with a 2-layer decision head and an OptionScorer at each option [MASK].
- 0.751 accuracy and 0.629 NLL on the 2,000-decision test set, against a Laya reference at 0.767 and 0.707.
- Shipped temperatures: choice 1.5, score 1.25, noul 1.5, fitted on a held-out train slice with cal_frac=0.1.
- Warm latency p50 59.6 ms and p95 87.8 ms per question on an RTX 3050 Laptop over 200 questions.
Watch out
Apache-2.0, but the card lists explicit non-claims: it is not a foundation or zero-shot System-1 model, not RLCD-trained, and does not claim to beat TypeSafe Jev or Laya. It runs only through the source repository's OCF stack, is not a drop-in for the external laya package, and its soft teacher labels disagree on about 41% of test decisions.
Questions about this entry
Short answers drawn from the linked project or page.
No. The card's non-claims say it does not claim to beat TypeSafe Jev or Laya; on the test set it scores 0.751 against the Laya reference's 0.767, and 0.629 NLL against 0.707.
Cross-entropy on teacher-argmax labels from the soft teachers in LocalLLaMA/typed-decisions, four epochs at effective batch 8, with encoder and head learning rates of 2.5e-5 and 1e-4.
choice 1.5, score 1.25, and noul 1.5, fitted on a held-out train slice with cal_frac=0.1 and written to temperatures.json.
It requires the OCF stack from the source repository (packages.classify plus models_ai.classify): from classify import load, then load('path/to/this/repo') reads config.json, model.pt, and temperatures.json. The card says it is not a drop-in for the external laya package.
About 164M parameters, max_len 640, with a 48-token budget per option and the choice, score, and noul primitives.
Warm, on an RTX 3050 Laptop: p50 59.6 ms and p95 87.8 ms per question across 200 questions. The card says this is not a cross-vendor claim.
The training targets are soft teacher labels, and the card notes the teachers disagree on about 41% of test decisions. Its reported ECE after equal crossfit is 0.030.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsRelated terms
Glossary definitions related to this entry.
More Jev resources like this
An unofficial cross-encoder fine-tune of ModernBERT-ja-310m for Jev Local: it reads a question-plus-state and one candidate per sequence and hits 92.62% on the JGLUE JNLI test, with CC BY-SA 4.0 weights.
Jev Models & Reimplementations#community#open-models#local
Community
The 322M mmBERT-base build of Laya for 100+ languages: state plus typed questions return typed answers with probabilities in one forward pass, with up to 8,192 tokens for long documents.
Jev Models & Reimplementations#community#open-models#local
Community
An open Thai + English System One decision model at 0.8B, Apache-2.0: choice with up to 255 options, fractional score, and noul answers in one forward pass over a Qwen3.5-0.8B text tower.
Jev Models & Reimplementations#community#open-models#local
Community
A LoRA scorer on Qwen3.5-4B-Base that scores (state, question, option) triples with a scalar head and softmaxes per question, reaching 0.707 accuracy and 0.044 ECE at T=1.75.
Jev Models & Reimplementations#community#open-models#local
Community
A 70.8M-parameter DeBERTa-v3-xsmall cross-encoder that answers typed decision questions in one batched pass with zero generated tokens, shipped with a calibration.json that the engine requires.
Jev Models & Reimplementations#community#open-models#local
Community
A 4B decision model taking text, JSON, or images up to 262,144 tokens: 52.75 on Decision Index 0.2.1, 204 of 231 on JevBench public, and 26.0 ms for a three-question request on one H200.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
From the Jev community
Posts from builders shipping with Jev right now.