Skip to content
JevDirectory.org
CommunityPractices & Patterns68 starsVerified 2026-09-22

OpenJev (Verdict)

An encoder-side reproduction of Jev built on a retrained GLiClass ModernBERT base (151M): it evaluates multiple typed questions in one non-autoregressive forward pass, returning choices, ordinal scores, and probabilities under 35 ms.

Open github.comHeman10x-NGU/Verdict-open-jev
Category
Practices & Patterns
Published by
Community
Author
Heman10x-NGU
Added
2026-09-22
Tagscommunitypythonopen-modelscalibrationbenchmarks

Highlights

  • On 231 public JevBench tasks, inference-only fixes raised standard-tier accuracy from 62.5% to 69.4% and cut hard-tier ECE from 0.298 to 0.118.
  • The weights are byte-identical to the published checkpoint heman10x/rlcd-modernbert-151m; the v1.4 changes are all inference-engine fixes.
  • Candidate labels are framed as NLI hypothesis sentences ('It is {description}') to match the GLiClass backbone's pretraining.
  • The context budget was cut from 1024 to 512 tokens after training on states under 71 tokens, avoiding out-of-distribution positional drift.
  • Hard-tier accuracy stayed at 36.9% across the update, while probability fidelity rose 10 points to 72.8.

Quickstart

bash
pip install -e .

Watch out

Licensed as Other (NOASSERTION), so reuse terms need checking; requires Python 3.10+, and the headline numbers are self-reported on the project's own benchmark slice.

More like this

SemIf
3.6kGitHub stars
An independent baseline that reads typed option probabilities straight from a frozen Qwen3.5-4B's logits in one forward pass, reproducing Jev's interface pattern with open models rather than Jev's undisclosed model or training.
Practices & Patterns#community#python#open-models
CommunitySemIf
Kev
3.2kGitHub stars
Apache-licensed, locally runnable Jev-style decision models (0.8B, 4B, 9B on Qwen3.5) with released weights, training code, a System One-compatible server, frozen eval suites, and a playground.
Practices & Patterns#community#python#open-models
Communitykev
decider
296GitHub stars
An independent open reproduction of the System One model class: Qwen3.5-based 2B and 35B mixture-of-experts models that return typed Choice, Score, and Noul probabilities in one forward pass, with nothing distilled from Jev.
Practices & Patterns#community#python#open-models
Communitydecider
Back to all resources

From the community

Posts from builders shipping with Jev right now.

Follow @typesafeai

Classifying rows in DuckDB

A playable 16-judgment demo

AI multiple choice, not essay writing

Screening agent actions with Jev

😎Nick 常胜
😎Nick 常胜
@isNickMa
·

Tested TypeSafe’s Jev (no-text, probability-only model) as an AI agent safety monitor. Checking each action first worked well caught most attacks with almost no false blocks, and much faster than Gemini.

Image
Image
Image
Diogo Almeida
Diogo Almeida
TypeSafe AI
@CompleteSkeptic

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x

Watch on X
Reply
Read 1 reply

Cua's small System One models

A 706K-parameter form filler

Arsh - 16 y/o builder
Arsh - 16 y/o builder
@be_arsh
·

cua open sourced a 706k param model that fills a whole form in one 50ms pass the llm agent doing the same form took 23 turns and 39.6 seconds the specialists are going to eat the generalists from the bottom

Cua
Cua
@trycua

1/ Introducing CUA-S1: a family of System One Models, small, specialized, and built for computer use. Today we're open-sourcing CUA-S1-FORMS, the first in the family: github.com/trycua/cua

Image
Reply
Read 1 reply