Are the 79.25% benchmark numbers real, or the project's own marketing?

It is the project's own measurement, and the README is explicit about what it means: top-1 agreement with a public teacher's argmax on the official test split of Typed Decisions, 400 cases and 2,000 questions, not a measured coding-agent success rate. Beating a row there does not mean a pull request merged or an incident was contained.

Is this just TypeSafe Jev with the name off?

No. It is an independent reimplementation, not affiliated with TypeSafe: a Qwen3-0.6B backbone with the language-model head removed, plus a small permutation-equivariant head that scores the candidate set so option order cannot smuggle in a ranking. Weights are published on Hugging Face and must be downloaded separately.

What is the catch with a 0.6B model that never decodes a token?

Context is 2,048 tokens and over-length input is refused outright rather than silently cropped, so a long diff or stack trace errors instead of degrading. Also, the margin on a Choice is a preference inside the set you passed, not an independent probability that the action will succeed; calibrate that yourself if you need it.

Can it write my prose or pick an option you did not offer?

It cannot write. Hidden states go to logits with no output vocabulary step, so there is nothing to parse. You supply the options (2-255 for Choice, 2-10 ordered levels for Score) and get a full distribution back. The intended slot is a gate, route, or score under a turn that a larger model then writes.

What is the model backbone and context limit?

Qwen3-0.6B with the language-model head removed, a 2,048-token context, and over-length input refused as an error rather than silently cropped.

What does the shared-prefix path change?

On a fixed load of 64 choice options plus one boolean, the unshared median was 609.65 ms and the shared-prefix path 298.91 ms, with a largest probability difference of 0.000508 and no change in the chosen option.

What does each primitive return?

Boolean returns value and the probability of true; Choice returns value, top_probability, margin, and the full map; Score returns the argmax level and a score computed as the sum of level index times probability.

Jev directory

CommunityJev Repos & SDKs

AgentJev-0.6B — malevrigns

An open 0.6B System One decision model that reads unstructured state and typed questions in one forward pass, returning calibrated distributions with zero decoded tokens; weights are published on Hugging Face.

See highlights Open github.com

Stars: 388
Language: Python
License: APACHE-2.0
Reading time: 3 min
Published by: Community
Author: malevrigns
Last verified: Sep 2026

malevrigns/agent-jev

User repository on GitHub

View on GitHub

AgentJev-0.6B - a fast 'System One' decision model for AI Agents: feed it any unstructured state (diffs, traces, logs) and structured questions, get calibrated probability distributions back in one ~50ms forward pass. Zero output-token decoding.

Forks: 36
Last pushed: Updated Oct 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

The job   ·   Primitives   ·   Forward pass   ·   Results   ·   Latency   ·   Run it   ·   HTTP   ·   Repository. AgentJev-0.6B does not write. You hand it a state and questions you have already phrased. One forward pass returns a probability for every option. The number of decoded tokens is zero.

Quoted from the project's README. Read the full README.

Tagscommunity python open-models confidence tool-calling

Works forJev for Evaluation Jev for Scoring Jev for Ranking

Used inJev for Software & DevTools

Highlights

Its hero benchmark reports 79.25% top-1 on 2,000 Typed Decisions questions from the official test split with zero tokens decoded.
Backbone is Qwen3-0.6B with the language-model head removed and a permutation-equivariant head that scores options so order adds no ranking.
Context is capped at 2,048 tokens and over-length input is refused rather than cropped; a shared prefix cuts one measured load from 609.65 ms to 298.91 ms.
Choice takes 2-255 options and Score 2-10 levels; one call allows 32 states, 128 questions, 1,024 candidate paths, and a 1 MB body.
Ships a loopback server, Python client, and a PreToolUse gate hook, and states that choice margin is not an independent success probability.

Quickstart

bash

pip install -r requirements.txt
pip install huggingface_hub safetensors

Watch out

Apache-2.0-licensed. Weights live on Hugging Face and must be downloaded separately; it is an independent reimplementation, not affiliated with TypeSafe, and the top-1 figure is the project's own measurement.

Questions about this entry

Short answers drawn from the linked project or page.

Reactions & coverage

Posts, threads, and videos about this entry from around the web.

X: Standard One SH (StandardThinking)

LinkedIn: What multimodal decision models actually change, and what they do not.

LinkedInJevDirectory.org

Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.

jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker News

Hacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.

1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker News

Other listings of this project

Forks and copies of this project in the directory. This entry is the canonical one.

AgentJev-0.6B

AgentJev-0.6B scores coding completion with a Qwen3-0.6B backbone stripped of its LM head: 57.8% accuracy (606/1048) on the coding set and 87.2% agreement with the teacher on invoice questions.

Jev Models & Reimplementations#community#open-models#local

Community

From the Jev guides

Original write-ups on what this entry is for.

The trust ledger: measuring confidence your domain has earned

Oct 16, 2026

ASSAY-001 calibrated Jev on CLINC150 but found it overconfident on Banking77 — one audit, two verdicts. How calibration research measures the model-and-domain pair, discounts stated confidence, and signs the evidence.

Jev Benchmarks & Evaluations#calibration#evaluation#open-models

Read article

The Open Reimplementations: Local Models That Speak System One

Sep 22, 2026

Closed weights have not stopped the ecosystem from rebuilding the shape in the open. API-compatible servers, retrained heads on small models, and inference-time tricks that skip generation entirely.

Jev Repos & SDKs#open-models#research#local