Is it actually beating Jev, or is that comparison rigged by the authors?

The Jev row was measured by the benchmark's own authors through the TypeSafe API, not by this project, and the shared scorer reproduces Laya's published numbers exactly. On 400 cases and 2,000 typed decisions it reports 73.7% against 72.7%, with KL 0.27 versus 1.44. That is a thin margin on one benchmark, and the README says as much.

Can I point my existing Jev SDK at this and stop paying TypeSafe?

No. It replaces the library, not the endpoint. The package runs in-process and there is no HTTP server and no drop-in API for the official Jev SDK, so existing client code has to change. What you get is Jev-style typed decisions from open-weight models on a GPU you already own.

Will packing several questions into one forward pass change the answers I get?

Yes, and the project measures how often rather than waving it away. Answers that change when a question is packed with its neighbours run 37% at 0.6B, 20% at 1.7B, 7.6% at 4B and 3.9% at 27B, against a 2.7% noise floor from padding alone. Below about 4B packing costs accuracy; for answer-level stability use separate mode.

Are the probability numbers trustworthy out of the box?

Not without work. Raw confidence runs about 5 points high; temperature scaling with one scalar fitted on held-out labels brings expected calibration error from 5.4% to 2.1% on MMLU and from 2.8% to 1.1% on RACE-H. Option order also biases results, so permutations=2 averages both orders at roughly 1.7x latency for better ECE and KL.

How do I install the open-alternative-jev library?

pip install open-alternative-jev, with optional extras open-alternative-jev[vllm] for vLLM and open-alternative-jev[quant] for quantization. The package is imported as so1 and is Apache-2.0.

What are the option-order permutation settings for?

Reversing option order moved the 4B model's yes/no accuracy by 13.5 points, so permutations=2 averages the given and reversed orders. For the 27B it added 1.9 points of accuracy and cut ECE from 0.020 to 0.0075 at 1.7x latency; it is off by default.

When should I use separate mode instead of packed?

Packing several questions into one forward pass can change 6 to 9 percent of individual answers even though aggregate accuracy holds, so use mode="separate" when you need answer-level stability. Interference shrinks with model size, and below about 4B it costs accuracy.

Jev directory

CommunityJev Repos & SDKs

Open Alternative to Jev

An Apache-licensed Python library that reads option-token probabilities from open-weight models through Transformers or vLLM, giving Jev-style typed decisions in one forward pass without an HTTP server.

See highlights Open github.com

Stars: 68
Language: Python
License: APACHE-2.0
Reading time: 3 min
Published by: Community
Author: ikermoel
Last verified: Sep 2026

ikermoel/open-alternative-jev

User repository on GitHub

View on GitHub

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with honest benchmarks

Forks: 14
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.Homepage

About this project

Open-source System One models: typed, calibrated decisions from any open-weights LLM, in one forward pass. An open alternative to the idea behind TypeSafe's Jev, running on your own GPU with models you already have. Python package open-alternative-jev, import name so1 ("System One"). Also known as: open Jev, Jev alternative, open-source System One model. This replaces the library, not the endpoint: it is a Python package you call in-process, and there is no HTTP server or drop-in API for the official Jev SDK.

Quoted from the project's README. Read the full README.

Tagscommunity python open-models calibration benchmarks

Works forJev for Evaluation

Used inJev for Software & DevTools

Highlights

On a 400-case typed-decisions benchmark, Qwen3.6-27B scored 73.7% accuracy with ECE 0.020 versus Jev 1.13.0 at 72.7% and ECE 0.144.
Probabilities were five times closer to the gold distribution (KL 0.27 versus 1.44), at 582 ms per case versus 710 ms for Jev.
On RACE-H, packing the shared state once processed 186,898 tokens at 4.55 questions/s versus 468,583 tokens at 1.66 questions/s one at a time.
It replaces the library, not the endpoint: the package runs in-process and there is no HTTP server or drop-in API for the official Jev SDK.
A live Hugging Face Space demo runs Qwen3.5-4B with no install, answering from the next-token distribution restricted to the given options.

Quickstart

bash

pip install open-alternative-jev
pip install "open-alternative-jev[vllm]"
pip install "open-alternative-jev[quant]"

Watch out

Apache-2.0. Needs Python and your own GPU for the larger models; it is a library rather than an endpoint, packed answers can shift with question order, and benchmarks compare modes of an open model, not Jev's training.

Questions about this entry

Short answers drawn from the linked project or page.

Reactions & coverage

Posts, threads, and videos about this entry from around the web.

X: Standard One SH (StandardThinking)

LinkedIn: What multimodal decision models actually change, and what they do not.

LinkedInJevDirectory.org

Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.

jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker News

Hacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.

1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker News

From the Jev guides

Original write-ups on what this entry is for.

The trust ledger: measuring confidence your domain has earned

Oct 16, 2026

ASSAY-001 calibrated Jev on CLINC150 but found it overconfident on Banking77 — one audit, two verdicts. How calibration research measures the model-and-domain pair, discounts stated confidence, and signs the evidence.

Jev Benchmarks & Evaluations#calibration#evaluation#open-models

Read article

Beyond text and boxes: multimodal decisions you can run yourself

Oct 16, 2026

Jev-Omni's merged classifier scores text, image, audio, and video options; VideoJev and Laya Vision show what open multimodal decision models accept, what they return, and which numbers are the authors' own.

Jev Models & Reimplementations#multimodal#open-models#vision

Read article

What the First Month of Jev Papers Found

Oct 14, 2026

More than eighty preprints now evaluate, attack, and reproduce the typed-decision class. The broadest board, the coherence gap, the rejection bottleneck, and the attacks that still work.

Jev Practices & Patterns#research#evaluation#calibration

Read article

From the Jev AI model community

Posts from builders shipping with Jev right now.

Follow @typesafeai

Open Alternative to Jev

About this project

Highlights

Quickstart

Questions about this entry

Is it actually beating Jev, or is that comparison rigged by the authors?

Can I point my existing Jev SDK at this and stop paying TypeSafe?

Will packing several questions into one forward pass change the answers I get?

Are the probability numbers trustworthy out of the box?

How do I install the open-alternative-jev library?

What are the option-order permutation settings for?

When should I use separate mode instead of packed?

Reactions & coverage

Related terms

More Jev resources like this

From the Jev guides

From the Jev AI model community