Skip to content
JevDirectory.org
Answers & Confidence

Calibration

The property that outcomes given a probability of 0.2 occur about 20% of the time, measured across many predictions.

Category
Answers & Confidence
Also known as
—
Related terms
4
Directory entries
37
Docs
docs.typesafe.ai
Added
2026-09-24

Definition

RLCD trains Jev so higher probability corresponds to a greater chance of being correct. Across many predictions, outcomes assigned 0.2 should occur about 20% of the time and outcomes assigned 0.8 about 80% of the time.

Calibration describes groups of predictions, not any single answer. A well-calibrated 0.9 can still be wrong, which is why the docs keep a review path for individual cases that matter.

Tagsconfidencetrainingevaluation

From the directory

Why TypeSafe trains decision models with RLCD instead of RLHF: calibrated probabilities where 0.2 outcomes happen about 20% of the time, and the case for machine-to-machine automation.
Sites & GuidesDocs#official#docs#evaluation
Official
How TypeSafe reports certainty, how it differs from probability, and how to use it architecturally to gate and route decisions.
Practices & PatternsDocs#official#confidence#routing
Official
632GitHub stars
Nokia Applied Research's package that turns any open LLM into a Jev-style decision model: typed questions are read from a single prefill, and the calibration fixes need no fine-tuning.
Repos & SDKs#community#python#open-models
A roughly 10,000-call black-box investigation that reconstructs Jev's likely architecture: a shared state prefix, isolated question branches, and direct probability readouts.
Practices & PatternsArticle#community#analysis#architecture
Community
A critical technical read of Jev's launch claims that digs into the calibration and eval numbers and asks what the architecture keeps secret.
Practices & PatternsArticle#community#evaluation#calibration
Community
A skeptical Japanese replication that approximates Jev's speed by reading first-token logits from small LLMs, with Gemma3 and a Mario benchmark.
Practices & PatternsArticle#community#article#open-models
Community

31 more matching entries in the full directory.

From the community

Posts from builders shipping with Jev right now.

Follow @typesafeai

Classifying rows in DuckDB

A playable 16-judgment demo

AI multiple choice, not essay writing

Screening agent actions with Jev

Tested TypeSafe’s Jev (no-text, probability-only model) as an AI agent safety monitor. Checking each action first worked well caught most attacks with almost no false blocks, and much faster than Gemini.

Image
Image
Image
Diogo Almeida
Diogo Almeida
TypeSafe AI
@CompleteSkeptic

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x

Reply

Cua's small System One models

A 706K-parameter form filler

cua open sourced a 706k param model that fills a whole form in one 50ms pass the llm agent doing the same form took 23 turns and 39.6 seconds the specialists are going to eat the generalists from the bottom

Cua
Cua
@trycua

1/ Introducing CUA-S1: a family of System One Models, small, specialized, and built for computer use. Today we're open-sourcing CUA-S1-FORMS, the first in the family: github.com/trycua/cua

Image
Reply