Skip to content
JevDirectory.org
Patterns

Self-consistency

Repeatability across repeated evaluations: Jev's probability standard deviation measured at about 0.01 in the consistency cookbooks.

Open docs.typesafe.aiAll 117 terms
Category
Patterns
Also known as
—
Related terms
4
Directory entries
2
Docs
docs.typesafe.ai
Added
2026-09-24

Definition

The Choice cookbook runs an 8-question moderation rubric 15 times per condition and measures a mean probability standard deviation of 0.0098, against 0.0245 to 0.0543 for five LLM distributions. The Noul cookbook measures 0.0102 on a 14-question claims rubric.

These are repeatability numbers, not accuracy numbers: Jev still flipped its top label on two questions across its own repeats, and a deterministic temperature-0 baseline can score perfect agreement by never abstaining.

Tagspatternsconsistencyevaluation

Related terms

Definitions that connect to this one.

RepeatabilityHow stable answers are across repeated evaluations, measured separately from accuracy and reported as probability standard deviation.Evaluation & TrainingUncertainty bandA middle range of probabilities, such as 0.30 to 0.70, that maps to an explicit uncertain outcome instead of a hard yes or no.Answers & ConfidenceCalibrationThe property that outcomes given a probability of 0.2 occur about 20% of the time, measured across many predictions.Answers & ConfidenceWorkflow evalsTypeSafe's published evaluations of four automation workflows, comparing Jev and frontier LLMs as structured workflows versus single prompts.Evaluation & Training

From the directory

Cookbook: Self-Consistency for Choices
A repeatability study: an 8-question moderation rubric run 15 times per condition shows a mean probability standard deviation of 0.0098, and a 0.60 uncertainty gate lifts agreement to 99.2%.
Cookbooks & DemosDocs#official#cookbook#consistency
Official
Cookbook: Self-Consistency for Nouls
A 14-Noul claims-triage rubric over one insurance claim, repeated 15 times: mean probability standard deviation of 0.0102 at 111 ms per call, with a 0.30 to 0.70 band for human review.
Cookbooks & DemosDocs#official#cookbook#consistency
Official

From the community

Posts from builders shipping with Jev right now.

Follow @typesafeai

Real-time Clippy that wakes up when you struggle

Marek Sotak
Marek Sotak
@sotak
·

I built real-time Clippy with Jev. It quietly watches how you use the product and only wakes up when it thinks you’re struggling. Hesitating? Confused? Stuck? Clippy knows. Even its reactions are controlled by Jev. 👀

Watch on X
Diogo Almeida
Diogo Almeida
TypeSafe AI
@CompleteSkeptic

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x

Watch on X
Reply
Read 11 replies

One-click invoice finder

A visual reference finder

Albiona Hoti
Albiona Hoti
@albicodes
·

I built a visual reference finder with Jev One single prompt → 100 images from Cosmos, NASA, and The Met my new rabbit hole for creative work 🌻  Show more

Watch on X
Diogo Almeida
Diogo Almeida
TypeSafe AI
@CompleteSkeptic

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x

Watch on X
Reply
Read 19 replies

Post scoring with SuperX

Rob Hallam
Rob Hallam
@robj3d3
·

Jev + SuperX = virality solved ✅ Every post gets 61 questions in ~1s for $0.0004 🤯 > fitted on 9,481 real posts from 207 creators > picks the viral post 2 in 3 times > never rewards reply bait So: write, score, rewrite, stop when it peaks. Free, no signup. try it below ↓

Watch on X
Rob Hallam
Rob Hallam
@robj3d3

I spent the last 8 hours building a viral post classifier with Jev. It's now better at spotting viral posts than I am. And it's as good as Fable 5.1, but 100x faster.

Image
Reply
Read 84 replies

A canvas you control by pointing and speaking

Jev plays Smash Bros. against itself