Docs
OfficialJev Practices & Patterns
Confidence
How TypeSafe reports certainty, how it differs from probability, and how to use it architecturally to gate and route decisions.
- Format
- Docs
- Reading time
- 2 min
- Published by
- TypeSafe AI
- Last verified
- Sep 2026
Works forJev for Routing
Used inJev for Software & DevTools
Highlights
- Confidence is a 0-1 statistic derived from the probability distribution; Noul answers carry no confidence at all.
- The page's interactive demo approximates three-option confidence as (3 x largest probability - 1) / 2.
- Suggested pattern: high confidence acts automatically, medium proceeds with caution, low routes to a human, clarification, or fallback.
- Example code routes anything below 0.5 confidence to a human and requires above 0.9 to auto-approve a transfer.
- You are never locked into TypeSafe's confidence formula; the full probabilities come back so you can compute your own measure.
Quickstart
python
action = response.answers["action"]
confidence = action.confidence
if confidence < 0.5:
route_to_human(user_message)
elif action.choice == "check_balance":
show_balance(account_id)
elif action.choice == "approve_transfer":
if confidence > 0.9:
confirm_then_execute(account_id)
else:
ask_user_to_confirm(account_id)Watch out
Threshold values depend on your domain and the model's performance on your use case, and the docs advise starting conservative and tuning against your own data.
Questions about this entry
Short answers drawn from the linked project or page.
Confidence is a 0-1 statistic derived from the distribution — the page's demo approximates three-option confidence as (3 x largest probability - 1) / 2 — while the full probabilities still come back so you can compute your own measure.
Conservative ones. The docs say values depend on your domain and the model's performance on your use case, and advise starting conservative and tuning against your own data; the example routes below 0.5 to a human and requires above 0.9 to auto-approve.
Choice and Score answers. Noul answers carry only a single 0-1 probability of yes and no separate confidence property.
A confidence of 1 means all the probability mass sits on one outcome; 0 means the distribution is spread evenly across the options or levels.
Gate each action at its own level. The page's example routes anything below 0.5 to a human but requires above 0.9 to auto-approve a transfer, while a read-only balance lookup is not gated as hard.
Yes. The page says confidence is one reasonable summary of a distribution, and the full probabilities are returned with every answer so you can compute whichever measure matters most to your application.
That none of the options is a clear winner over the others. For a Score it often means the levels are ambiguous or multi-dimensional, or the state does not contain enough to go on.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
Instagram: Shadow mode collects the real distribution before any user sees the answer.
Facebook: Intake Router keeps diagnosis out of the answer space at the front desk.
FacebookClinicnotes
X: OpenRouter Decision Model Rankings
X: herdr + jev, an agent router
More Jev resources like this
Use the answer to decide what to do and confidence to decide whether to act: the voice-banking example routes below 0.6 to a human and needs 0.85 or more to auto-approve a transfer.
Jev Practices & PatternsDocs#official#patterns#confidence
Official
Classify intent with a Choice and complexity with a Score, then send each branch to deterministic code, a specialist model, or a person, with a 0.5 intent-confidence floor.
Jev Practices & PatternsDocs#official#patterns#routing
Official
Ask everything the system might need in one request, then let code throw away what it does not use. The ticket-triage example sends a category plus four speculative questions in parallel.
Jev Practices & PatternsDocs#official#patterns#batching
Official
An independent calibration study of Jev against a frontier fallback on Banking77 and Web of Science, packaged as the janus-decide tool that measures a routing threshold on your own data instead of shipping a default one.
Jev Practices & Patterns#community#python#routing
Community
FirasSX914/JanusPython
FirasSX914/JanusPythonAn open lab for using Jev correctly next to Claude Code: a patched jev-router with per-turn model routing, a health-gated fallback with alerts, and write-ups of two silent bugs and of where tokens actually go.
Jev Practices & Patterns#community#catalog-import#logicrw
Community
pasblinn/jev-labTypeScript
pasblinn/jev-labTypeScriptJevCascade allocates inference-time compute for prefill-only decision models using readout stability: when only the candidate menu changes, post-intervention accuracy is already determined by the cached first-pass distribution. Menu-only interventions are predicted to within 4.2 points across six model families.
Jev Practices & PatternsArticle#community#research#arxiv
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.