Answers & ConfidenceUtility matrixLoss matrix

Cost matrix

The table of what each kind of error costs, which turns a probability into the threshold that minimizes expected cost on a given action.

Open docs.typesafe.ai All 164 terms

Category: Answers & Confidence
Also known as: Utility matrix, Loss matrix
Related terms: 5
Directory entries: 10
Docs: docs.typesafe.ai
Added: 2026-10-12

Definition

A cost matrix assigns a cost to each outcome: a false yes that triggers an auto-refund costs more than a false no that waits for a human. Given a calibrated probability and the costs, the optimal threshold is where the expected cost of acting equals the expected cost of abstaining.

This is why a single global threshold is usually wrong. The cost of being wrong is a property of the action, not the model, so the threshold belongs per action and should be derived from the matrix rather than picked by feel.

Tagsconfidencedesign

Related terms

Definitions that connect to this one.

ThresholdThe value your code compares against a score, a Noul probability, or a confidence to turn an answer into an action.Answers & Confidence Base rateHow common the positive class is in your traffic, which sets the starting probability a decision model has to move away from.Answers & Confidence Confidence gateA threshold on confidence that decides whether code acts, confirms first, or escalates to a person or a bigger model.Answers & Confidence Selective decisionA decision that is allowed to abstain instead of answering, trading coverage for accuracy on the cases it does handle.Answers & Confidence Fail openWhen Jev is down or slow, the gate allows the action, including rules that would otherwise block it.Ecosystem

From the Jev directory

Confidence

How TypeSafe reports certainty, how it differs from probability, and how to use it architecturally to gate and route decisions.

Jev Practices & PatternsDocs#official#confidence#routing

Official

Inside Halv's 57.1% Cost Reduction: SWE-rebench Data

Halv reports a paired SWE-rebench comparison where its workflow, an Astra coordinator, a Luna investigation stage, and JEV selecting worker models and reasoning effort, recorded $144.67 in model cost versus $337.50 for vanilla Codex across 42 runs, with both arms passing 25 verifier runs.

Jev Benchmarks & EvaluationsArticle#community#benchmark#routing

Community

I tried TypeSafe's System One model

Joe Maddalone on using Jev for routing and filtering before a generative model, with demo code in a transcript-analyzer repo.

Jev Media & TalksVideo#community#video#routing

Community

Jev: System One model

ZAUEY on a model that picks from options you define and returns a probability, and why many questions on one state stay cheap.

Jev Media & TalksVideo#community#video#explainer

Community

Room

A natural-language smart-room demo: describe a mood and Jev decides lights, curtains, TV, music, volume, and desk lamp in one request.

Jev CookbooksPlayground#community#playground#smart-home

Community

Jev is the FIRST of a Whole New Class of AI Models

Cole Medin's week of hands-on testing, covering where Jev gates replace regex or slow LLM calls, plus demos that play his indie game and triage every open pull request.

Jev Media & TalksVideo#community#video#tutorial

Community

4 more matching entries in the full directory.

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai