Langfuse blog: Using TypeSafe's Jev for evals

A specialized decision model TypeSafe launched in mid-September 2026 that makes decisions and cannot write a single sentence. You send a state plus typed questions and get typed answers with probabilities, and it gives you no reasoning back.

Anywhere in a pipeline or eval harness where decisions are repeated, high volume and the possible answers are known before the call: agent and tool routing, document and ticket classification, escalation decisions, and eval scoring where a rubric verdict is all you need.

Choice picks one option from a set you define, up to 255, returning each option's probability plus a confidence value. Score rates ordered rubric levels, up to 10, returning a probability-weighted value, the full distribution and confidence. Noul answers yes or no and returns the probability it is true.

needs_review comes back at 0.88, so it should be queued. severity lands at 1.89, just short of delivered a wrong result, with 0.44 confidence. failure_mode is inconclusive: missing_context wins at 0.58, but confidence is 0.51 because wrong_approach is close behind at 0.24.

Jev answers up to four classifications in one pass, while Luna makes a parallel call per classification, so its cost scales with the count. The post says Jev is faster and cheaper and returns a decision only, while Luna adds reasoning; both runs are in Langfuse's public sample project.

As one Noul question with a scope: whether the last user message clearly communicates that the user believes the assistant's prior response, reasoning, assumption, work or approach was mistaken. Criteria define what true and false each include, and if there is no prior assistant response, the answer is false.

Finding the criteria is still on you. The post says Jev forces you to think in distinct categories and yes/no decisions to define what good means, and if you do not, the answer comes back with low confidence. TypeSafe's docs insist each question be atomic, matching guidance on writing good evaluators.