If it only returns a float, how do I debug a wrong decision?

You cannot ask it why, which is his main discomfort: this is a regression toward black-box systems, since even an LLM can at least be asked to justify itself. Your only recourse is structured evals, which is affordable here because a few hundred experimental prompts cost cents.

Is the probability solid enough to put behind a threshold?

He does not claim it is verified, he reports the calibration claim as TypeSafe's framing and points elsewhere for weaknesses. He also flags the bias problem directly and hopes nobody ranks job applicants with it, since a single float can conceal bias that would be hard to unpick experimentally.

Can I self-host this shape of model instead of paying per input token?

There is a wave of open-weight recreations, with Kev building 0.8B, 4B and 9B models on Qwen 3.5, plus a JevBench comparison for Jev-class decision models. None of that is Jev itself, and the trade-off between self-hosted control and measured agreement is yours to test.

What is it documented to be bad at, beyond the marketing?

The Jev 1.13 jaggedness guidance is where he sends you: not great with numbers, dates, or adversarial content. His own bias probe is the cautionary example, scoring Bay Area cities on whether they are a good city and ranking Cupertino top and East Palo Alto bottom.

What does Jev cost, and how is it billed?

Jev charges only for input; output is free. The input price of the first model is $0.042 per million tokens, cheaper even than OpenAI's GPT-5 Nano at $0.05 per million.

What kinds of state can I send, and how are many questions handled?

The state object can hold a string, an array of strings, or a set of name-value pairs describing the record being judged. You send it with one or more questions and get a reply for each; questions are evaluated in parallel, so many questions should take about the same time as one.

What did the author's experiment on Jev and bias show?

He had Jev score every San Francisco Bay Area city on whether it was a Good city and it rated Cupertino top and East Palo Alto bottom. He warns against using Jev to rank job applicants, since a single float can conceal unseen bias, and says evals matter more for these models.

CommunityJev Guides & Articles

Jev introduces a new shape of LLM

Simon Willison's overview of System One: one state, many parallel questions, and Noul, Choice, and Score instead of generated text.

See highlights

Open simonwillison.net

Format: Article
Published by: Community
Author: Simon Willison
Last verified: Sep 2026

Tagscommunity article explainer

Works forJev for Classification

Used inJev for Software & DevTools

Highlights

21 September 2026. He prefers the name decision models, and credits Maggie Appleton for that phrasing.
He reports the CEO saying on Hacker News that Noul is short for Bernoulli.
The post quotes $0.042 per million input tokens and free output, against GPT-5 Nano at $0.05.
Questions on one state run in parallel. He points at the jaggedness docs for numbers, dates, and adversarial content.

Watch out

The price comparison restates TypeSafe's published rates. The page carries a Greptile sponsorship line.

Questions about this entry

Short answers drawn from the linked project or page.

Reactions & coverage

Posts, threads, and videos about this entry from around the web.

Hacker News: Simon Willison's overview of Noul, Choice, and Score, and where a decision model fits.

benwerd53 pointsJev introduces a new shape of LLMRead the thread on Hacker News

Hacker News: swyx posts the Latent Space interview with Diogo Almeida.

swyx4 pointsJev: System One Models for Prod, Not God – With Diogo Almeida, CEO, TypeSafe AIRead the thread on Hacker News

X: Diogo Almeida in five minutes

X: LLMs vs. Jev, clearly explained

More Jev resources like this

InfoQ: Jev returns typed probabilities

InfoQ's 1 October 2026 news brief on Jev: a parallel pass over typed questions, TypeSafe's published price and latency, and where to read the jaggedness notes.