Article
CommunityJev Media & Talks
Doug Does: JevBench tables rank 48 decision models
Doug Does' benchmark dashboard for JevBench: 534 questions, 48 ranked models, composite scores that already include cost, head-to-head comparison, latency and a Pareto frontier. Jev 1.13.0 tops the table at 74.40.
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Author
- Doug Schonholtz
- Last verified
- Oct 2026
Used inJev for Software & DevTools
Highlights
- On 534 JevBench questions, Jev 1.13.0 ranks first of 48 models with a composite score of 74.40 out of 100.
- SemIf (Qwen3.5-4B) is second at 73.09 and djev (DiffusionGemma) third at 73.03; GPT-5.6 Luna at low effort sits 14th at 65.94.
- The composite score already includes cost, and the site plots scores against published dollars per 1,000 decisions with a Pareto frontier.
- Jev 1.13.0's measured price is $0.03991 per 1,000 decisions, and the site places it on the observed frontier.
- The dashboard compares any two of the 48 models, filters by open weights, context and modality, and exports charts as PNG.
Watch out
Copy reviewed against the published benchmark page (updated 2026-09-23) on 2026-10-17. It is one author's dashboard: ranks come from the site's own composite that already includes cost, price labels mix measured and estimated values, and the field includes third-party ports, fine-tunes and rerankers alongside hosted models.
Questions about this entry
Short answers drawn from the linked project or page.
A benchmark dashboard comparing scores, cost and latency across three independent test suites, built around the JevBench set of 534 questions with 48 ranked models, an optional open-weights filter and head-to-head model comparison.
Jev 1.13.0 at 74.40, followed by SemIf (Qwen3.5-4B) at 73.09, djev (DiffusionGemma) at 73.03, Winnow-12B Q8 at 71.22 and reflex 4B at 70.32.
It is a published 0-100 composite that already includes cost; the page plots it against published USD per 1,000 decisions on a log scale, with filled markers on the observed Pareto frontier and hollow markers dominated.
Jev 1.13.0 carries a measured price of $0.03991 per 1,000 decisions and sits on the observed frontier; the page notes several cheaper entries, including some open ports, score well below it.
GPT-5.6 Luna at low reasoning effort ranks 14th at 65.94 with a measured price of $0.24191 per 1,000 decisions, which the chart marks as off the frontier.
Yes. The dashboard lets you pick any two of the 48 models, filter by open weights, minimum context, modality and score range, connect model families, and export the comparison chart as a PNG.
It is one author's dashboard updated 23 September 2026: ranks come from the site's own composite, prices mix measured and estimated labels, and the model field includes third-party ports, fine-tunes and rerankers alongside hosted APIs.
More Jev resources like this
A community roundup of every public Jev evaluation: TypeSafe's own workflow evals plus at least twenty independent builder benchmarks. The site reports speed medians of 127-422 ms per call leading every LLM compared, while accuracy splits by task.
Jev Media & TalksArticle#community#press#calibration
Community
Langfuse's September 18 post presents Jev as a decision model for routing, classification and eval verdicts, and walks through scoring production traces with Jev-as-a-judge in Langfuse.
Jev Media & TalksArticle#community#press#explainer
Community
A playground operator's audit of every published Jev number: TypeSafe's four-workflow evals, the favourable homepage demo, independent tests and its own calls. It concludes the price and speed are real and the intelligence claim is roughly mid-tier frontier on decision workflows.
Jev Media & TalksArticle#community#press#explainer
Community
Celeris's launch post for Celeris-1-decision, a hosted diffusion language model for typed decisions. It reports 80.0% on the full jev-bench, the lowest Brier scores of five decision APIs tested, a 67 ms median and $0.04 per million input tokens.
Jev Media & TalksArticle#community#press#competition
Community
Interactive comparison of 21 decision models, with direct-provider and OpenRouter routes measured separately on 1,100 public jev-bench items. The charts plot accuracy against benchmark cost, official prices and median latency, with Jev 1.13 and 1.14 measured separately.
Jev Guides & ArticlesArticle#community#press#calibration
Community
Per WSJ's search excerpt, the three-week-old Jev has set Silicon Valley abuzz with talk of alternatives to large language models and is already sparking copycats. It classifies inputs to predetermined outputs: yes or no, a numerical score, or answers on a list.
Jev Media & TalksArticle#community#press#competition
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.