Article
CommunityJev Guides & Articles
Jev News compare: Jev vs Laya, Luna and the rest
Side-by-side tables placing similar Jev tests next to each other, from the JevBench API board to Vercel paid-team reach, with the desk's warning that differing sets, labels and comparators mean the rows are not a ranking.
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Author
- Jev News
- Last verified
- Oct 2026
Works forJev for Evaluation
Used inJev for Software & DevTools
Highlights
- States that sets, labels and comparators differ across tables, so the rows are not a ranking, and that the desk did not rerun any of them.
- Celeris jev-bench table on 3,210 items: celeris-1-decision 0.800, pplx-decider-v1.1-27b 0.790, Mercury Decide 0.753, Jev 1.13.0 0.731, gpt-6-luna 0.690.
- OpenRouter Armor's 492 tool calls: Jev 1.13 at 90.7% for $0.0021 a call, ahead of Clef at 88.4%, Sonnet 5 at 88.2% and GPT-6 Luna at 82.7%.
- JevBench API board: kettei-fat 77.8 first of 39, d1-ultra 76.1 second, and Jev 1.13.0 seventh at 71.5.
- Vercel paid-team reach places nearly 13% at 24 hours against 35.3% over two weeks, with Sonnet 5.5 named at 18.1%.
Watch out
Copy reviewed against the compare page (updated October 17, 2026) on 2026-10-17. The desk warns it did not rerun any table and that differing sets, labels and comparators mean the rows are not a ranking; each table keeps its own scale and print date.
Questions about this entry
Short answers drawn from the linked project or page.
Side-by-side tables that place similar tests the desk has already covered next to each other - JevBench boards, Vercel gateway reach, Celeris jev-bench, Armor tool calls and more - each keeping its own scale and print date.
No. The page states that the sets, labels and comparators differ, so the rows are not a ranking, and it keeps prints from different dates separate rather than merging them.
Depending on the table: Laya, d1-ultra, Sage 1.3.0, kettei-fat, kettei-chub, pplx-decider, Mercury Decide, Clef, gpt-6-luna, Celeris-1 Decision, Microsoft-Decision-1, hosted d1 and the GPT-5.6 family.
On 3,210 items: celeris-1-decision 0.800 accuracy and 0.272 Brier at 67 ms; pplx-decider-v1.1-27b 0.790 and 0.288 at 122 ms; Jev 1.13.0 0.731 and 0.380 at 135 ms, with Mercury Decide and gpt-6-luna below.
Jev 1.13 Armor default agreed on 90.7% of 492 synthetic tool calls at $0.0021 a call, ahead of Clef (88.4%), Sonnet 5 (88.2%) and GPT-6 Luna decisions (82.7%) - figures the desk says it did not rerun.
The page pairs, for example, Aviz Maeir's 82% on 180 scenes with the desk's own Continue-button IoU of 0.93, and warns not to merge 82% with IoU 0.93 because they are different tests.
One-row write-ups live on the evals page, failure modes on the limits page, and stories grouped by tag on topics; RAGTruth rows stay on their own tests and JevBench stays separate from TypeSafe's workflow test.
More Jev resources like this
Blackwell Systems' adversarial technical due diligence on Jev, built only from public material and live API calls. The author reproduces TypeSafe's published addressee numbers within a point and finds a real but noise-type-specific robustness moat.
Jev Guides & ArticlesArticle#community#press#competition
Community
Interactive comparison of 21 decision models, with direct-provider and OpenRouter routes measured separately on 1,100 public jev-bench items. The charts plot accuracy against benchmark cost, official prices and median latency, with Jev 1.13 and 1.14 measured separately.
Jev Guides & ArticlesArticle#community#press#calibration
Community
The desk's index of every Jev measurement it has published, keeping each figure's original caveat: JevBench boards, Vercel gateway reach, Celeris and Artificial Analysis results, and the desk's own Jev Day probes which it says it did not rerun.
Jev Guides & ArticlesArticle#community#press#calibration
Community
A podcast episode on Jev as a decision model rather than a chat model: hosts Luis Calderon and Carina discuss roughly 20x faster Coach testing, why that preliminary figure differs from TypeSafe's vendor benchmarks, and why type safety does not guarantee a correct decision.
Jev Guides & ArticlesArticle#community#press#podcast
Community
Braintrust's provider page covers two ways to use TypeSafe Jev: instrumenting System One calls as decision spans, and running Jev as an LLM judge that turns typed choices into scores or labels.
Jev Guides & ArticlesArticle#community#press#explainer
Community
Langfuse's guide to decision-model evaluators runs OpenAI's gpt-6-luna or TypeSafe Jev over observations, where each Choice, Score or Yes/no question becomes one Langfuse score.
Jev Guides & ArticlesArticle#community#press#explainer
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.