Article
CommunityJev Media & Talks
System One Models: every public Jev evaluation in one roundup
A community roundup of every public Jev evaluation: TypeSafe's own workflow evals plus at least twenty independent builder benchmarks. The site reports speed medians of 127-422 ms per call leading every LLM compared, while accuracy splits by task.
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Author
- System One Models
- Last verified
- Oct 2026
Used inJev for Software & DevTools
Highlights
- Independent Jev speed medians land between 127 ms and 422 ms per call, faster than every LLM the roundup compared.
- The roundup says Jev wins topic classification, routing, reranking and agent-failure attribution, but loses to Claude Haiku 4.5 on phishing.
- On the community Jev Decision Index, Jev sits third of 117 rows at 60.11, 2.64 points behind pplx-decider v1.1 at 62.75.
- The separate Artificial Analysis Decision Index ranked Jev first of 11 hosted decision models at launch; Liquid's d1-ultra now leads it.
- TypeSafe's own workflow evals give Jev 67.8% agreement at $0.0004 per case, but the labels are two frontier LLMs' consensus, not ground truth.
Watch out
Copy reviewed against the published roundup (updated 16 October 2026) on 2026-10-17. The page compiles TypeSafe's vendor-run evaluations and mostly single-author independent tests, states that none of the tests is peer reviewed, and carries TypeSafe's workflow numbers as vendor claims the site did not reproduce.
Questions about this entry
Short answers drawn from the linked project or page.
It gathers every public Jev evaluation it could find: TypeSafe's own workflow evals and launch-post multiples, at least twenty independent builder benchmarks since the 15 September 2026 launch, the Jev Decision Index, and the Artificial Analysis Decision Index.
Independent single-call medians land between 127 ms and 422 ms, and the page says every LLM Jev was compared with was slower; TypeSafe's own launch figures are 70 ms to 500 ms end to end against 3 to 329 seconds for frontier models.
Third of 117 rows on the Full score with 60.11, behind pplx-decider v1.1 at 62.75 and Fastino GLiDE at 60.21; the board's 0.9-point tie rule gives Jev, GLiDE and Torchcast Decision 27B a shared second place.
It lost clearly to Claude Haiku 4.5 on phishing, to OpenAI structured outputs on medical-note extraction, and to plain classifiers trained on labelled data, and the page notes it gets worse outside English.
No. The page states that none of the tests is peer reviewed and that most are single runs by one author, generally on jev-1.13.0, so its numbers are evidence to rerun, not settled findings.
Four code-driven tasks: security incidents, agent-trace observability, invoice processing and customer service. Accuracy means agreement with an average of GPT-6 Astra and Claude Fable 5.1 answers, not ground truth, which the roundup flags as a caveat.
Launched 13 October 2026, it ranked Jev first of 11 hosted decision models at 57.5; after Liquid's d1-ultra was added by 15 October, the roundup says Jev ranks second of 13 as of 16 October 2026.
More Jev resources like this
Doug Does' benchmark dashboard for JevBench: 534 questions, 48 ranked models, composite scores that already include cost, head-to-head comparison, latency and a Pareto frontier. Jev 1.13.0 tops the table at 74.40.
Jev Media & TalksArticle#community#press#competition
Community
Fortune reports Jev went viral among developers, with nearly 13% of Vercel's paid teams trying it within a day of its gateway arrival. Three weeks after Jev's launch, OpenAI rolled out a competing Decisions API built on GPT-6 Luna.
Jev Media & TalksArticle#community#press#competition
Community
TNW reports Jev reached nearly 13% of Vercel's paid teams within 24 hours, the AI Gateway's fastest adoption: twice the GPT-5.6 family's share and more than 6x Fable 5.1's, at $0.042 per million input tokens with output unmetered.
Jev Media & TalksArticle#community#press#adoption
Community
INXEN's syndication of Financial Times reporting says TypeSafe, last valued at $200 million, is fielding offers that would value it at $10 billion or more days after launching Jev, with Vercel citing record paid-developer interest.
Jev Media & TalksArticle#community#press#funding
Community
Langfuse's September 18 post presents Jev as a decision model for routing, classification and eval verdicts, and walks through scoring production traces with Jev-as-a-judge in Langfuse.
Jev Media & TalksArticle#community#press#explainer
Community
A playground operator's audit of every published Jev number: TypeSafe's four-workflow evals, the favourable homepage demo, independent tests and its own calls. It concludes the price and speed are real and the intelligence claim is roughly mid-tier frontier on decision workflows.
Jev Media & TalksArticle#community#press#explainer
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.