Jev directory
CommunityJev Benchmarks & Evaluations
Convex Decision Evals
A Convex page for comparing models that answer verified multiple-choice questions about Convex, scoring them with source-weighted results and showing latency, cost and inspectable outcomes alongside the scores.
- Reading time
- 2 min
- Published by
- Community
- Author
- Convex
- Last verified
- Oct 2026
Highlights
- Compares models answering verified multiple-choice questions about Convex on a leaderboard-style page.
- Reports source-weighted scores together with latency, cost and inspectable outcomes.
- Sits under Convex's evals section at convex.dev/evals; the page body is client-rendered and did not reach the digest.
Watch out
Grounded only in the page's title and meta description captured on 2026-10-15; the page body is client-rendered and none of it reached the digest, so the models compared, the question count and the scoring formula are unknown here.
Questions about this entry
Short answers drawn from the linked project or page.
An evals page titled 'Decision Evals - Compare model performance'. Its description says it compares models answering verified multiple-choice questions about Convex, with source-weighted scores plus latency, cost and inspectable outcomes.
Verified multiple-choice questions about Convex, according to the page's own description. The digest did not capture the questions themselves, so this entry cannot describe their format or difficulty.
The description lists source-weighted scores, latency, cost and inspectable outcomes. That suggests the board pairs quality with speed and price rather than reporting accuracy alone, but the exact presentation is not documented in the digest.
The captured text does not name any of the models. It only says models are compared on verified multiple-choice questions about Convex, so check the live page for the current lineup.
The description promises inspectable outcomes, which implies you can look past the aggregate score, but the digest did not capture the page body and cannot describe what an inspection view contains.
The digest for this page does not name any model, Jev included. It appears in the cobanov and yibie Jev awesome lists, which is the list context this import came from, but the page text itself was not captured.
Open https://www.convex.dev/evals/decision in a browser; the fetch succeeded with HTTP 200 but the page is client-rendered, so the comparison loads in the browser after JavaScript runs.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Microsoft-Decision-1 against Jev on a private eval
X: Vals AI's independent evaluation of Jev
X: An eval suite for System One models
X: Jev and Laya play chess instead of benchmarks
More Jev resources like this
Head-to-head benchmark for System One-style classification models across choice, noul and score questions. Jev led both hash-locked suites at 0.974 micro accuracy on v1 (8 tasks, 78 cases) and 0.964 on v2 (49 tasks, 866 cases), ahead of local Von 1.1, GLiNER2 and Laya.
Jev Benchmarks & Evaluations#community#catalog-import#cobanov
Community
jabr/classifier-benchmarkPython
jabr/classifier-benchmarkPythonBenchmark harness that scores TypeSafe Jev against any OpenRouter model or local checkpoint on labelled classification data: accuracy, calibration (ECE), confidence distribution, latency and cost across five tasks with 300 items each.
Jev Benchmarks & Evaluations#community#catalog-import#cobanov
Community
4esv/jev-evalPython
4esv/jev-evalPythonIndependent measurements of typed-decision models: Jev (TypeSafe API) against Laya (open weights) on 40 Chinese support-ticket classifications, plus a local-first cascade that matches Jev's accuracy at about 1.8x the speed.
Jev Benchmarks & Evaluations#community#catalog-import#logicrw
Community
yibie/laya-jev-labPython
yibie/laya-jev-labPythonA reproducible experimentation framework and report series for Jev: it runs shared tasks at scale across benchmarks, routing and numeric prediction, and publishes bilingual findings on where the model holds up and where it does not.
Jev Benchmarks & Evaluations#community#python#evals
Community
keta1930/what-the-jevPython
keta1930/what-the-jevPythonA measured audit of 300 public pages for AI citability using nine Jev decisions per page, checked against an LLM judge: Jev agreed on 85.8% of 900 decisions and cost 158 times less per page, with latency measured for both.
Jev Benchmarks & Evaluations#community#catalog-import#logicrw
Community
stas4000/jev-geo-auditPython
stas4000/jev-geo-auditPythonBenchmark results for JEV 1.13 in Choice mode across MetaTool, When2Call and BFCL V4: tool selection, abstention and hallucination tables with bilingual reports and CSV data.
Jev Benchmarks & Evaluations#community#catalog-import#cobanov
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.