Article
CommunityJev Benchmarks & Evaluations
EditJev: System One Routing for Image Editing Models
An arXiv paper introducing EditJev, a System One routing model that predicts sample-level preference scores over candidate image editing models in a single forward pass, then combines them with model speed scores to trade editing quality against latency.
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Added
- Oct 2026
Highlights
- Routes among candidate image editing models per image and instruction.
- A lightweight preference head on a vision-language backbone emits a fixed-dimensional score vector.
- No autoregressive model-name generation or response parsing at inference.
- Trained on pairwise preferences from offline candidate outputs with a pairwise ranking objective.
- Reports quality gains from 4.540 to 4.592 on ImgEdit and 8.194 to 8.255 on GEdit-En.
Questions about this entry
Short answers drawn from the linked project or page.
Instruction-based image editing models vary in quality per sample, and the best model often differs across requests, while inference latency also varies widely. EditJev routes each image and instruction to a candidate editor instead of deploying one fixed model.
A lightweight preference head on a vision-language backbone produces a fixed-dimensional score vector in a single forward pass, avoiding autoregressive model-name generation and response parsing.
Pairwise preferences are constructed from offline candidate outputs, and a pairwise ranking objective is optimized so the model learns relative suitability among candidate editors.
At inference it combines predicted preference scores with model-level speed scores, so selection reflects both editing quality and generation latency.
Experiments were run on ImgEdit and GEdit-En, comparing aggregate quality against the strongest fixed editor.
Aggregate quality improves from 4.540 to 4.592 on ImgEdit and from 8.194 to 8.255 on GEdit-En, with estimated end-to-end latency reduced by 60.8% and 31.7% respectively.
No. It is a System One routing model that outputs a fixed-dimensional score vector over candidate image editing models rather than generating text.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
Facebook: Intake Router keeps diagnosis out of the answer space at the front desk.
FacebookClinicnotes
X: OpenRouter Decision Model Rankings
X: herdr + jev, an agent router
X: A group chat bot with decision room routing
More Jev resources like this
An evaluation of JEV-as-a-Judge, a decision-only judge that returns label probabilities and uses its confidence to accept a verdict or escalate to a reasoning judge. Against sixteen generative and reward-model judges it comes within three points of GPT-6 where a verdict can be read off the text, at 0.36% of its fee.
Jev Benchmarks & EvaluationsArticle#community#research#arxiv
Community
Case study evaluating Jev 1.13.0 on time-series decision tasks, including classification, event type prediction, oil temperature direction, and stock movement, converting continuous forecasts into discrete decisions and scoring accuracy and AUC.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv paper introducing PrimaCons, a four-axis benchmark measuring whether Jev gives consistent decisions when the same semantic state is expressed as different typed query primitives, plus the Cross-Primitive Residual Calibrator (CPRC) proposed to reduce that inconsistency.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv study comparing Jev 1.13 with Clef, Microsoft-Decision-1, Laya, GPT-6 Luna and GPT-6 Sol on 172 rheumatology exam questions and 70 diagnostic vignettes, reporting accuracy, calibration and language effects.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv paper introducing GDGraph, a guideline-derived decision graph framework that compiles radiology guidelines into an executable graph and has a System One model answer localized clinical questions, reporting 90.8% accuracy across 347 cases.
Jev Benchmarks & EvaluationsArticle#arxiv#research#evaluation
Community
Reference audit of typed decision models that measures what a returned probability licenses, comparing open-weight Laya checkpoints with Jev 1.13 over 190 exchangeable evaluations, locale coverage, drift, and explanation limits. Releases frozen answers, a datasheet, a promise check, and code.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.