Article
CommunityJev Models & Reimplementations
Learning to Formalize: Bridging Open-Ended Workloads and Fast Decision Models
arXiv paper on a small candidate formalizer trained on offline teacher-generated lists that builds explicit, validated alternatives from task context for a frozen decision model, evaluated on CALVIN-D episodes and annotations.
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Added
- Oct 2026
Highlights
- A small formalizer constructs explicit, validated alternatives from task context for a frozen decision model.
- On 200 CALVIN-D test episodes, Jev-1.13.0 scores 11% with generic motions versus 46.5% with DeepSeek V4 Flash pose candidates.
- A six-arm privileged-state evaluation covers 1,011 CALVIN-D validation annotations.
- A locally deployed decision model reaches at most 52.6% success with the fine-tuned formalizer versus 11.8% without it.
- End-to-end latency is kept to 2.09 s, compared with 3.67 s for Qwen3.5-27B in the same setting.
Questions about this entry
Short answers drawn from the linked project or page.
Decision models offer fast choices over supplied alternatives, while general-purpose language models interpret open-ended tasks through generation and reasoning at the cost of sequential decoding. Many workloads expect a generative backend to decide what can be done and provide no well-defined candidate list.
It is a small formalizer trained on offline teacher-generated lists. It constructs explicit, validated alternatives from task context, and a frozen decision model then selects an entry.
On 200 CALVIN-D test episodes, official TypeSafe Jev-1.13.0 reports 11% success with generic motions, versus 46.5% when DeepSeek V4 Flash pose candidates are supplied.
Across 1,011 CALVIN-D validation annotations, a locally deployed decision model achieves at most 52.6% task success rate with the fine-tuned formalizer, compared with 11.8% without it.
End-to-end latency is kept to 2.09 s, whereas Qwen3.5-27B takes 3.67 s under the exact experiment setting.
The described pipeline keeps the decision model frozen and selects an entry from the formalizer's alternatives; the formalizer is the component trained on offline teacher-generated lists.
The reported ceiling is 52.6% task success with the fine-tuned formalizer on the privileged-state evaluation, so workloads whose alternatives cannot be validated from task context still fall short of the generative baseline's flexibility.
More Jev resources like this
Open-weight vision-language judge that takes a task instruction plus 10 frames of a robot manipulation attempt and predicts success or failure with calibrated probabilities in a single forward pass. Released with inference code and a 20,628-prediction evaluation set.
Jev Models & ReimplementationsArticle#arxiv#open-models#evaluation
Community
arXiv paper proposing JevRec, which encodes recommendation items as variable-length semantic identifiers and adds a Jev-like decision modeling component with separate heads for length prediction and content generation, evaluated on three public datasets.
Jev Models & ReimplementationsArticle#arxiv#research#benchmark
Community
A family of 0.8B-9B ophthalmic decision models that score candidate answers without autoregressive decoding, trained on RetiJev-Data: 78,158 clinical-decision and diagnosis questions and 50,074 guideline-grading questions. RetiJev-9B answers a case in 158 ms on one GPU.
Jev Models & ReimplementationsArticle#arxiv#open-models#benchmark
Community
arXiv paper on Decision-Level Continual Learning from Typed Feedback, a method that keeps Jev-style accept and revoke signals in bounded channels above a frozen scorer, and introduces TCPA-Bench, which credits a correction only through its effect on later decisions.
Jev Models & ReimplementationsArticle#arxiv#research#evaluation
Community
arXiv paper introducing Flide, an exact inference engine for decision models such as Jev. It decouples the logical tree of a decision request from physical execution and reports answers up to 9.5 times faster than vLLM and SGLang.
Jev Models & ReimplementationsArticle#arxiv#benchmark#evaluation
Community
Apache-2.0 Hugging Face repository for RSI-JEV-27b, a 27B multimodal decision model based on Qwen3.8-27B that answers supplied multiple-choice questions over images, returning a selected option and probabilities for all options in one forward pass.
Jev Models & Reimplementations#open-models#benchmark#calibration
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.