Article
CommunityJev Benchmarks & Evaluations
SatOps-Decision
arXiv paper introducing SatOps-Decision, a locally adapted Laya encoder with its native decision head that selects among supplied satellite-operations answers in one forward pass, plus a benchmark of 1,024 held-out questions with latency and accuracy comparisons.
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Added
- Nov 2026
Highlights
- Median latency of 90.3 ms per decision in the measured FP32/MPS implementation.
- 5.47x faster than Qwen3.5-0.8B generation and 4.30x faster than OneJev1.6-0.8B on median request times.
- Mean raw accuracy 85.3%, versus 42.5% for frozen Laya and 58.6% for OneJev.
- Cloud references: original Jev 1.14/1.13 reach 87.4/81.9% accuracy.
- All main accuracy comparisons use the same 1,024 held-out questions.
Questions about this entry
Short answers drawn from the linked project or page.
It is a method for low-latency decisions in satellite communication operations. A locally adapted Laya encoder with its native decision head selects among supplied operational answers in a single forward pass.
Median latency is 90.3 ms per decision, versus 493.9 ms for Qwen3.5-0.8B generation and 388.4 ms for OneJev1.6-0.8B, giving 5.47x and 4.30x speedups on median request times in the measured FP32/MPS implementations.
Mean raw accuracy is 85.3%, compared with 42.5% for frozen Laya, 40.7% for Qwen generation and 58.6% for OneJev. As cloud accuracy references, original Jev 1.14/1.13 reach 87.4/81.9%.
The paper describes it as combining authored operational contracts, controlled state changes and independently checked labels in an auditable construction.
All main accuracy comparisons use the same 1,024 held-out questions, kept separate from the training and development data.
It states this is the first application of Jev-style general-purpose discriminative decision models to satellite operations, and that the accompanying benchmark is the first dedicated to this approach.
No. The scorer selects among supplied operational answers rather than generating them, which is why evaluation prioritizes complete-request latency and raw decision accuracy.
More Jev resources like this
Evaluation of MedJev, an open System One model that runs locally on a laptop, across ten medical benchmarks for knowledge, diagnosis, and patient-level prediction on MIMIC IV and MIMIC IV ED, with a two-parameter recalibration method and a released evaluation harness.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv study applying Jev and Clef-Flash to manufacturing defect detection with as few as two reference images per class, including agentic refinement of text descriptions and probability calibration. Ships replication code, an R decisions package, and a no-code web app.
Jev Benchmarks & EvaluationsArticle#arxiv#calibration#classification
Community
Case study evaluating Jev 1.13.0 on time-series decision tasks, including classification, event type prediction, oil temperature direction, and stock movement, converting continuous forecasts into discrete decisions and scoring accuracy and AUC.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv paper introducing PrimaCons, a four-axis benchmark measuring whether Jev gives consistent decisions when the same semantic state is expressed as different typed query primitives, plus the Cross-Primitive Residual Calibrator (CPRC) proposed to reduce that inconsistency.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv paper presenting FlipJev, a fast black-box query-based red-teaming attack that finds short universal adversarial suffixes for Jev. It reports experiments on 12 datasets, 10,240 paired prompt-injection tests, and cross-task suffix transfer of 90-100%.
Jev Benchmarks & EvaluationsArticle#arxiv#security#evaluation
Community
arXiv paper scoring candidate pages by the citation share they will receive, comparing pretrained rerankers, Jev, eight open decision models and distilled students on GEO-Bench queries answered by ten engines. Releases GEO Citation Ranker 4B, the benchmark, all scorer predictions and code.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.