Docs
CommunityJev Benchmarks & Evaluations
Jev Alt SystemOne Eval
Evaluation and training artifacts from Jev Alt System One work: teacher merge outputs, baseline summaries and Kev-0.8B plus Kev-4B typed-decision records with the scripts that built them.
- Format
- Docs
- Reading time
- 2 min
- Published by
- Community
- Author
- marcmagn1
- Last verified
- Oct 2026
Works forJev for Evaluation
Used inJev for Software & DevTools
- Languages
- Python
- Capabilities
- Open source
Highlights
- Baselines labelled B1 and B2, each with a summary JSON, plus an aggregate report.json and rows.json under baselines/.
- Teacher artifacts: teacher.json, teacher-nothinking.json, kev4b-traincal.json and judge-metrics.json.
- Training summaries for kev-08b-typed-v1 in blend, rows and summary form, plus a kev-4b-typed-v1 summary.
- code/ holds the pipeline scripts: build_train.py, convert_typed.py, fetch_replay.py, merge_teacher.py and train_driver.py.
Watch out
The dataset card publishes no README, no stated licence and no prose description, so everything available has to be read from file names and repository metadata. There are no download, benchmark or licence claims to verify, and the JSON artifacts should be inspected directly before they are relied on.
Questions about this entry
Short answers drawn from the linked project or page.
No. No README is available, so the repository has to be read from its file names and metadata: baselines, teacher and train JSON artifacts plus the Python scripts under code/.
Two labelled B1 and B2, each with a summary JSON, plus an aggregate report.json and a rows.json, all under the baselines/ directory.
teacher.json and teacher-nothinking.json for teacher outputs, kev4b-traincal.json for Kev-4B training calibration and judge-metrics.json, all under teacher/.
kev-08b-typed-v1-summary.json, kev-08b-typed-v1-rows.json, kev-08b-typed-v1-blend-summary.json and kev-4b-typed-v1-summary.json under the train/ directory.
The scripts named build_train.py, convert_typed.py, fetch_replay.py, merge_teacher.py and train_driver.py, covering typed conversion, replay fetching, teacher merging and the training driver.
No licence is stated on the card, so the artifacts carry no declared reuse terms; the repository metadata lists only the region:us tag.
No verifiable claims are published. Without a README or licence, the JSON summaries are working artifacts to inspect rather than documented benchmark results.
More Jev resources like this
System One Mosaic Benchmark: a Python evaluator and Next.js viewer comparing Jev and open decision models across 137 English Choice, Noul and Score benchmarks from 106 dataset subsets, with recorded evaluation conditions and a public Hugging Face leaderboard.
Jev Benchmarks & Evaluations#community#benchmark#leaderboard
Community
hotchpotch/S1MBPython
hotchpotch/S1MBPythonReal human-labeled data reformatted into System One questions: 22 configs, 166,054 rows and 22,773 test records where choice, score and noul answers are scored as probability distributions.
Jev Benchmarks & EvaluationsDocs#community#benchmark#evaluation
Community
Complete Decision Index 0.2.1 runs of AutoTrust's JEV typed-decision models: engine outputs for JEV-9B, JEV-27B and JEV-Gemma4-26B-A4B, each covering 150,317 scoreable requests with zero errors.
Jev Benchmarks & EvaluationsDocs#community#benchmark#evaluation
Community
326,619 typed decision records from the frozen community-hard-mix-v2-final mixture: a redistributable projection of the data behind Open-Jev 27B training, with 2,053 Wikispeedia rows omitted.
Jev Benchmarks & EvaluationsDocs#community#benchmark#evaluation
Community
English, Turkish and German decision-model test data with probability targets: 3,116 held-out rows across three languages, where Deem-4B scores 94.7% in English against Kev-4B's 84.7%.
Jev Benchmarks & EvaluationsDocs#community#benchmark#evaluation
Community
1,601,157 training and 66,482 test decision questions for JevEmbed, covering choice, score and noul types across ten Parquet shards of at most 200,000 rows each.
Jev Benchmarks & EvaluationsDocs#community#benchmark#evaluation
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.