Docs
CommunityJev Benchmarks & Evaluations
Oss Jev Distill
Teacher-labelled distillation pack for the one-token Qwen decision track: 10,044 rows across train, eval and calibration JSONL files, plus 1,708 labelled fixtures. Labels come from jev-1.13.0, and the card says not to train on the eval or calib splits. Apache-2.0.
- Format
- Docs
- Reading time
- 2 min
- Published by
- Community
- Author
- gyanbhambhani
- Last verified
- Oct 2026
- Languages
- Shell
- Capabilities
- Open source
Highlights
- 10,044 rows total: 7,909 train, 1,074 eval and 1,061 calibration, plus 1,708 labelled fixtures.
- Teacher is jev-1.13.0; the schema version is oss-jev-distill-v1 for the one-token Qwen decision track.
- Root layout is train.jsonl, eval.jsonl, calib.jsonl and manifest.json; do not train on eval or calib.
- Jobs mount shown on the card: -v hf://datasets/gyanbhambhani/oss-jev-distill:/data.
- Apache-2.0, task category text-classification, JSON files, created 2026-10-16.
Watch out
Card reviewed 2026-10-17; the card documents counts, the teacher id and the jobs mount only, so the row schema, question types and label construction are not described. The eval and calib splits are explicitly excluded from training, and no evaluation results or baseline comparisons for the track are published on the card.
Questions about this entry
Short answers drawn from the linked project or page.
A teacher-labelled distillation pack for the one-token Qwen decision track, published under the schema version oss-jev-distill-v1. Each record is part of a training corpus meant to teach a Qwen model the typed-decision behaviour of its teacher.
10,044 rows in total: 7,909 train, 1,074 eval and 1,061 calibration, plus 1,708 labelled fixtures. The card calls it the about-10k pack, and the split table sums to exactly 10,044.
The card names jev-1.13.0 as the teacher for every record. Teacher probabilities are what the student learns from, so the pack reproduces a specific Jev version's decisions rather than fresh human labels.
The card shows the jobs mount as -v hf://datasets/gyanbhambhani/oss-jev-distill:/data, which places the files at /data inside the job. The root layout is train.jsonl, eval.jsonl, calib.jsonl and manifest.json.
Train only. The card says explicitly not to train on eval or calib, so those splits remain usable as held-out evaluation and calibration sets for whatever you train on the pack.
Apache-2.0, with the task category text-classification and JSON files. The data is plain JSONL plus a manifest, so no custom loading code is needed beyond mounting or downloading the repository.
The row schema, how the questions and distributions are constructed, the model sizes involved, and any evaluation results for the track. The card is limited to counts, the teacher id, the mount line and the root layout.
More Jev resources like this
The training and validation data behind jaeyong2's clef-flash-jev-claude-distill fine-tune: 37,932 states and 150,827 typed questions with soft target distributions, plus a 19,025-record language pool. The card reports the fine-tune lifts the Decision Index public suite from 52.55 to 54.96. Apache-2.0.
Jev Benchmarks & EvaluationsDocs#community#evaluation#distillation
Community
m-lagnajit's typed-decision dataset v2.1: 21,591 short-text rows with a human gold label, a calibrated soft label and the raw teacher ingredients, plus v2.1's corrected label that swaps the teacher's wrong top picks.
Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration
Community
2.5 million typed decisions recast from 670 sources: choice, score, and noul rows with per-row licenses, soft annotator targets, and Jev-audited mix weights.
Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration
Community
The research record of the Xyntetik typed-decisions track: TD-000 measurements and the preregistered TD-001 and TD-002 tests of whether /v1/decide probabilities from Xyntetik-Kvist-14B are usable as decisions. Both tests are INCONCLUSIVE; the record holds questions, scored rows, analyses and the harness. Apache-2.0.
Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration
Community
johannhartmann's German translation of the LocalLLaMA/typed-decisions benchmark at revision c806f6c: states, instructions, options and rubrics translated to German while keys, ids and gold distributions stay as in the original. Machine-translated without human review. Apache-2.0.
Jev Benchmarks & EvaluationsDocs#community#evaluation#typed-decisions
Community
Twelve frozen, mostly synthetic typed-decision configs from the independent Open-Jev project, published as redistributable parquet splits with manifests, raw records and reconstruction instructions; the card states these are not official TypeSafe training data.
Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.