Docs
CommunityJev Benchmarks & Evaluations
Open-Jev Typed Decision Datasets (11-47)
Twelve frozen, mostly synthetic typed-decision configs from the independent Open-Jev project, published as redistributable parquet splits with manifests, raw records and reconstruction instructions; the card states these are not official TypeSafe training data.
- Format
- Docs
- Reading time
- 2 min
- Published by
- Community
- Author
- 11-47
- Last verified
- Oct 2026
- Capabilities
- Open source
Highlights
- Twelve configs with train, calibration, validation, test and OOD splits; combined totals reach about 520,000 rows and overlap.
- release-v2-redistributable is contained in browser-drone-expansion-v1-redistributable, so totals must not be added as a clean corpus.
- Decision shapes: a yes/no probability, a distribution over choices, independent label probabilities, or a numeric or ordinal decision.
- Repo keeps original manifests, exact raw records, source code and reconstruction instructions under CC0-1.0.
- The card says the tasks are controlled and mostly synthetic reference data, not TypeSafe/Jev training data or capability evidence.
Watch out
Card reviewed 2026-10-17; the card itself states the tasks are controlled and mostly synthetic reference data, not official TypeSafe/Jev training data, model predictions or evidence of general capability, and that Open-Jev is independently implemented and unaffiliated with TypeSafe. Config totals overlap because one config contains another.
Questions about this entry
Short answers drawn from the linked project or page.
A dataset repo publishing twelve frozen, mostly synthetic typed-decision configs from the independent Open-Jev project, together with original manifests, exact raw records, source code and reconstruction instructions.
Four: a yes/no probability, a distribution over choices, independent label probabilities, and a discrete numeric or ordinal decision. The card describes the tasks as controlled and mostly synthetic reference labels.
Yes. release-v2-redistributable is contained in browser-drone-expansion-v1-redistributable, so the twelve config totals add up with overlap rather than to one clean corpus.
Totals run from 4,000 rows in citation-control-v1 to 160,797 in browser-drone-expansion-v1-redistributable; combined the twelve configs reach about 520,000 rows with overlap, each split into train, calibration, validation, test and OOD.
No. The card says Open-Jev is independently implemented and unaffiliated with TypeSafe, and that these are not official TypeSafe/Jev training data, model predictions or evidence of general capability.
The repository declares CC0-1.0, so the redistributable configs carry no attribution requirement, but the card does not suggest the mixture matches Jev's own training data.
Rows follow the same request shape: a state plus typed questions answered with probabilities, such as yes/no, choice distributions or discrete scores, with reference labels rather than generated text.
More Jev resources like this
The research record of the Xyntetik typed-decisions track: TD-000 measurements and the preregistered TD-001 and TD-002 tests of whether /v1/decide probabilities from Xyntetik-Kvist-14B are usable as decisions. Both tests are INCONCLUSIVE; the record holds questions, scored rows, analyses and the harness. Apache-2.0.
Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration
Community
johannhartmann's German translation of the LocalLLaMA/typed-decisions benchmark at revision c806f6c: states, instructions, options and rubrics translated to German while keys, ids and gold distributions stay as in the original. Machine-translated without human review. Apache-2.0.
Jev Benchmarks & EvaluationsDocs#community#evaluation#typed-decisions
Community
Twelve frozen typed-decision configs from the independent Open-Jev project, publishing controlled and mostly synthetic tasks with probability or ordinal targets: from the 113,568-row default release to the 160,797-row browser and drone expansion. CC0 1.0; English, Chinese and Turkish.
Jev Benchmarks & EvaluationsDocs#community#evaluation#typed-decisions
Community
m-lagnajit's typed-decision dataset v2.1: 21,591 short-text rows with a human gold label, a calibrated soft label and the raw teacher ingredients, plus v2.1's corrected label that swaps the teacher's wrong top picks.
Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration
Community
8,916 LLM-written items with several typed questions each, following the Jev request shape: 28,661 choice, noul and score decisions across 314 workflows, labelled by averaged model probabilities.
Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration
Community
326,619 typed decision records from the frozen community-hard-mix-v2-final mixture: a redistributable projection of the data behind Open-Jev 27B training, with 2,053 Wikispeedia rows omitted.
Jev Benchmarks & EvaluationsDocs#community#benchmark#evaluation
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.