Jev directory
CommunityJev Benchmarks & Evaluations
Medical Jev 4B Data
Hugging Face dataset of medical clinical-decision multiple-choice items in Jev request shape, with a clinical_question state and a decision question carrying type, instructions, and criteria. Between 100K and 1M rows across Chinese, English, Japanese, and four more languages.
- Reading time
- 2 min
- Published by
- Community
- Added
- Nov 2026
Highlights
- Multiple-choice medical and clinical-decision dataset hosted on Hugging Face under JuneYao.
- Rows follow Jev request shape: state.clinical_question plus questions.decision with type, instructions, and criteria.
- Between 100K and 1M rows; languages include Chinese, English, Japanese, and four more.
- Carries gold answers, per-option rationales, quality flags, splits, and SHA-256 hashes for questions, items, and answers.
- Includes source evidence excerpts with locators and paths, plus expert-review and frozen-evaluation-isolation fields.
Questions about this entry
Short answers drawn from the linked project or page.
It is a Hugging Face dataset of medical and clinical-decision multiple-choice items, tagged medical, clinical-decision, and multiple-choice, and structured so each row can be read as a Jev-style request with a state and a decision question.
Each row has a request object with state.clinical_question and questions.decision, where decision holds type, instructions, and criteria. Alongside it sit fields such as gold, options, option_rationales, and decision_signature.
The dataset card lists Chinese, English, Japanese, and four more languages, with a size band of 100K to 1M rows. It is listed under the Multiple Choice task category.
Yes. Rows carry a gold field, an options field, option_rationales, and per-option evidence blocks for A, B, C, and D, each with a status, a reason, and an evidence list.
Rows include quality_flags, label_status, expert_reviewed and clinical_expert_reviewed booleans, source_evidence entries with excerpt, locator, path, and sha256, plus question_sha256, item_sha256, and answer_text_sha256 hashes.
Rows expose training_approved, historical_training_input, split, and frozen_evaluation_isolation fields, so training and held-out evaluation subsets can be separated by filtering on those columns.
The row schema contains a license field, but the material shown does not state a single dataset-level license value. Check the dataset card on Hugging Face before redistributing or reusing the data.
More Jev resources like this
Head-to-head benchmark for System One-style classification models across choice, noul and score questions. Jev led both hash-locked suites at 0.974 micro accuracy on v1 (8 tasks, 78 cases) and 0.964 on v2 (49 tasks, 866 cases), ahead of local Von 1.1, GLiNER2 and Laya.
Jev Benchmarks & Evaluations#community#catalog-import#cobanov
Community
jabr/classifier-benchmarkPython
jabr/classifier-benchmarkPythonIndependent experiment scoring 400 real job postings against 30 fictional résumés with Jev, producing 12,000 evaluations in about 197 seconds for roughly $0.17, with a 100-posting answer key and an analysis of what the model got wrong.
Jev Benchmarks & Evaluations#github#madewithjev#evaluation
Communitykaucap08/jev-job-matching
Spring Boot and Spring AI benchmark comparing Java keyword rules, Jev 1.13.0, and Claude Haiku 5.5 on warehouse goods-receipt triage, with a 200-remark synthetic dataset and a Python analysis script.
Jev Benchmarks & Evaluations#benchmark#evaluation#calibration
Communitymohammedahmadi1990/jev-triage
Python project that measures how much CUAD license-agreement text Jev can bin before an expensive model reads it, reporting bin rates, lost lawyer-marked provisions, and estimated cost savings against a Claude Sonnet baseline.
Jev Benchmarks & Evaluations#evaluation#calibration#benchmark
CodeCut article by Khuyen Tran comparing TypeSafe Jev, RapidFuzz, and Sentence Transformers on the Abt-Buy product matching benchmark, with per-tool merge counts and a walkthrough of asking Jev which product is the same.
Jev Benchmarks & EvaluationsArticle#evaluation#benchmark#classification
Community
Hugging Face dataset of teacher judgments for ROR affiliation point matching, named for Jev distillation. Each row pairs an affiliation text with ROR identifiers and calibrated probabilities, across 200k rows in train and validation splits.
Jev Benchmarks & Evaluations#calibration#evaluation#classification
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.