Jev directory
CommunityJev Benchmarks & Evaluations
ROR Set Point Matching OpenAlex Jev Distill Dataset
Hugging Face dataset of teacher judgments for ROR affiliation point matching, named for Jev distillation. Each row pairs an affiliation text with ROR identifiers and calibrated probabilities, across 200k rows in train and validation splits.
- Reading time
- 2 min
- Published by
- Community
- Added
- Nov 2026
Highlights
- Hugging Face dataset of teacher judgments for ROR affiliation point matching, named for Jev distillation.
- Single teacher_judgments subset with 200k rows: 196k train and 3.98k validation.
- Rows pair affiliation text with ROR IDs and probabilities such as 0.98 and 0.02.
- Tagged multilingual, with English, Chinese, and Spanish affiliation strings in the samples.
- Listed libraries: Hugging Face Datasets, pandas, and Polars; files auto-converted to Parquet.
Questions about this entry
Short answers drawn from the linked project or page.
Each row has an id string, a text field holding an affiliation or institution string, and a judgments list of objects that pair a ror_id URL with a probability p, such as 0.98 or 0.02.
The single teacher_judgments subset holds 200k rows, divided into a train split of 196k rows and a validation split of 3.98k rows.
The Hugging Face dataset card lists the license as other. Check the card and the repository files on Hugging Face for the exact terms before redistributing the data or training on it.
The dataset is tagged multilingual, and the sample rows include affiliation strings in English, Chinese, and Spanish, covering institutions in India, China, Mexico, France, Japan, and Canada.
It is published on Hugging Face with the Datasets library listed alongside pandas and Polars, and the files are auto-converted to Parquet, so it can be loaded through the Datasets library or read directly as Parquet.
Each judgment pairs a ROR organization identifier with a value p between 0 and 1, so one row can carry several candidate organizations at different confidence levels, for example 0.98 for one and 0.02 for another.
It is a teacher-judgment dataset for ROR affiliation matching, not Jev runtime code, an SDK, or documentation. Projects that need to install or run Jev should look at SDK and guide entries instead.
More Jev resources like this
Independent experiment scoring 400 real job postings against 30 fictional résumés with Jev, producing 12,000 evaluations in about 197 seconds for roughly $0.17, with a 100-posting answer key and an analysis of what the model got wrong.
Jev Benchmarks & Evaluations#github#madewithjev#evaluation
Communitykaucap08/jev-job-matching
Controlled benchmark of Jev against Claude Sonnet 5 on one coding-agent decision: given a bug report, which file needs editing? It scores 200 real SWE-bench Verified issues with cost, latency, and confidence-band routing tables.
Jev Benchmarks & Evaluations#benchmark#evaluation#calibration
CommunityK624/jev-localization-bench
Spring Boot and Spring AI benchmark comparing Java keyword rules, Jev 1.13.0, and Claude Haiku 5.5 on warehouse goods-receipt triage, with a 200-remark synthetic dataset and a Python analysis script.
Jev Benchmarks & Evaluations#benchmark#evaluation#calibration
Communitymohammedahmadi1990/jev-triage
CodeCut article by Khuyen Tran comparing TypeSafe Jev, RapidFuzz, and Sentence Transformers on the Abt-Buy product matching benchmark, with per-tool merge counts and a walkthrough of asking Jev which product is the same.
Jev Benchmarks & EvaluationsArticle#evaluation#benchmark#classification
Community
Portuguese-language technical study of Jev, TypeSafe AI's System One decision model, benchmarked against a structured-output LLM baseline. Ships a Python 3.11 harness, a 45-task synthetic eval set, and a runner that records decisions, confidence, latency, tokens, and estimated cost.
Jev Benchmarks & Evaluations#github#evaluation#calibration
CommunityGabriel-Abi-Acl/jev-study
arXiv paper comparing two releases of Ru-Jev, a 307M-parameter multilingual cross-encoder trained on thirteen Russian-language tasks, to test whether an augmentation that presents labels in several surface forms helps when callers supply their own option wording.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.