Docs
CommunityJev Benchmarks & Evaluations
Oss Jev Train Src
gyanbhambhani's source package for running the oss-jev training job on Hugging Face Jobs without a custom Docker image: jobs_uv_entry.py is a PEP 723 entry point, and it downloads the oss-jev-distill pack in-script when /data is not mounted. Apache-2.0.
- Format
- Docs
- Reading time
- 2 min
- Published by
- Community
- Author
- gyanbhambhani
- Last verified
- Oct 2026
Works forJev for Evaluation
Used inJev for Software & DevTools
- Languages
- Python
- Capabilities
- Open source
Highlights
- Source package for hf jobs uv run; the card says no custom Docker image is needed.
- Entry point is jobs_uv_entry.py, a PEP 723 script that declares its own dependencies inline.
- Downloads gyanbhambhani/oss-jev-distill inside the script when /data is not present.
- Published 2026-10-16 under Apache-2.0, tagged oss-jev, training and jobs.
Watch out
Card reviewed 2026-10-17; the card is four lines long, so the training recipe, hardware, hyperparameters, output artifacts and evaluation of the oss-jev track are not documented here. The package downloads a separate teacher-labelled draft pack, and no results of any run of this job are recorded on the card.
Questions about this entry
Short answers drawn from the linked project or page.
gyanbhambhani's source package for running the oss-jev training job on Hugging Face Jobs. The card describes it as the source for hf jobs uv run without a custom Docker image, with a PEP 723 entry point and a companion distill dataset the script fetches when no data is mounted.
The entry point is jobs_uv_entry.py, a PEP 723 script that declares its own dependencies, so no custom Docker image is built. The card gives no command line beyond describing the hf jobs uv run flow, so check the repository files for the exact invocation.
From gyanbhambhani/oss-jev-distill: the card says the script downloads that distill pack when /data is absent. That companion dataset is a teacher-labelled pack for the one-token Qwen decision track, so the source package and the distill data are designed to be used together.
Apache-2.0, per the repository metadata. That covers the source package; the distill data it downloads is a separate repository, and the card does not restate terms for the models or outputs the job produces.
It is part of the community oss-jev distillation pipeline, not official TypeSafe tooling. The distill pack it downloads is labelled by jev-1.13.0 on that pack's own card, so the pipeline reproduces the teacher's typed decisions, but this card itself documents only the packaging.
The companion distill pack holds 10,044 rows: 7,909 train, 1,074 eval and 1,061 calibration, plus 1,708 labelled fixtures. Those counts come from the distill card; this source package declares no sizes of its own.
Everything operational: hardware, runtime, hyperparameters, output artifacts and evaluation. It is a four-line card covering the entry point and the distill download, so treat the repository files as the authority for anything beyond packaging.
More Jev resources like this
A worked example of using Jev as an LLM judge: ten synthetic frozen refund-support traces are scored by three yes/no criteria and one ordered helpfulness rubric in a single Jev request and recorded as a Comet Opik experiment.
Jev Benchmarks & Evaluations#community#catalog-import#madewithjev
Community
patchy631/jev-as-judgePython
patchy631/jev-as-judgePythonA fast, typed Python framework from TrustifAI for evaluating LLMs, RAG pipelines and agents with decision models: Jev by default, Microsoft Decision-1 on OpenRouter or OpenAI's Decisions API through the same backend. It returns typed scores, thresholds and status, with optional human-label calibration.
Jev Benchmarks & Evaluations#community#github#python
Community
trustifai/typed_evalsPython
trustifai/typed_evalsPythonA BEIR reranking recipe that asks up to 30 parallel Noul relevance questions in one Jev state, with RESULTS.md against cross-encoders and hosted rerankers.
Jev Benchmarks & Evaluations#community#evaluation#python
Community
hev/rerankerPython
hev/rerankerPythonSystem One Mosaic Benchmark: a Python evaluator and Next.js viewer comparing Jev and open decision models across 137 English Choice, Noul and Score benchmarks from 106 dataset subsets, with recorded evaluation conditions and a public Hugging Face leaderboard.
Jev Benchmarks & Evaluations#community#benchmark#leaderboard
Community
hotchpotch/S1MBPython
hotchpotch/S1MBPythonEvaluation scripts, MCP tools, and a skill router built to measure where jev-1.13.0 fits in agent harnesses, with raw results under eval/.
Jev Benchmarks & Evaluations#community#evaluation#agents
Community
Aitejiu/jev-harness-labPython
Aitejiu/jev-harness-labPythonAn educational lab for DeepEval plus Jev: replay 17 recorded Jev evaluations of fictional support cases, inspect raw requests and responses, adjust release gates and drift playback, or run fresh evaluations with your own key.
Jev Benchmarks & Evaluations#community#catalog-import#logicrw
Community
shyamsridhar123/judgejevPython
shyamsridhar123/judgejevPythonFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.