Jev directory
CommunityJev Models & Reimplementations
Jeff 1
Gestalt Labs' Jeff is a LoRA adapter on Qwen3-4B-Instruct-2507 that answers Jev-style choice, noul and score questions with labels and probabilities instead of prose, runs locally, and ships a train-your-own guide; on 9,730 fact-checking examples it trails Jev 1.13.0 on accuracy (0.8183 vs 0.8283).
- Reading time
- 2 min
- Published by
- Community
- Author
- Gestalt Labs
- Last verified
- Oct 2026
Used inJev for Software & DevTools
Highlights
- LoRA adapter for Qwen3-4B-Instruct-2507: Jev-style Choice, Noul and Score questions return labels and probabilities.
- Against live Jev 1.13.0 on 9,730 human-labelled fact-checking examples: accuracy 0.8183 vs 0.8283 and Brier 0.2839 vs 0.2750, but ECE 0.0807 vs 0.0932.
- Quickstart is uv: git clone the repo and uv sync, then SystemOneClient(adapter="GestaltLabs/Jeff-1") or a local server on 127.0.0.1:8079.
- Documented weaknesses: weaker on not_enough_info, can treat partial support as supported, does not retrieve evidence, and scores questions separately.
- Apache-2.0 for code and adapter; CUDA and Apple Silicon MPS are supported, and a train-your-own walkthrough covers the Colab fine-tune.
Quickstart
bash
git clone https://github.com/Gestalt-Lab/jeff
cd jeff
uv syncWatch out
Copy reviewed against the model card fetched on 2026-10-15. The comparison is the authors' own run on 9,730 human-labelled examples from FEVER, VitaminC, SciFact and Climate-FEVER; they state that set has also been used for error analysis and is not an untouched benchmark, and that a lower ECE does not guarantee an individual prediction is correct.
Questions about this entry
Short answers drawn from the linked project or page.
A LoRA adapter for Qwen3-4B-Instruct-2507 that returns labels and probabilities instead of generated prose. It supports three question types - choice, noul and score - with label names and descriptions supplied at call time.
It supports Jev-style Choice, Noul and Score requests and is an independent project, not affiliated with TypeSafe AI. The authors note that API compatibility does not imply identical judgments or performance, so treat the numbers as its own.
On 9,730 human-labelled fact-checking examples from FEVER, VitaminC, SciFact and Climate-FEVER, Jeff scored 0.8183 accuracy versus Jev 1.13.0's 0.8283, with Brier 0.2839 versus 0.2750, but a lower ECE (0.0807 versus 0.0932).
Clone github.com/Gestalt-Lab/jeff and run uv sync; the walkthrough uses uv run python for examples. The first load downloads the base model and adapter. A local HTTP server (uv run python -m scripts.jev_clf_server) exposes POST /v1/systemone, GET /health and GET /v1/models.
Inference needs enough memory for the 4B base model; the card says CUDA and Apple Silicon MPS are both supported. After the initial download, inference does not require a hosted model API.
It can mark a claim as supported when evidence supports only part of it or merely discusses the same topic, is weaker when the right answer is not_enough_info, does not retrieve or verify evidence itself, and scores questions separately, so labels needing whole-document context suffer.
Yes. The training walkthrough covers representing a task as states, questions and labelled answers, preparing train, validation and test splits, fine-tuning a LoRA on a Colab GPU, and evaluating and loading the adapter with the same client.
More Jev resources like this
Zefan Cai's LoRA adapter and scalar decision head for Qwen3.5-9B score caller-supplied candidates for choice, yes/no and score answers without generating text, served over a local /v1/systemone endpoint.
Jev Models & Reimplementations#community#catalog-import#madewithjev
Community
HAL-X AI's 322M non-autoregressive decision model returns calibrated answers in one forward pass of about 10 ms, with native number, excerpt and citation outputs beyond Jev's choice, noul and score types. Apache-2.0, post-trained in 18 languages.
Jev Models & Reimplementations#community#catalog-import#madewithjev
Community
An open recipe for a 27B multimodal decision model trained with full-weight SFT on 73,000 examples over one H200, serving choice, noul and score on /v1/systemone with per-checkpoint provenance and released calibration plots.
Jev Models & Reimplementations#community#calibration#qwen
Community
denis-pplx/autojevPython
denis-pplx/autojevPythonA local Jev-compatible server that reads Choice, Score and Noul answers from the logits of any MLX checkpoint or OpenAI-compatible endpoint with no training, serving POST /v1/systemone as a drop-in for the official TypeSafe SDK.
Jev Models & Reimplementations#community#mlx#calibration
Community
bradAGI/rulingPython
bradAGI/rulingPythonAn open-source, Jev-compatible System One model: Dex answers choice, score and yes/no questions over text in a single forward pass from an Ettin-1B encoder, and speaks the same /v1/systemone shape as Jev.
Jev Models & Reimplementations#community#catalog-import#madewithjev
Community
canadaornaw/dexPython
canadaornaw/dexPythonA LoRA plus 24-slot decision head over Qwen3.8-27B from Ming Xu's train-jev project: it serves TypeSafe's System One API with 0.851 Kev transfer-v4 accuracy and ECE 0.053, and needs an 80 GB GPU.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.