User repository on GitHub
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Models & Reimplementations
Jev-MedQA
Jev-style medical QA implementation on Qwen3.5-4B that selects an answer from candidates in one forward pass instead of generating text. It reports 69.42% accuracy versus 67.41% for standard generation and a 10.37x speedup across 153,889 questions from nine benchmark sets.
- Stars
- 2
- Language
- Python
- License
- MIT
- Published by
- Community
- Last verified
- Oct 2026
About this project
Choose directly from the candidate answers, using the same model for clinical text and medical images. Medical exams and image-based questions often ask for one answer from a short list: which diagnosis best fits this case, or which finding appears in this image? A generative model may produce several paragraphs before giving its choice. When the task needs a selected answer, generating that response adds time to every question.
Quoted from the project's README. Read the full README.
- Languages
- Python
- Capabilities
- Open sourceRuns locally
Highlights
- Selects answers by reading Qwen3.5-4B option scores in one forward pass, with no additional training
- 153,889 questions from nine medical QA sets: 69.42% vs 67.41% accuracy, 1.891 s to 0.182 s per question
- Handles text and image questions through one interface; MedXpertQA-MM runs at 1/69.87 of generation time
- Multi-GPU launcher validates a sample, shards the workload and supports --resume after interruptions
- Built from SemIf; code is MIT while datasets and model weights follow their upstream terms
Watch out
MIT code, but datasets and weights keep upstream licenses, and this is a Jev-style paradigm implementation rather than TypeSafe's hosted Jev. Measurements used RTX 3090 GPUs, one question at a time, thinking mode disabled.
Questions about this entry
Short answers drawn from the linked project or page.
No. It applies the Jev paradigm, reading scores for candidate answers in one forward pass, to an untrained Qwen3.5-4B checkpoint. The code is MIT, but this is an independent research implementation.
Across 153,889 questions from nine medical QA benchmark sets it reports 69.42% accuracy versus 67.41% for the same model generating answers normally, at 0.182 seconds per question.
One Qwen3.5-4B forward pass over the question, any images and the candidate list. The raw scores the model gives the option letters are logits, and applying softmax turns them into relative scores across the choices, so the highest-scoring letter is returned along with the other candidates' scores. No additional training is involved.
10.37x overall, from 1.891 s to 0.182 s per question. The image sets vary sharply: OmniMedVQA 4.75x, PMC-VQA 15.21x, and MedXpertQA-MM 69.87x, where accuracy is essentially tied at 27.95% against 28.00% for standard generation.
Both methods were measured on RTX 3090 GPUs, one question at a time, with thinking mode disabled. The evaluated portions cover MedQA, MedMCQA, six MMLU medical subjects, MedXpertQA-Text, Medbullets, AfriMed-QA MCQ, MedXpertQA-MM, OmniMedVQA and PMC-VQA, and overall accuracy is total correct answers divided by total questions.
Create a Python 3.10 conda environment, pip install -r requirements.txt, convert your datasets to the documented JSONL, then run scripts/evaluate.py with --gpu-ids, --model, --inputs, --answers and --output. The launcher checks GPU status, validates a sample, shards the workload and supports --resume. Code is MIT and built from SemIf, but datasets and weights follow their upstream terms.
Yes, through one interface. The forward pass covers the question, any accompanying images and the candidate list together, which is why MedXpertQA-MM runs at 1/69.87 of generation time, though accuracy there is a tie at 27.95% against 28.00%. The speedup comes from never generating text, not from doing better on the hard image questions.
More Jev resources like this
130GitHub stars
An open recipe for a 27B multimodal decision model trained with full-weight SFT on 73,000 examples over one H200, serving choice, noul and score on /v1/systemone with per-checkpoint provenance and released calibration plots.
Jev Models & Reimplementations#community#calibration#qwen
Community
denis-pplx/autojevPython
denis-pplx/autojevPython59GitHub stars
A preregistered measurement of a Jev-style interface on frozen Qwen3-4B: read the option letter's logits instead of generating grammar-constrained JSON.
Jev Models & Reimplementations#community#python#local
Community
r-ms/mini-jevPython
r-ms/mini-jevPython8GitHub stars
Open recipe for calibrated decision models: an LLM teacher writes training data from a one-sentence task description, a LoRA plus decision head on Qwen3-4B-Base answers Choice, Score and Noul questions with calibrated probabilities, evaluated next to Jev on identical items.
Jev Models & Reimplementations#community#calibration#training
Community
scienthoon/lucePython
scienthoon/lucePython3GitHub stars
A training-free runtime that turns dense Qwen3 or Qwen3.5 checkpoints into a Jev-style Noul, Choice and Score service, sharing one state prefill across isolated questions and reading each answer from candidate-token logits.
Jev Models & Reimplementations#community#qwen#local-inference
Community
HopLee6/QwevPython
HopLee6/QwevPythonUnsloth's guide to turning a generative LLM into a Jev-style decision model that returns a choice with a probability instead of text, plus a companion page for serving Laya behind the TypeSafe API.
Jev Models & Reimplementations#community#fine-tuning#open-models
Community
The fine-tuned Laya checkpoint for four typed-decision workflows: 0.766 accuracy on 2,000 held-out decisions, above Jev 1.13.0's published 0.727 and the teacher self-agreement ceiling it trained past.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Jev takes text only; Clef-omni adds image, audio, and video input and Liquid d1 adds images, each keeping the same typed-answer shape. Where that matters, where it does not, and the costs the vendors underplay.
Jev Models & Reimplementations#multimodal#open-models#decision-model
Read article
One model in September, a dozen by October. Clef, Liquid d1, Perplexity, and OpenAI all ship the same typed-answer shape now. How to read the rival benchmarks and pick on your own task.
Jev Models & Reimplementations#decision-model#benchmarks#pricing
Read article
Hundreds of repositories carry the system-one tag, but the ecosystem is a dozen ideas in different sizes. A tour of the families, the licenses, and what a download does not include.
Jev Models & Reimplementations#open-models#hugging-face#local
Read article
From the Jev community
Posts from builders shipping with Jev right now.