Jev directory
CommunityJev Models & Reimplementations
UltraBERT Jev 380M Softmax Yes/No
A Hugging Face checkpoint that fine-tunes UltraBERT-380M into a Jev-style decision model: it reads a state, a question and a list of options and picks one in a single forward pass. Reports 72.6% mean accuracy over 13 datasets on the test split.
- Reading time
- 2 min
- Published by
- Community
- Added
- Nov 2026
Highlights
- Reads a state, a question and a list of options, then picks one option in a single forward pass without generating text.
- Fine-tuned from dmis-lab/UltraBERT-380M with one mask token placed in front of each option.
- Reports 72.8% mean accuracy on the dev split and 72.6% on the test split across 13 datasets.
- Trained for one epoch at batch 32 and learning rate 1.5e-5 on the Reason+Long mixture with Nimble added.
- Requires fp32 weights; bf16 changed accuracy by up to 8 points on long documents in MuSiQue.
Questions about this entry
Short answers drawn from the linked project or page.
Use transformers with AutoTokenizer and AutoModelForMaskedLM from the repo Athekunal/ultrabert-jev-380m-softmax-yesno, loading the weights in fp32 and calling model.to('cuda').eval(). The model card states it was tested with transformers 5.20.2 and torch 2.9, which include the UltraBERT model class.
It picks one option from a supplied list in a single forward pass and does not generate text. Option k is scored from the masked-token logits at its own mask, using a recipe of softmax over ' yes' minus ' no'.
The masked-token logits are large, around 200, and the model card reports that bf16 weights changed accuracy by up to 8 points on long documents in MuSiQue. All reported numbers use fp32 weights.
No. The model card warns that the base repo's remote code names the weights differently and would leave them unloaded, so the checkpoint should be loaded without trust_remote_code=True.
The model card reports 72.8% mean accuracy on the dev split and 72.6% on the test split, averaged over 13 datasets.
The checkpoint is released under apache-2.0 and is a fine-tune of dmis-lab/UltraBERT-380M. It is tagged as a decision-model, multiple-choice encoder for English.
No. The model was trained on a specific prompt format using the tokens ' yes' and ' no', and the card states that plain fill-mask use will not give the reported results. The prompt must be used exactly as documented.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsMore Jev resources like this
MLX precision variants of Jev-Style-2B-Decision-v4.2, a 1.88B-parameter text-only decision model for Apple silicon. Ships bf16 and 8-bit weights with a custom runtime that returns a probability for every supplied option.
Jev Models & Reimplementations#open-models#local#python
Community
LoRA adapter for Qwen3.5-4B that scores medical multiple-choice options in a single forward pass and returns a short JSON object with the chosen answer key and per-option probabilities, without explanations or chain-of-thought.
Jev Models & Reimplementations#open-models#python#classification
Community
Russian-first open typed-decision model on Hugging Face. Given a context and a set of candidate options it returns a calibrated probability and picks one, covering yes/no, choice, and ordinal scale decisions. Built on mmBERT-base with 307M parameters under an MIT license.
Jev Models & Reimplementations#open-models#calibration#classification
Community
v-jev 1.4 is a Jev-style decision model on Hugging Face in 0.8B and 4B sizes, built on Qwen3.5 backbones. It returns probabilities for typed questions in one forward pass and adds value extraction and image decisions.
Jev Models & Reimplementations#open-models#classification#calibration
Community
A 7.7M-parameter decision model on Hugging Face that reads text or speech and returns one probability per option for choice, score, and noul questions. Speech runs through femto-asr, giving a 10.1M-parameter spoken pipeline.
Jev Models & Reimplementations#open-models#classification#local
Community
A 1.7B-parameter SocialJev preview model on Hugging Face, fine-tuned from Qwen3-1.7B for typed classification in the social sciences. Runs through Transformers with trust_remote_code or the social-jev package with a vLLM backend, returning a choice and probabilities per query.
Jev Models & Reimplementations#open-models#classification#python
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.