CommunityJev Models & Reimplementations
Laya Multilingual
The 322M mmBERT-base build of Laya for 100+ languages: state plus typed questions return typed answers with probabilities in one forward pass, with up to 8,192 tokens for long documents.
- Published by
- Community
- Author
- Convai Innovations
- Last verified
- Oct 2026
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- 322M total: mmBERT-base backbone (307M, 22 layers, hidden 768, 256k vocab) plus a decision head trained from scratch.
- On all 51 MASSIVE languages it reaches 0.366 macro accuracy against the English checkpoint's 0.227, with 45 of 51 clearing 3x random.
- XNLI: 0.731 across 14 non-English languages against 0.521, while English slips slightly to 0.843 from 0.860.
- Faster than the English build: 337 ms per call at 50 questions, 6.8 ms per question, on one T4.
- Trained from scratch with RLCD in 15,987 updates over 4 epochs, about 4.97 hours.
Watch out
Ships uncalibrated with temperature fixed at 1.0, and the card calls it systematically over-confident at 0.75–0.83 mean confidence; refitting one temperature per question type and option count on held-out data moves mean ECE from 0.314 to 0.106. Its default 1,024-token limit cuts long documents off unless max_len=8192 is passed.
Questions about this entry
Short answers drawn from the linked project or page.
A typed answer with probabilities from a single forward pass: there is no text generation, so nothing to parse and nothing to hallucinate. Each option is scored at its own [MASK] marker and softmaxed over that question's options, so the answer space is defined per request without retraining.
On all 51 MASSIVE languages its macro accuracy is 0.366 against the English checkpoint's 0.227, and 45 of 51 languages clear three times random against English's 23, with macro ECE 0.387 versus 0.733.
The checkpoint ships with a 1,024-token limit; pass max_len=8192 for long documents. The card reports 16 to 18 of 20 requests correct with up to about 4,000 tokens before the question and 8 to 17 of 20 beyond that, and a 4,000-token input takes about 1.7 s on an Apple GPU.
No. The card says it ships uncalibrated with temperature [1.0, 1.0, 1.0] and is systematically over-confident, with mean confidence 0.75–0.83 against much lower accuracy; refitting one temperature per question type and option count on held-out data moves mean ECE from 0.314 to 0.106.
Router() keeps both resident, and preload(['english', 'multilingual']) loads them up front so the first request of each language is just a forward pass. Routing is decided from the input's script before the forward pass, and router.attach('multilingual', agent) registers an Agent you built yourself.
Faster at every batch size in the card's table: 32.8 ms versus 39.5 ms for one question, 72.3 ms versus 158.6 ms for ten, and 337 ms versus 771 ms for fifty on one T4, or 103–332 questions per second batched.
Run with USE_TF=0. The card explains that transformers probes for TensorFlow at import, and when TF is installed its abseil runtime can deadlock model construction.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsMore Jev resources like this
An unofficial cross-encoder fine-tune of ModernBERT-ja-310m for Jev Local: it reads a question-plus-state and one candidate per sequence and hits 92.62% on the JGLUE JNLI test, with CC BY-SA 4.0 weights.
Jev Models & Reimplementations#community#open-models#local
Community
An Open Classification Foundation smoke checkpoint: a 164M-parameter ModernBERT-base with a cross-entropy decision head, 0.751 accuracy and 0.044 ECE on 2,000 held-out typed decisions.
Jev Models & Reimplementations#community#open-models#local
Community
An open Thai + English System One decision model at 0.8B, Apache-2.0: choice with up to 255 options, fractional score, and noul answers in one forward pass over a Qwen3.5-0.8B text tower.
Jev Models & Reimplementations#community#open-models#local
Community
A 4B decision model taking text, JSON, or images up to 262,144 tokens: 52.75 on Decision Index 0.2.1, 204 of 231 on JevBench public, and 26.0 ms for a three-question request on one H200.
Jev Models & Reimplementations#community#open-models#local
Community
A decision model that scores its own clarifying questions by value of information: at AUC .799 against an oracle .797, it asks only when asking is expected to change the answer, then decides or hands off.
Jev Models & Reimplementations#community#open-models#local
Community
The Hugging Face hub for Laya's open System 1 checkpoints: the 421M English model at the repo root, a 322M multilingual build in a subfolder, and the typed-decisions fine-tune, all Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
From the Jev community
Posts from builders shipping with Jev right now.