Jev directory
CommunityJev Models & Reimplementations
JEV-35B AWQ INT4 W4A8
A 4-bit AWQ plus GPTQ build of AutoTrust's JEV-35B from AI Outfitters: INT4 weights with FP8 and INT8 activations, calibrated on all 256 experts, and served by vLLM on a single 24 GB GPU.
- Reading time
- 2 min
- Published by
- Community
- Author
- AI Outfitters
- Last verified
- Oct 2026
Used inJev for Software & DevTools
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- Quantizes AutoTrust's Qwen3.5-35B-A3B decision model to INT4 weights, with routed experts on FP8 and the rest of the text stack on INT8 activations.
- The 264-slot decision head stays bf16, and the vision tower, MTP layer, routers and shared_expert_gate are untouched.
- Calibrated on all 256 experts using the 13,766-row jev-distill-corpus-v3 calibration split in the exact System 1 prompt format.
- Self-reported on 500 held-out rows: KL mean 0.03, ECE 0.1548, noul KL 0.005 and choice top-1 0.8596.
- Serves on one 24 GB GPU via vLLM with Marlin and Humming W4A8 kernels: 57.3 ms median at concurrency 1 and 17.44 requests/s.
Watch out
Card reviewed 2026-10-17; all validation numbers are self-reported by the publisher running the checkpoint's own serve_decide.py against the provider's held-out split, there is no comparison against the unquantized JEV-35B for calibration drift, and no third-party reproduction is linked.
Questions about this entry
Short answers drawn from the linked project or page.
A 4-bit quantization of autotrust/JEV-35B built by AI Outfitters with llm-compressor: INT4 weights with 8-bit activations, keeping the System 1 decision behaviour and serving from a single 24 GB GPU.
Routed experts are int4 with fp8 e4m3 per-token activations on the Humming W4A8 MoE kernel; attention and shared-expert linears are int4 with int8 activations on Marlin W4A8; lm_head and vision stay bf16.
On the provider's own calibration split of SargeDev/jev-distill-corpus-v3, 13,766 rows sampled kind-balanced at 45% choice, 30% noul and 25% score into the checkpoint's exact System 1 prompt format, 1024 samples of 2048 tokens.
The card says llm-compressor's AWQ grid search needs a forward pass per expert per grid point, roughly 10k passes per layer and about 20 hours for the model, and that partial smoothing of a shared norm is measurably harmful.
On 500 held-out rows: KL mean 0.03 with median 0.0081, ECE 0.1548, noul KL 0.005 with top-1 0.6732, choice KL 0.0732 with top-1 0.8596, and score KL 0.0415 with top-1 0.8682.
At concurrency 1 the card reports 17.44 requests/s with a 57.3 ms median and 57.9 ms p95. A 26,200-character decision prefills 4,644 tokens in 0.29 s cold and 0.06 s cached.
AI Outfitters built it with llm-compressor 0.14, compressed-tensors 0.19, transformers 5.17 and torch 2.14 on an RTX 4090. The licence is Apache-2.0, matching autotrust/JEV-35B.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsMore Jev resources like this
prithivMLmods' GGUF builds of autotrust/JEV-35B, the Qwen3.5-35B-A3B MoE decision model that returns calibrated choice, noul and score answers over text and images; files run from BF16 to Q3_K_L for llama.cpp.
Jev Models & Reimplementations#community#open-models#local
Community
A community Q8_0 GGUF of Mapika/decider-0.8b, the smallest System One typed-decision model: one forward pass returns calibrated Choice, Score and noul answers, reporting 0.405 on JevBench hard-public at 920 MB resident.
Jev Models & Reimplementations#community#open-models#local
Community
A 4-bit W4A8 AWQ plus GPTQ build of AutoTrust's JEV-27B-VL from AI Outfitters that keeps the rank-32 LoRA decision head and a bf16 vision tower, served by vLLM on a single 24 GB GPU.
Jev Models & Reimplementations#community#open-models#local
Community
Mapika's NVFP4 fine-tune of Gemma-4-31B-it answers typed Choice, Noul and Score questions with calibrated probabilities and no generated text, self-reporting a 64.27 chance-corrected public Decision Index and 39 ms median latency on a B300.
Jev Models & Reimplementations#community#open-models#local
Community
A LoRA plus 24-slot decision head over Qwen3.8-27B from Ming Xu's train-jev project: it serves TypeSafe's System One API with 0.851 Kev transfer-v4 accuracy and ECE 0.053, and needs an 80 GB GPU.
Jev Models & Reimplementations#community#open-models#local
Community
A 24 GB-GPU alternative from the train-jev project: Qwen3.5-4B plus a rank-16 LoRA and 24-slot decision head scoring 0.647 PK accuracy, 40.4 Decision Index and 43 ms p50 latency, ahead of Kev-4B at the same size.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.