Jev directory
CommunityJev Models & Reimplementations
PPLX Decider v1 27B EXL3
An EXL3 4.00 bpw conversion of Perplexity AI's pplx-decider-v1-27b by ramgpt: a 15.35 GB artifact that keeps the custom readout.safetensors decision head, with card-reported RTX 4090 medians from 135 ms at 128 tokens to 2.777 s at 8,192.
- Reading time
- 3 min
- Published by
- Community
- Author
- ramgpt
- Last verified
- Oct 2026
Used inJev for Software & DevTools
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- EXL3 4.00 bpw conversion of perplexity-ai/pplx-decider-v1-27b at source revision 5117a6c, requiring ExLlamaV3 1.5.2+cu128.torch2.10.0.
- Safetensors size 15.35 GB (14.29 GiB) against about 48.6 GiB of BF16 weights in the source checkpoint.
- Ships pplx_decider_exl3.py, which registers the Qwen3_5Model architecture and performs the custom readout.safetensors decision head.
- RTX 4090 medians: 135 ms at 128 input tokens (7.34 decisions/s), 725 ms at 2,048 and 2.777 s at 8,192 (0.36 decisions/s).
- Hand-authored sanity suite reports 43/43, including 9/9 routing; reversed option order kept the same choice 30/30 times.
Watch out
Card reviewed 2026-10-17; the RTX 4090 latencies and the 43/43 sanity results are self-reported local measurements, not official benchmarks, and the card states no BF16-vs-EXL3 accuracy or parity study has been run. The artifact is explicitly not recommended for standard TabbyAPI usage: the smoke test failed startup with the error AssertionError: Unknown architecture Qwen3_5Model, and the decision readout cannot be served through a normal /v1/chat/completions path.
Questions about this entry
Short answers drawn from the linked project or page.
ramgpt's EXL3 4.00 bpw conversion of perplexity-ai/pplx-decider-v1-27b, stored as 15.35 GB of safetensors. It keeps the source model's custom decision readout instead of a normal LM head, and the repository includes pplx_decider_exl3.py to register and run it.
It is a local serving artifact for a competing decision model: option probabilities come from a hidden state passed through the model's custom decision head, which is the System One shape Jev popularized. It covers decision inference only and generates no explanatory text.
On the card's RTX 4090 24 GB test, peak PyTorch allocated VRAM ranged from 12.67 GiB at 128 tokens to 13.89 GiB at 8,192 tokens, with about 15.0 GiB total GPU memory use reported by nvidia-smi at the end of the 8K run.
Median latency per decision on an RTX 4090: 135 ms at 128 input tokens, 278 ms at 512, 430 ms at 1,024, 725 ms at 2,048, 1.400 s at 4,096 and 2.777 s at 8,192, from 10 sequential requests per row after warm-up.
A local smoke test with TabbyAPI and ExLlamaV3 1.5.2 failed at startup with AssertionError: Unknown architecture Qwen3_5Model, and the card notes that even registering the architecture is not enough because TabbyAPI's standard path expects an LM head, not the custom decision readout.
A small hand-authored suite: routing 9/9, sentiment 6/6, entailment 9/9, arithmetic 6/6, yes/no urgency 8/8, severity 5/5 (43/43 total), 30/30 stable decisions with reversed option order and 5/5 identical repeats with a maximum probability delta of 0.0. The card calls these basic synthetic checks, not an accuracy benchmark.
Apache-2.0, inherited from the source model. It requires ExLlamaV3 1.5.2+cu128.torch2.10.0 with Torch 2.10.0+cu128, and the card notes model load was 3.09 s with a warm OS file cache, with 12.57 GiB resident immediately after load.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsMore Jev resources like this
prithivMLmods' GGUF builds of autotrust/JEV-35B, the Qwen3.5-35B-A3B MoE decision model that returns calibrated choice, noul and score answers over text and images; files run from BF16 to Q3_K_L for llama.cpp.
Jev Models & Reimplementations#community#open-models#local
Community
A calibrated image-capable 27B decision model that fits an 8 GB GPU: the JEV LoRA-and-head recipe trained on Prism ML's ternary Bonsai 2, returning typed probabilities with no generated text.
Jev Models & Reimplementations#community#open-models#local
Community
A community Q8_0 GGUF of Mapika/decider-0.8b, the smallest System One typed-decision model: one forward pass returns calibrated Choice, Score and noul answers, reporting 0.405 on JevBench hard-public at 920 MB resident.
Jev Models & Reimplementations#community#open-models#local
Community
A 4-bit AWQ plus GPTQ build of AutoTrust's JEV-35B from AI Outfitters: INT4 weights with FP8 and INT8 activations, calibrated on all 256 experts, and served by vLLM on a single 24 GB GPU.
Jev Models & Reimplementations#community#open-models#local
Community
A 4-bit W4A8 AWQ plus GPTQ build of AutoTrust's JEV-27B-VL from AI Outfitters that keeps the rank-32 LoRA decision head and a bf16 vision tower, served by vLLM on a single 24 GB GPU.
Jev Models & Reimplementations#community#open-models#local
Community
mradermacher's static GGUF quantizations of IqSantiagow's 0.8B edceleste decider for llama.cpp, from Q2_K (0.5 GB) to f16 (1.6 GB), with Q4_K_M and Q4_K_S marked fast and recommended; no imatrix quants are planned by this quanter.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.