CommunityJev Models & Reimplementations
Jev-Omni MLX 4-bit
Ruiruiz30's Apple Silicon MLX conversion of Jev-Omni: 4-bit affine language weights on a Mac mini M4 16GB, about 7 GB peak Metal memory and roughly 963 ms text decisions, Apache-2.0.
- Published by
- Community
- Author
- Ruiruiz30
- Last verified
- Oct 2026
Highlights
- Language weights are 4-bit affine with group size 64; the vision tower stays BF16 and the 256-way decision head stays FP32.
- Measured on a Mac mini M4 with 16 GB unified memory: about 7.0 GB peak Metal memory, median 963 ms text and 994 ms image decisions.
- JevBench public 195 groups / 231 decisions: 87.88% micro accuracy, 85.90% group-macro, ECE-10 0.04497 raw.
- Four upstream unified cases matched argmax after conversion, plus 6/6 red-blue-green color checks; maximum probability difference 0.244.
- Ships a temperature file (T=1.11517) that cuts public ECE-10 to 0.03069 but worsens the held-out split.
Watch out
This is an independent conversion, not an official release, and it is not a new fine-tune. The temperature file was fit on even public JevBench rows and checked on odd ones, where ECE-10 worsened from 0.06261 to 0.06774, so the card calls it a post-hoc artifact rather than a confidence guarantee. DecisionBench Medium was not fully published.
Questions about this entry
Short answers drawn from the linked project or page.
A Mac mini with an Apple M4, 10 CPU cores and 16 GB of unified memory, running macOS 26.5.1, Python 3.13.12, MLX 0.32.2 and MLX-VLM 0.7.1, single request at batch size 1 with no token generation.
Ten warm requests after one warm-up: median 963 ms and P95 998 ms for text, 994 ms and 1,021 ms for an image at the recommended 20 visual tokens, both around 7.0 GB peak Metal memory. The first request includes warm-up.
Only the language-model weights, quantized to 4-bit affine with group size 64. The vision weights stay BF16 and the decision head stays FP32, and the card says it is an independent conversion rather than a new fine-tune.
On the public 195 groups and 231 decisions it measured 87.88% micro accuracy and 85.90% group-macro, with raw ECE-10 of 0.04497 and 0.03069 after temperature scaling. It is a local public-set measurement, not a reproduction of the upstream protocol.
Partly. The global temperature T=1.11517 was fit on 116 even-indexed public rows and checked on 115 odd rows, where ECE-10 moved from 0.06261 to 0.06774. The card calls it a transparent post-hoc artifact, not a guarantee on game footage.
The release targets Apple Silicon and was measured only on the M4 Mac mini configuration above. The card notes the published H200 numbers are not transferable to this machine, and says to validate visual-token counts on your own frames.
Run python -m omni_mlx.classifier with --model ., optionally --calibration calibration.json, then --image, --state, --question and --options. Use --image-tokens 20 by default or 70 when the scene has small, dense details.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsMore Jev resources like this
An Apple Silicon MLX bf16 build of the v1 Jev-Style decision model: 77 ms decisions on an M1 Max with calibrated choice, bool and score probabilities, Apache-2.0 weights.
Jev Models & Reimplementations#community#open-models#local
Community
Reza2kn's llama.cpp Q4_K_M build of Jev-Omni: a 6.87 GiB multimodal decision backbone plus projector and FP32 head, measured at 8.98 GiB VRAM on an RTX 5080 Laptop GPU.
Jev Models & Reimplementations#community#open-models#local
Community
GGUF builds of the JevK5 family for llama.cpp: Q8_0 is effectively lossless (229 of 231 answers match the 4B v0.3 bf16), and every published file clears the card's 95% agreement gate.
Jev Models & Reimplementations#community#open-models#local
Community
A CPU-first BitNet student: Microsoft's BitNet b1.58 2B quantized to an I2_S GGUF with a float32 pointer head, distilled from Kev 9B logits and run through the bit-jev Python package with no separate open-weight licence.
Jev Models & Reimplementations#community#open-models#local
Community
1GitHub stars
Serves Shanghai AI Lab's Intern-Decision-0.8B vision decision model on Apple Silicon behind a /v1/systemone-shaped endpoint: screenshots plus typed questions in, calibrated Choice, Score and Noul probabilities out, for thresholding and escalation.
Jev Models & Reimplementations#community#apple-silicon#local
AutoTrust's combined System 1 and System 2 bundle on Gemma-4-26B-A4B-it: Decision Index 0.2.1 of 58.05, a 24-slot head with noul, choice and score, and the base chat model left bit-identical.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
From the Jev community
Posts from builders shipping with Jev right now.