Which files do I need for multimodal decisions?

All three: Jev-Omni-Unified-Q4_K_M.gguf for the backbone, mmproj-jev-omni.gguf for the projector and decision-head-f32.npz for the head, plus jev_omni_gguf_decide.py. For text-only decisions the projector is optional.

How is the quantization built?

The Gemma 4 12B backbone was converted with llama.cpp revision 6e60f356 and quantized with llama-quantize Q4_K_M 8 without an importance matrix, from source revision c050d513. The quantizer reported 4.95 bits per weight overall, since Q4_K_M is mixed precision.

How much memory should I expect on GPU?

Observed readings: 8,980 MiB VRAM with the model loaded text-only, 9,148 MiB with the projector loaded and 9,256 MiB after an image decision, on an RTX 5080 Laptop GPU with full offload at 2,048 context. These are not maximums across all inputs.

Can the Q4 build run on the CPU?

Yes, with a caveat. The card measured 12.29 GiB of process PSS after a text decision and 13.49 GiB after a 16-frame video decision on CPU, with a median 11.30 seconds per six-question request on a Ryzen AI 9 HX 370 using eight llama.cpp threads.

Does the Q4 match the original probabilities?

No, and the card warns against assuming it. On four published text verification examples the quantized build matched the winning option three times, with a maximum absolute probability difference of 0.210, and image, audio and video routes passed only small smoke checks.

How does it do on the Persian benchmark?

It returned 624 of 624 valid answers: 237/240 on choice (98.75%), 150/160 on noul (93.75%) and 76/80 on score within ±0.5 levels (95%). It leads the Bev baseline on choice and score and trails slightly on noul.

Can I open it with plain llama-cli generation?

No. This is a classifier returning probabilities over supplied options; the card says plain llama-cli text generation does not apply the trained decision head. Keep the embedding endpoint bound to loopback because it exposes internal model states.

CommunityJev Models & Reimplementations

Jev-Omni Q4_K_M GGUF

Reza2kn's llama.cpp Q4_K_M build of Jev-Omni: a 6.87 GiB multimodal decision backbone plus projector and FP32 head, measured at 8.98 GiB VRAM on an RTX 5080 Laptop GPU.

See highlights

Open huggingface.co

Published by: Community
Author: Reza2kn
Last verified: Oct 2026

Tagscommunity open-models local quantization gguf

Works forJev for Classification Jev for Scoring Jev for Decision gating

Used inJev for Software & DevTools Jev for Media & Entertainment

Languages: Python
Capabilities: Open sourceSelf-hostedRuns locally

Highlights

The quantized backbone is 6.87 GiB on disk, the multimodal projector 116.38 MiB and the required FP32 decision head 3.78 MiB.
Measured 8,980 MiB of VRAM for text-only on an RTX 5080 Laptop GPU, 9,148 MiB with the projector and 9,256 MiB after an image decision.
On the Jev Persian Benchmark it answered 624 of 624 questions: choice 237/240, noul 150/160 and score 76/80 within ±0.5.
llama-quantize reported 4.95 bits per weight overall; the projector and decision head keep their original precision.
Median per six-question request: 1.18 s on the RTX 5080 Laptop GPU and 11.30 s on a Ryzen AI 9 HX 370 CPU-only.

Watch out

Do not assume source-equivalent probabilities: on four published text verification examples the Q4 matched the winning option 3 times with a maximum absolute probability difference of 0.210. Video, image and audio routes passed only smoke checks, the adapter is unvalidated on macOS, Windows, Metal and Vulkan, and the Persian comparison pits two systems with different prompts against each other.

Questions about this entry

Short answers drawn from the linked project or page.

Reactions & coverage

Posts, threads, and videos about this entry from around the web.

LinkedIn: What multimodal decision models actually change, and what they do not.

LinkedInJevDirectory.org

Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.

jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker News

Hacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.

1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker News

Hacker News: A one-token logprob wrapper that also classifies webcam frames.

allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker News

Related terms

Glossary definitions related to this entry.

Open weights

More Jev resources like this

JevK5 GGUF

GGUF builds of the JevK5 family for llama.cpp: Q8_0 is effectively lossless (229 of 231 answers match the 4B v0.3 bf16), and every published file clears the card's 95% agreement gate.