Jev directory
CommunityJev Models & Reimplementations
GLiNER2.5-Decide Core ML
Fluid Inference's fixed-shape Core ML conversion of the classification path of Fastino's GLiNER2.5-Decide for Apple silicon: the DeBERTa-v3-large encoder and shared label classifier score up to four decision heads per call, in fp16, 8-bit or 6-bit packages.
- Reading time
- 3 min
- Published by
- Community
- Author
- Fluid Inference
- Last verified
- Oct 2026
Used inJev for Software & DevTools
- Platforms
- macOS
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- Core ML export of the classification path of fastino/GLiNER2.5-Decide at revision 65624f1a; span and count heads are not exported.
- Each package carries the DeBERTa-v3-large encoder and shared label classifier, 436,022,273 exported parameters.
- On the 17-domain Fast Decisions development split it matches native PyTorch: 62.93% average fp16, 63.01% w8 and 63.03% lut6.
- Package sizes: fp16 936 MB, w8 477 MB, lut6 361 MB; weight-only compression shrinks downloads, not latency.
- L512 p50 latency 44.0 ms for a three-head request on an M5 Pro with compute units ALL; CPU_AND_NE measured 689 ms.
Quickstart
bash
uv sync
uv run python runtime.py --model-dir . --precision lut6 --length 256 \
--text "My transfer is still pending and I used the wrong sort code. Can you stop it?" \
--tasks '{"intent": ["transfer_pending", "transfer_cancel", "beneficiary_add", "card_lost"], "urgency": ["low", "normal", "high"]}'Watch out
Card reviewed 2026-10-17; the accuracy table reproduces the native checkpoint on the public development split, which the card explicitly asks not to report as the 60.2% benchmark; the scores and latencies are self-reported local measurements for one request shape on an M5 Pro. Bucket overflows raise ValueError instead of truncating.
Questions about this entry
Short answers drawn from the linked project or page.
Fluid Inference's Core ML export of the classification path of fastino/GLiNER2.5-Decide, pinned to revision 65624f1a. Fastino made the Apache-2.0 source model; each package contains the DeBERTa-v3-large encoder and the shared label classifier at 436,022,273 exported parameters.
It scores label sets rather than generating text, the same one-pass idea behind Jev, but this release is classification only: the span and count heads are not exported and the card documents a local Python CoreMLDecide runtime, not a System One HTTP endpoint.
Up to four decision heads over the same text, for example intent, urgency and route. Single-label heads use softmax and multi-label heads use sigmoid with cls_threshold, exactly as classify_text does upstream.
That the Core ML packages reproduce the native checkpoint: 62.93% average and 61.38% pooled for fp16 and native, 63.01% and 61.52% for w8, 63.03% and 61.52% for lut6. The card asks these not be reported as the 60.2% held-out benchmark.
fp16 at 936 MB is the exact baseline; w8 at 477 MB changed 7 of 2,900 heads and lut6 at 361 MB changed 42, both at roughly the same measured accuracy and latency. The rejected per-tensor LUT4 lost 4 points on paper_field.
Run uv sync, then uv run python runtime.py --model-dir . --precision lut6 --length 256 with a --text and --tasks argument, or import CoreMLDecide from runtime in Python. Download with local_dir set, because Core ML compilation rejects the Hub cache's symlinked weight files.
Three fixed buckets: L128 (4 by 8 labels), L256 (4 by 32, 95% of Fast Decisions heads fit) and L512 (4 by 32, 100%). Every row must fit the bucket; requests that exceed it raise ValueError, and grouped-channel palettization needs an iOS 18 target and was not tried.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsMore Jev resources like this
TurinTech's independent GGUF conversion of the 27B Kev decision model for llama.cpp: the default IQ4_XS file is 13.91 GB and keeps 97.8% of the BF16 original's answers over 2,000 test decisions, with zero changed confident answers.
Jev Models & Reimplementations#community#open-models#local
Community
An 8-bit MLX conversion by SirSahOl of chaoliangUNSW/Jev-Style-0.8B-Decision-v3 built for Apple Silicon native GPU inference: about 990 MB of active VRAM, a 262,144-token context and an average of 8.25 bits per weight.
Jev Models & Reimplementations#community#open-models#local
Community
A Q8_0 GGUF of knowlet's Gemma-4-12B-Unified-System-One with a native System One decision runtime and an independently fitted GGUF temperature, shipped with an F16 mmproj for the model's native image and audio projections.
Jev Models & Reimplementations#community#open-models#quantized
Community
TurinTech's independent GGUF conversion of Jared Palmer's Kev-4B decision model; the default Q4_K_M file is 2.54 GB and returns the same answer as the BF16 original on 93.0% of 2,000 test decisions with no confident-answer changes.
Jev Models & Reimplementations#community#open-models#quantized
Community
TurinTech's independent GGUF conversion of Jared Palmer's Kev-9B decision model; the default IQ3_M file is 4.78 GB and matches the BF16 original on 92.2% of 2,000 test decisions with a minus 1.2-point accuracy change.
Jev Models & Reimplementations#community#open-models#quantized
Community
The 9B entry in vLLM Semantic Router's Decision 2.0 family: a 7.94B decision model with a 16,384-token context that answers choice, yes/no and score questions in one pass, reporting 68.1 JevArena and 18.4 ms median single-question latency.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.