CommunityJev Models & Reimplementations
Laya Typed-Decisions GGUF
ggmlc-compiled GGUFs of the Laya typed-decisions specialist on ModernBERT-large: F16, Q8_0, and UD_Q4_K_M files for invoice, security, customer-service, and harness presets, Apache-2.0.
- Published by
- Community
- Author
- mys
- Last verified
- Oct 2026
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- Sizes are ~811 MB F16, ~434 MB Q8_0, and ~404 MB UD_Q4_K_M, whose 1D norms and biases stay F32.
- Not llama.cpp files: ggmlc lowers PyTorch, JAX, Flax, and Keras models to GGML, and loading these in llama.cpp will fail.
- Typed questions are scored in one encoder pass with no autoregressive token generation, under max_len 1024.
- The laya binary lists presets invoice, security, customer_service, and harness, with --device defaulting to auto.
Watch out
Apache-2.0, same as the upstream convaiinnovations/laya-typed-decisions weights (ModernBERT-large, 421M, context 1024). --models-dir routing will not pick this family automatically unless --family typed-decisions is passed or the question ids match a specialist workflow, and llama.cpp cannot load the files.
Questions about this entry
Short answers drawn from the linked project or page.
No. The card says they are not llama.cpp or llama-cli GGUFs; they are produced by ggmlc, a neural network compiler that lowers PyTorch, JAX, Flax, and Keras models to GGML execution, and loading them in llama.cpp will fail.
Three: laya_typed_decisions_f16.gguf at about 811 MB in full precision, laya_typed_decisions_q8_0.gguf at about 434 MB, smaller and still accurate, and laya_typed_decisions_ud_q4_k_m.gguf at about 404 MB, the smallest, with 1D norms and biases kept in F32.
It is fine-tuned for invoice match, SOC alerts, customer-service next action, and agent-trace harnesses, exposed as the presets invoice, security, customer_service, and harness.
Pass --family typed-decisions, for example laya decide --models-dir . --family typed-decisions --preset invoice. The card says --models-dir routing does not pick this family automatically unless that flag is set or the question ids match a specialist workflow.
Download a laya binary from the ggmlc releases, then run laya decide with a preset, for example laya decide laya_typed_decisions_f16.gguf --preset invoice --device auto --cuda-graph. --device defaults to auto, picking CUDA or Metal if present and the CPU otherwise.
laya serve <file> --port 8080 starts Decision Studio at GET / and a POST /api/decide endpoint; a separate daemon mode speaks newline JSON-RPC over stdin and stdout.
It is compiled from convaiinnovations/laya-typed-decisions, a ModernBERT-large model at 421M parameters with a 1,024-token context, and the weights are Apache-2.0, the same licence as upstream. The ggmlc compiler is MIT-licensed.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: A one-token logprob wrapper that also classifies webcam frames.
allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker NewsMore Jev resources like this
The v3 Jev-Style decision model at 2B in GGUF: F16 3.78 GB and Q8_0 2.01 GB match PyTorch FP32 on 100% and 99.7% of parity rows, 73.6% on JevBench v1.4.1 public items.
Jev Models & Reimplementations#community#open-models#local
Community
A decision model that scores its own clarifying questions by value of information: at AUC .799 against an oracle .797, it asks only when asking is expected to change the answer, then decides or hands off.
Jev Models & Reimplementations#community#open-models#local
Community
Mithril's open, Jev-shaped typed-decision model on DeBERTa-v3-large: choice up to 255 options, score and noul from one forward pass, 0.854 in-domain with ECE 0.022, trained on public gold labels only.
Jev Models & Reimplementations#community#open-models#local
Community
TokenRhythm's 4B prefill-only decision model built on NeoHorse-1-4B: given a state and application-defined questions it returns Choice, Noul, and Score probabilities without generating text, reporting a 77.70 six-group aggregate under Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
The GGUF edition of TokenRhythm's NeoHorse-Jev-4B: BF16 through Q2_K quantizations that each bundle the language backbone, visual weights, decision head, and tokenizer, plus a shared runtime for text and single-image requests.
Jev Models & Reimplementations#community#open-models#local
Community
A Qwen2.5-1.5B-Instruct fine-tune that emits typed dependency verdicts, safe, caution, or block with a recommendation, for Dependabot and Renovate pipelines, reporting 95.7% F1 on the card's 4,800-update OSV-Bench.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
Training recipes, looped architectures, and language ports: what the open side of the typed-decision class shipped this month, and how to pick by runtime and calibration instead of the leaderboard.
Jev Models & Reimplementations#open-models#training#local
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.
Jev Practices & Patterns#open-models#evaluation#benchmarks
Read article
From the Jev community
Posts from builders shipping with Jev right now.