Jev directory
CommunityJev Models & Reimplementations
Torchcast Decision 12B
Torchcast AI's 12B decision checkpoint: a LoRA fine-tune of google/gemma-4-12B-it that answers noul, choice and score questions in one forward pass, with a card-reported, author-run 203/231 (87.88%) on the JevBench v1.4 public items.
- Reading time
- 3 min
- Published by
- Community
- Author
- Torchcast AI
- Last verified
- Oct 2026
Highlights
- LoRA fine-tune of google/gemma-4-12B-it trained on a 50/50 mix of gold labels and a larger instruction model's option distributions, not Jev outputs.
- Card's author-run public-231 JevBench v1.4 result: 203/231 (87.88%) — easy 48/48, original 70/72, hard 85/111, all answers valid and no failed requests.
- One forward pass reads option letters at the answer position; per-type temperatures in readout_config.json (choice and score 1.0, noul 0.2).
- Served context limit 16,384 tokens with HTTP 422 beyond it; evaluation interface is POST /v1/systemone with one question named decision.
- Release check ran BF16 on one NVIDIA H100 80 GB with vLLM 0.30.0: serial latency 30 ms p50 and 43 ms p95 on easy+original items.
Quickstart
bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install vllm==0.30.0Watch out
Card reviewed 2026-10-17; the 203/231 JevBench result is author-run, not an official score or rank, and the card discloses that public results influenced checkpoint and serving-config selection, that an 8-gram screen removed overlapping training rows and that the measurements are not an untouched holdout. The weights are CC BY-NC 4.0, so commercial use is excluded, and Google's Gemma Prohibited Use Policy also applies.
Questions about this entry
Short answers drawn from the linked project or page.
A text-based decision checkpoint by Torchcast AI built on google/gemma-4-12B-it. It supports yes/no probabilities (noul), choice distributions and ordinal score distributions, and the card says to use the supplied serving package because generic Hub text-generation examples do not reproduce the evaluated interface.
It is an independent competing decision model: the card states the project is independent of TypeSafe AI and the JevBench maintainers, training deliberately excluded Jev outputs, and the 203/231 public-231 result is an author-run measurement rather than an official JevBench score or rank.
noul yes/no probabilities, choice distributions and score distributions. Each decision is one forward pass that reads the option letters at the answer position, with per-type temperatures in readout_config.json fitted on public non-JevBench validation or test splits.
On the 231 public JevBench v1.4 items: 203/231 correct (87.88%), split easy 48/48, original 70/72 and hard 85/111. Serial client latency on an H100 was 30 ms p50 and 43 ms p95, with a mean of 704 input tokens and one output token per decision.
The served context limit is 16,384 tokens including the rendered request, and inputs beyond it receive HTTP 422. Serving runs BF16 with vLLM 0.30.0; the release check used one NVIDIA H100 80 GB and an L40S 48 GB appeared during development.
Public JevBench results influenced checkpoint and serving-configuration selection. An 8-gram screen of an earlier training set found procedurally generated rows overlapping 14 public items; those rows were removed and the released training data shares no 8-gram with the public items, but the card does not claim a zero-contamination holdout.
CC BY-NC 4.0, for non-commercial use, with the caveat that the label does not establish clearance of every training-source right. The serving code is MIT, Google's Gemma Prohibited Use Policy applies, and upstream notices for Gemma and the serving foundation must be preserved.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsMore Jev resources like this
A 27B decision model from torchcast-ai that answers choice, noul and score questions together in one forward pass over the TypeSafe-compatible /v1/systemone format; the card reports Decision Index 0.2.1 of 65.00, +7.09 over Jev 1.13, at 33 ms per yes/no question.
Jev Models & Reimplementations#community#open-models#local
Community
The Ollama build of iapp's OpenThai-SystemOne: a 0.8B Thai and English decision model whose v0.3 weights are fine-tuned for Ollama's /v1/systemone letter readout, reporting a 74.3 macro across 13 public subsets for Q8_0.
Jev Models & Reimplementations#community#open-models#local
Community
A 27B decision model from StartLux Labs that takes text, JSON or images up to 262,144 tokens and returns a probability per option through the TypeSafe /v1/systemone format, reporting Decision Index 0.2.1 63.88 and 208 of 231 on JevBench public.
Jev Models & Reimplementations#community#open-models#local
Community
The Q8_0 8-bit GGUF build of StartLux-Decision-v1-27B, 28.60 GB for llama.cpp, served through a decision wrapper that the card measures at 100% same decisions as the original and 209 of 231 on JevBench public.
Jev Models & Reimplementations#community#open-models#quantized
Community
The Q8_0 8-bit GGUF build of StartLux-Decision-v1-9B, 9.53 GB for llama.cpp, with the card reporting 100.0% same decisions as the original weights and 200 of 231 correct (86.6%) on JevBench public.
Jev Models & Reimplementations#community#open-models#quantized
Community
A local daemon that pulls open decision models and serves them on TypeSafe's /v1/systemone, so a Jev client switches hosts with TYPESAFE_BASE_URL.
Jev Models & Reimplementations#community#rust#open-models
Community
ollaya-dev/ollayaRust
ollaya-dev/ollayaRustFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.