User repository on GitHub
deck31b: frozen Gemma 4 31B FP8 decision server for the TypeSafe /v1/systemone format
- Forks
- 0
- Last pushed
- Updated Oct 2026
Repo stats from the GitHub API, cached Oct 2026.
Jev directory
CommunityJev Repos & SDKs
Deck31b
A frozen decision server that serves Gemma 4 31B — quantized to FP8 at startup — in TypeSafe's /v1/systemone format, reading the next-token distribution instead of generating tokens. It needs one FP8 NVIDIA GPU with 80 GB and has no authentication.
- Stars
- 9
- Language
- Python
- License
- APACHE-2.0
- Reading time
- 3 min
- Published by
- Community
- Author
- krishna-gogineni-765
- Last verified
- Oct 2026
Used inJev for Software & DevTools
Highlights
- Frozen decision model: quantizes google/gemma-4-31B-it linear layers to FP8 e4m3 with TorchAO at startup, then reads the next-token distribution.
- Prompt and letter readout follow the public Cygnet recipe: per-letter probability mass renormalized over options, then p^(1/T) with T=3.4.
- POST /v1/systemone accepts noul, choice, and score questions; up to 26 options are scored in one pass, and inputs over 16,384 tokens are rejected.
- Needs one NVIDIA GPU with FP8 support and at least 80 GB; the README's H100 80 GB run held about 44 GB steady-state after quantization.
- README's local measurement: 215/231 JevBench public items correct, ECE10 0.044 at T=3.4, p50 1.05 s — labeled not an official JevBench result.
Quickstart
bash
git clone https://github.com/krishna-gogineni-765/deck31b
cd deck31b
python -m venv .venv && source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install --index-url https://download.pytorch.org/whl/cu128 torch==2.11.0
python -m pip install -e .
deck31b-serve --port 8090
curl --fail http://127.0.0.1:8090/healthWatch out
README reviewed 2026-10-17; the digest truncates the install block after the first pip line, so the full setup shown here comes from the same README's install section. The server has no authentication and listens on 127.0.0.1 unless --host is set, and all JevBench figures are the author's local measurement, not an official run.
Questions about this entry
Short answers drawn from the linked project or page.
A frozen decision model from krishna-gogineni-765 that serves the TypeSafe /v1/systemone format: you send a state and typed questions, and it returns probabilities over the options it was given instead of generating text.
At startup it quantizes linear layers to FP8 e4m3 weights and activations with TorchAO, then reads the next-token distribution. Per-option letter probabilities are summed over vocabulary tokens and renormalized following the public Cygnet recipe at T=3.4.
POST /v1/systemone accepts noul, choice, and score questions, and up to 26 options are scored in one pass. Inputs over 16,384 tokens are rejected, and the server requires no authentication.
One NVIDIA GPU with FP8 support and at least 80 GB of memory — the README's reference run used an H100 80 GB, with about 44 GB steady-state memory after quantization. The first start downloads the base checkpoint from Hugging Face.
No. The README labels them a local measurement, not an official JevBench result: 215/231 public items correct with ECE10 0.044 at T=3.4, p50 1.05 s. The sealed run should be treated as the official number.
The prompt and letter readout follow blockbrain-ai/cygnet-recipe at a pinned commit: one letter per option, probability mass summed over every vocabulary token that decodes to that letter, renormalized over the listed options, then p^(1/T) with the recipe temperature T=3.4.
The server code is Apache-2.0. The Gemma 4 31B weights are not redistributed by the repository and remain under the Gemma Terms of Use; the server downloads the base checkpoint on first start.
More Jev resources like this
HTTP server for the Mica v0.5 4B Preview decision model: GGUF weights run through llama.cpp and answer noul, choice and score questions with calibrated probabilities over the TypeSafe /v1/systemone format.
Jev Repos & SDKs#community#github#python
Community
akivet/MicaPython
akivet/MicaPythonServing code for the Jeff 1.0 family of Jev-style decision models: a state, a typed question and its options go in, and one forward pass returns a probability for each option. Four open models from 4B to 31B parameters are served the same way.
Jev Repos & SDKs#community#github#python
Community
j-geuter/jeff-servePython
j-geuter/jeff-servePythonA System One layout optimizer for decision-model workloads: it reorders rows and fields so a prefix-caching backend reuses input computation, then returns per-record decisions in original order. Its bundled backend serves AutoTrust JEV-9B through vLLM.
Jev Repos & SDKs#community#github#python
Community
0814wdwd/solo_jevPython
0814wdwd/solo_jevPythonAn open-source, locally executed decision engine in the spirit of Jev (TypeSafe AI): state becomes features, and Noul, Choice and Score heads return typed, calibrated probabilities without generating text. It ships a cheap-to-strong cascade and a full train-calibrate-evaluate-deploy cycle.
Jev Repos & SDKs#community#github#python
Community
slavabars/decision-cascadePython
slavabars/decision-cascadePythonAn open typed-decision layer with choice, score and noul questions, calibrated probabilities and one batched call, built as a numpy-only base with eight swappable engines. The README positions it as rebuilding Jev's closed API shape on hardware you control.
Jev Repos & SDKs#community#github#python
Community
Franzferdinan51/SystemOnePython
Franzferdinan51/SystemOnePythonAn open-source, non-autoregressive System One decision engine inspired by TypeSafe AI: it evaluates state context against typed questions in a single forward pass, returning probability distributions and calibrated confidence without generating text.
Jev Repos & SDKs#community#github#python
Community
Atulmishra22/jev-litePython
Atulmishra22/jev-litePythonFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.