User repository on GitHub
Jev-like model inference engine + Jev-compatible API
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
Jev directory
CommunityJev Repos & SDKs
Jevinf
A Jev-kind model inference engine and compatible API server in Python: each candidate path runs as segmented forwards with prefix reuse, and the Jev wire contract is served on top with NanoJev as the wired backend. The README reports 2.57x single-request throughput on MPS at 100% argmax agreement.
- Stars
- 1
- Language
- Python
- License
- MIT
- Reading time
- 3 min
- Published by
- Community
- Author
- zerodegress
- Last verified
- Oct 2026
Used inJev for Software & DevTools
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- Serves /v1/systemone plus /api/evaluate from a local checkpoint; point any Jev client at it with TYPESAFE_BASE_URL, and the README says the official SDK works.
- Measured on MPS: 2.57x on one API-sized request (25.80 s to 10.06 s) and 2.27x end to end on the dev split, at 100% argmax agreement; CUDA shows 2.15x/2.07x.
- The speedup comes from segmented forwards with prefix reuse — from the arrangement, not the device, per the README.
- Backends: torch-mps (default) and torch-cuda are wired up; torch-cpu and torch-rocm are declared but refuse to run.
- Weights and the evaluation split are not in the repository: AGENTS.md pins the download commands and the sha256 of every file they produce.
Quickstart
bash
uv sync
uv run jevinf serve -m models/NanoJev --port 8226
TYPESAFE_BASE_URL=http://127.0.0.1:8226 TYPESAFE_API_KEY=sk-any uv run python your_jev_client.pyWatch out
Grounded in the truncated README and GitHub metadata reviewed on 2026-10-15; the excerpt ends mid-row in the architecture table, the checkpoint and dev split must be obtained separately per AGENTS.md, torch-cpu and torch-rocm refuse to run, decider-2b's faster kernels are Linux-only, and the prototype is tuned for Apple silicon.
Questions about this entry
Short answers drawn from the linked project or page.
A Python inference engine for decision models of the Jev kind: each candidate path runs as segmented forwards with prefix reuse, and the Jev wire contract is served on top. NanoJev is the backend wired up today, with decider-2b as an alternative architecture.
The README reports 2.57x on a single API-sized request on MPS (25.80 s to 10.06 s) and 2.27x end to end on the dev split, both at 100% argmax agreement. The same two shapes measure 2.15x and 2.07x on CUDA, so the speedup comes from the arrangement, not the device.
Yes: jevinf serve exposes /v1/systemone and /api/evaluate, and the README shows pointing any client — the official SDK included — at it with TYPESAFE_BASE_URL=http://127.0.0.1:8226 and any key value, so client code does not change.
The development setup requires Python 3.14 or newer and uv: run uv sync to build the venv from uv.lock, then uv run jevinf --help. Weights and the evaluation split are not in the repository; AGENTS.md pins the download commands and file hashes.
torch-mps is the default on Apple silicon and torch-cuda is wired up for NVIDIA GPUs; torch-cpu and torch-rocm are declared but refuse to run. The prototype is tuned for Apple silicon with unified memory, and decider-2b's faster kernels are Linux-only because they need triton.
jevinf serve serves the two service entry points from a checkpoint directory, on a chosen port; passing --api-key sk-local enforces HTTPBearer on /v1/* endpoints. The README also shows selfcheck and bench commands with --states to validate a checkpoint and measure it.
Two: nanojev, the default — a Qwen3-0.6B causal decoder plus a trained decision head with three-stage prefix sharing — and decider-2b, a Qwen3.5-2B causal decoder where three of every four layers are linear attention. The README excerpt cuts off mid-row in that table.
More Jev resources like this
Self-hosted Python SDK and System One-compatible API for runtime-defined Choice, Boolean and Score decisions on pinned open models, across Torch, vLLM, MLX, llama.cpp and WebGPU. Each result records probabilities, token usage, timing, model revision and prompt version, with committed row-level benchmarks.
Jev Repos & SDKs#community#sdk#self-hosted
Community
chengyongru/fastjevPython
chengyongru/fastjevPythonA native Swift implementation of OpenJev, the Jev-compatible typed-decision server: send a state and typed questions over a Jev-compatible HTTP API and read every answer from model probabilities, on Apple silicon Macs or inside iPhone and Mac apps.
Jev Repos & SDKs#community#catalog-import#cobanov
Community
algorythm-canada/openjevswiftSwift
algorythm-canada/openjevswiftSwiftA lightweight Python engine that mimics Jev's structured decisions on local LLMs via Ollama, vLLM or llama.cpp: choice, score and noul questions come back as JSON with probabilities and confidence, plus an OpenAI-compatible gateway.
Jev Repos & SDKs#community#catalog-import#logicrw
Community
tapsin/jev-localPython
tapsin/jev-localPythonA local FastAPI inference server for the 150M-parameter Phocinae-Largha decision model that speaks the TypeSafe /v1/systemone contract: one POST returns noul, choice and score answers from a single forward pass, serving only 127.0.0.1 and never implementing chat or OpenAI-compatible endpoints.
Jev Repos & SDKs#community#catalog-import#logicrw
Community
phocinae/phocinae-serverPython
phocinae/phocinae-serverPythonA correctness-validated Laya runtime for Apple Silicon that runs upstream Laya on the MLX GPU and the Apple Neural Engine, using the ANE only after an artifact passes a parity gate on your machine, plus a local Jev-compatible server.
Jev Repos & SDKs#community#catalog-import#madewithjev
Community
tc3oliver/laya-applePython
tc3oliver/laya-applePythonAn independent reimplementation of the OpenJev mechanism on PrismML's 1.75-bit Bonsai weights: typed decisions read straight off the next-token distribution of a 5.95 GB 27B model in one forward pass.
Jev Repos & SDKs#community#catalog-import#cobanov
Community
nicolailassen/open-bonsai-jevPython
nicolailassen/open-bonsai-jevPythonFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.