Will my existing TypeSafe client work against it unchanged?

Yes. It speaks the same /v1/systemone wire format as the hosted API, so clients, SDKs and benchmarks built for TypeSafe work against it unchanged.

Can I run the model on a plain CPU-only machine?

Yes. It runs a quantized ONNX build via onnxruntime with no GPU required, though you need outbound access once to download the model into the cache volume.

Are the KEV versus Jev benchmark numbers trustworthy?

The digest quotes the kev-4b model card on frozen items: close to Jev in-distribution but trailing out-of-domain on calibration and confident-error rate.

Will one instance scale with concurrent requests?

It handles concurrent requests safely but queues them over a fixed thread pool, so run multiple replicas behind a balancer for more throughput.

What are the system requirements?

Docker Engine and Compose v2, CPU only with about 4 cores and roughly 4 GB RAM, about 3 GB free disk on first run, and outbound access to huggingface.co on first startup to fetch the model and tokenizer, which are then cached.

How do I configure it?

All configuration is environment variables, normally in .env: KEV_MODEL, KEV_MODEL_FILE (default model_q4.onnx), KEV_MODEL_DIR, ONNX_NUM_THREADS, PORT, KEV_MAX_TOTAL_TOKENS, and the minimum and maximum score levels.

What is the default model and its license?

onnx-community/kev-4b-ONNX on Hugging Face, Apache-2.0, downloaded and cached at startup. This repository contains only the serving code and redistributes no model weights.

CommunityJev Models & Reimplementations

Kev Onnx

Self-hosted CPU-only server that speaks the /v1/systemone wire format for the KEV decision model, packaged as one container with ONNX Runtime and a health check.

See highlights

Open github.com

Stars: 9
Language: Python
License: APACHE-2.0
Published by: Community
Author: aleskxyz
Last verified: Oct 2026

aleskxyz/kev-onnx

User repository on GitHub

View on GitHub

Self-hosted, TypeSafe-compatible /v1/systemone API server for the KEV decision model - CPU-only ONNX inference via FastAPI, no GPU required.

Forks: 0
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

Self-hosted, TypeSafe-compatible API server for KEV, a decision model: instead of generating text, you hand it a state (a document, a support ticket, any piece of context) plus one or more typed questions, and it hands back a calibrated probability for every possible answer — in one forward pass, no parsing required. Drop-in compatible. Speaks the same /v1/systemone wire format as TypeSafe's own hosted API, so any client, SDK, or benchmark built for it works against this self-hosted server unchanged.

Quoted from the project's README. Read the full README.

Tagscommunity github

Works forJev for Classification

Used inJev for Software & DevTools

Languages: Python

Highlights

Drop-in /v1/systemone compatibility so existing SDKs work without changes.
CPU-only quantized ONNX build with model and tokenizer cached in a volume.
Answers choice, noul and score questions plus single-model /v1/models listing.
Async server with shared InferenceSession; scale out with replicas for throughput.

Watch out

Single-process CPU server; concurrent requests share a fixed compute pool and latency varies with hardware and load. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

gut

Python helper that turns judgment calls into one-liners like likely, classify and rate, defaulting to Jev while also running on local or OpenAI-compatible backends.