Are the benchmark numbers real?

They are repo-reported runs on a public held-out suite with a named Ollama model and GPU setup, with results, methodology, and bench commands committed for reruns; the private OpenJev mix is cited without inventing a score against it, so rerun before trusting them.

Why self-host this instead of paying for hosted Jev?

You keep the Apache-2.0 control plane, run on your own GPU or API with mock support for CI, and keep the same choice, score, and noul question shape for routing, scoring, and gating without per-call vendor dependence.

Do I need to train or fine-tune anything?

No. You point it at a backend model, send state plus questions, and threshold the returned probabilities; there is no training step, only inference plus the API, SDK, CLI, and playground around it.

What does setup actually require?

Node 20 plus pnpm 9, repository clone with install and build, the server on its local port with playground and health routes, and optionally an Ollama model pulled with backend env vars set before restarting the server.

What does the held-out accuracy table actually report?

One model, qwen3.5:9b through Ollama at 100% GPU, on 600 items per task: Banking77 83% against 1.3% chance and Jev's roughly 82% held-out, CLINC OOS 86%, AG News 86.5% against 25% chance, SST-5 89.5% within one level, and Civil Comments 81% on a noul against 50% chance. OpenJev's private 10k mix is cited only, at Jev 85.4% and OpenJev 84.0%, because those questions are not fully public.

What are the speed and stability numbers?

The preferred path is letter-token readout when the backend exposes logprobs, falling back to constrained decode on Ollama. On qwen3.5:9b at full GPU, two questions run p50 504 ms and p95 803 ms, parse-fail and format hallucination are 0%, and rerun agreement flip is 0% with zero KL. Scaling questions is close to linear: 443 ms at one question to 8,011 ms at fifteen, with confidence at fifteen only about 1.17 times that at one.

What else ships besides the engine?

Three npm packages under @kev-ai plus a Python client on PyPI: sdk with LangChain and LlamaIndex adapters at the same import path, server for the decision API, playground and OpenAPI document, and cli providing kev and kev-mcp. On port 3000 the server exposes /playground/, /health and /openapi.json, and there is a MIGRATION.md for coming from Jev.

CommunityJev Practices & Patterns

Kev — arjun988

Self-hosted Apache-2.0 decision engine with a Jev-style wire format that turns a model you point at it into typed choice, score, and noul answers.

See highlights

Open github.com

Stars: 20
Language: TypeScript
License: APACHE-2.0
Published by: Community
Author: arjun988
Last verified: Oct 2026

arjun988/Kev

User repository on GitHub

View on GitHub

Open-source System One decision engine. Typed choice / score / noul with calibrated probabilities. Self-host with Ollama or any OpenAI-compatible model. Apache-2.0.

Forks: 2
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.Homepage

This is one of several Jev listings for the same project. The canonical entry is Kev — jaredpalmer.

About this project

Typed decisions for software. Send context. Ask choice , score , or noul . Get calibrated probabilities — not a paragraph to parse. Install · Docs site · Quick start · Benchmarks · Why Kev · Playground · SDKs · Migrate from Jev. LangChain / LlamaIndex: @kev-ai/sdk/langchain · @kev-ai/sdk/llamaindex PyPI coming later. NotJev : Kev is an open-source System One decision engine: Apache-2.0, self-hosted, Jev-style wire format. Bring your own model (Ollama, vLLM, OpenAI-compatible) — or start with the built-in mock.

Quoted from the project's README. Read the full README.

Tagscommunity awesomejev catalog-import

Works forJev for Classification Jev for Ranking

Used inJev for Software & DevTools

Languages: TypeScript

Highlights

Brings your own backend such as Ollama, vLLM, or OpenAI-compatible models, or starts with the built-in mock.
Ships a TypeScript SDK, decision API server with playground, CLI and MCP tools, plus LangChain and LlamaIndex adapters.
Answers many questions in one request with typed keys, full distributions, confidence, and threshold-ready noul.
Documents accuracy and ops benchmarks with held-out suites and runnable bench commands for rechecking.

Watch out

Benchmarks depend on the backend, checkpoint, hardware, and task; the OpenJev private mix is cited only since its questions are not fully public. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

claude-jev

Claude Code plugin that routes code review, debugging, planning, and search triage through typed Jev judgements with fixed thresholds applied in code.