Jev Decision Service

A lightweight Jev / System One compatible HTTP decision gateway. It converts finite-candidate decisions — noul, choice and score questions — into candidate token scoring tasks and reuses an already deployed SGLang or vLLM inference service, so no model training or weight modifications are needed.

No. There is no SFT, LoRA or extra classification head: an ordinary generative LLM works directly as the decision model. The gateway itself loads no model, no PyTorch, no Transformers and no local tokenizer; it only communicates with the remote inference service over HTTP.

The prompt goes through next-token scoring, the logprobs of candidates like A / B / C are read, run through softmax, and the gateway builds the structured decision result from the probabilities rather than from the model's output format. Each question needs only one scoring call, with no explanation or chain of thought.

The same model and SGLang service on a task making three judgments in one call (choice, noul, score): both approaches were 3/3 correct with valid JSON and schema; average latency was 1282.3 ms for chat completion versus 232.1 ms for the gateway, with 212 output tokens (185 reasoning) versus 3. Raw data is committed in benchmarks/results/20260921_174806/.

Candidate probabilities, ready for threshold checks, human approval and agent routing, in request and response structures compatible with Jev / System One noul, choice and score. It is a standard HTTP service, so Python, Go, Java, agent and workflow systems can integrate it.

An existing SGLang or vLLM deployment. The gateway needs no GPU or model files itself and ships with one-command Docker deployment; the published benchmark used 4× NVIDIA L20 with a 27B model on tensor parallel 4.

It is one simple decision task where both approaches produced correct final answers, averaged over three runs on the author's hardware, so it demonstrates latency and token savings rather than accuracy advantages. The repository metadata we captured also lists no license.