User repository on GitHub
Any chat model, any engine (SGLang, vLLM, transformers) as a Jev-compatible probability decision service: one prefill, one label token
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Tools & CLIs
Llm2jev — tic-top
Turns any chat model into a Jev-compatible decision service: state, question and all options go in one prefill, and the logprobs of the option labels at a single position become probabilities. Runs on SGLang, vLLM, transformers or MLX.
- Stars
- 8
- Language
- Python
- License
- MIT
- Published by
- Community
- Author
- tic-top
- Last verified
- Oct 2026
This is one of several Jev listings for the same project. The canonical entry is LLM2Jev — Yinsongxu.
About this project
Turn any chat model into a Jev-compatible probability decision service, on SGLang, vLLM, plain transformers or MLX (Apple silicon), with no model or engine changes. One forward per question, however many options (up to 255). No generation, no sampling. All options in one prompt, so options compete directly ("none of the above" can depend on the others). Standard chat template, thinking off. Works with instruct models zero-shot; fine-tuned checkpoints use the same prompt.
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
- Languages
- Python
Highlights
- One forward pass per question regardless of option count, up to 255; all options sit in one prompt so they compete directly against each other.
- Choice returns the argmax, the full distribution and confidence as `1 − H(p)/log K`; score returns the expected level `Σ i·pᵢ` with per-level probabilities.
- A parity harness scores each backend against the transformers reference: argmax must agree and probabilities must stay within 0.03.
- Each label token is verified at startup to be exactly one new token at the real prompt ending, so tokenizer spacing is checked rather than guessed.
Watch out
This is an independent implementation of the documented System One wire format, not affiliated with TypeSafe, and its probabilities are uncalibrated until you fit them — the temperature flag rescales the distribution without ever moving the argmax, so treat raw confidence as a ranking signal rather than a probability. Engine versions matter more than they should: one SGLang release loads some models and then scores them wrongly on text, vLLM 0.26 accepts media parts and ignores them, and MoE runs need a looser tolerance because of bf16 routing noise. Expect small probability differences between backends and read them as kernel noise, not bugs. The JevBench table reports public-item accuracy plus an estimated composite that assumes the sealed items. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
They start as softmaxed logprobs and stay uncalibrated until you make them so. The engine returns the full-vocabulary logprob of each option label at a single position with a single generated token, and those are softmaxed over that question's labels. The temperature flag rescales the distribution and never changes the argmax, so it cannot repair ordering — only the shape of the confidence attached to it. The README's advice is to fit it once, globally, on held-out data if calibrated probabilities matter to you.
There is a parity harness for exactly that. It scores the same rendered prompts through your engine and through the in-process transformers reference, and requires every question to agree on the argmax while staying within 0.03 in probability. The README also documents where that tolerance is exceeded and why: bf16 kernel noise accounts for 0.01 to 0.03, MoE routing noise needs a looser tolerance, and certain SGLang versions load models they then score incorrectly on text. Run parity before trusting a backend on a new model.
Because the endpoint rejects the request without them. Two flags are needed for logprobs and token ids, and a third opens the tokens-in endpoint used for images, video and audio. Without them vLLM answers every request with a 400, which the client converts to a 422 — but the message you get back is vLLM's, not a hint about missing flags, which is exactly the confusion the README flags. There is also a name-matching trap: the model name you pass must be the one the server answers on.
Yes, through the in-process backends. MLX on Apple silicon runs text-only, taking Hugging Face ids or quantized community repos. Plain transformers also runs in-process as the reference implementation, and the README flags it as slow with no prefix cache — each question is a full prefill, which is the price of not having SGLang's radix cache or vLLM's automatic prefix caching. Both are useful for establishing a parity baseline on your own machine before moving to a served engine.
pip install llm2jev brings the client and server while the engine runs separately; extras add the in-process transformers backend with vision, or MLX on Apple silicon. Start SGLang, vLLM, plain transformers or MLX, then run llm2jev --model ... --backend ... --url ..., pointing at that engine.
The request is rendered once with the model's own chat template with thinking off, the assistant turn is prefilled with Answer:, and the engine returns the full-vocabulary logprob of each label token at that one position. Then p is softmax of logprobs over T across the question's labels, so one forward pass answers however many options there are, up to 255.
Labels are A to Z, then two-letter labels such as AA and AB. At startup each label is checked against the real prompt ending and must be exactly one new token there, so whether a model writes A or a spaced A after Answer: is decided by the tokenizer rather than guessed.
More Jev resources like this
34GitHub stars
Pure-HTTP automation for the TypeSafe register, magic-link login, onboarding, and API-key creation flow, with pluggable temp-mail backends, concurrent batch runs, and key verification exports.
Jev Tools & CLIs#community#awesomejev#catalog-import
Community
2951461586/Jev-Register-ToolPython
2951461586/Jev-Register-ToolPython36GitHub stars
Python toolkit plus 0.8B multimodal model that takes a message, document, or image and returns calibrated choice, yes/no, and score answers in one forward pass for local decision workloads.
Jev Tools & CLIs#community#awesomejev#catalog-import
Community
PsiACE/dohnutsPython
PsiACE/dohnutsPython39GitHub stars
Video tool where Jev scores subtitle segments and code assembles a short highlights cut plus a de-watered full cut with timestamped summaries.
Jev Tools & CLIs#community#jevbest#catalog-import
Community
cclank/jevclipPython
cclank/jevclipPython30GitHub stars
Forkable weekday radar that asks Jev one yes-or-no question per interest over each new arXiv announcement and publishes a page plus RSS feed.
Jev Tools & CLIs#community#jevbest#catalog-import
Community
Eliot5566/JEV-Paper-RadarPython
Eliot5566/JEV-Paper-RadarPython38GitHub stars
Async supervisory layer for Hermes Agent that locks Definitions of Done, proves mechanical criteria locally, and calls Jev only for unresolved semantic checks under a token budget.
Jev Tools & CLIs#community#jevbest#catalog-import
Community
keeltrace/hermes-nervePython
keeltrace/hermes-nervePython15GitHub stars
Two-tier decision engine pairing a 22.6M-parameter MiniLM encoder on the Apple Neural Engine with the on-device Apple Foundation Model as fallback for uncertain inputs.
Jev Tools & CLIs#community#awesomejev#catalog-import
Community
kerryrm/systemANEPython
kerryrm/systemANEPythonFrom the Jev guides
Original write-ups on what this entry is for.
The request shape barely changes across surfaces; the surrounding contract does. When to call the API directly and when a gateway, a Worker binding, or a router entry earns its extra hop.
Jev Tools & CLIs#gateway#vercel#cloudflare
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Jev labels a record that already exists. Code moves, flags, or holds it. How the document, mail, and transfer tools keep the action out of the model.
Jev Tools & CLIs#classification#email#security
Read article
From the Jev community
Posts from builders shipping with Jev right now.