Jev directory
CommunityJev Models & Reimplementations
Wolf Defender Jev
Patronus Studio's 151M-parameter Jev-like decision model for AI-agent security: given a serialized agent scene, a security question and answer options it returns calibrated probabilities in one pass, reporting 92.8 balanced accuracy on 10,576 held-out decisions.
- Reading time
- 2 min
- Published by
- Community
- Author
- Patronus Studio
- Last verified
- Oct 2026
Used inJev for Security & Privacy
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- Fine-tune of jhu-clsp/mmBERT-small with 151M parameters; MIT licensed and small enough for a laptop CPU.
- Card reports 39 ms for a 51-token scene on an Apple M4 CPU, with no GPU and no API call.
- Self-reported 92.8 balanced accuracy and 5.4% false positives on 10,576 held-out decisions, ahead of 9.41B Clef-flash at 81.0 and 10.7.
- Covers prompt injection, jailbreaks, exfiltration, poisoned tool metadata, secrets and consent checks through choice, noul and score questions.
- Planted approval notes or forged trusted blocks flip at most 0.7% of detected attacks when the scene carries agent context.
Watch out
Card reviewed 2026-10-17; every benchmark number is self-reported on the card's own model-index panel with no third-party reproduction, and the card's steering section shows planted claims mainly make it flag benign documents more often. On the 10 external benchmarks Clef-flash leads 90.2 to 85.1, and XSTest over-refusal shows a 35.2% false-positive rate.
Questions about this entry
Short answers drawn from the linked project or page.
A Jev-like decision model for AI-agent security from Patronus Studio: 151M parameters and 15 languages, built to run on a laptop CPU and return a calibrated probability for every answer option you give it.
It takes the named-question shape of Jev-like models: a state (a serialized agent scene) plus questions of type choice, noul or score, and returns calibrated probabilities with no text generation and one forward pass per question.
A serialized agent scene: system context, user goal, tool calls, untrusted tool output or retrieved content, and the action the agent wants to take next, together with a security question and two or more answer options embedded in the input.
151M parameters, fine-tuned from jhu-clsp/mmBERT-small. The card reports 39 ms for a short 51-token scene on an Apple M4 CPU with no GPU and no API call, and says it runs on a laptop.
On 10,576 held-out decisions the card reports 92.8 balanced accuracy at 5.4% false positives, ahead of Clef-flash at 81.0 and 10.7 and Kev-4B at 74.3 and 12.1. The model-index lists 100.0 balanced accuracy on deepset prompt-injections with a 0.0 false-positive rate.
Patronus Studio publishes it under MIT as a fine-tune of jhu-clsp/mmBERT-small, with 15 supported languages and validation accuracy between 91% and 98% in each of the 14 non-English ones.
It leads on its held-out panel but trails Clef-flash on the 10 external benchmarks (90.2 versus 85.1), and the card's steering section shows planted notes mainly make it flag benign documents more often. XSTest over-refusal shows 73.9 balanced accuracy with a 35.2% false-positive rate.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
YouTube: Kubernetes admission control with a decision model, fail-closed.
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsMore Jev resources like this
A Qwen2.5-1.5B-Instruct fine-tune that emits typed dependency verdicts, safe, caution, or block with a recommendation, for Dependabot and Renovate pipelines, reporting 95.7% F1 on the card's 4,800-update OSV-Bench.
Jev Models & Reimplementations#community#open-models#local
Community
Superagent's security triage model: a continual fine-tune of AutoJev-27B that returns calibrated probabilities for prompt-injection and content checks, with a release policy at 0.70 unsafe.
Jev Models & Reimplementations#community#open-models#local
Community
Convai Innovations' 421M non-autoregressive decision model on Ollama: fine-tuned from ModernBERT-large, Apache-2.0, answering one question in under 10 ms on an M5 Max with 0.060 in-task calibration error.
Jev Models & Reimplementations#community#open-models#local
Community
A DistilBERT prototype from akumar0205 that sorts historical email into legitimate and spam, reporting 0.987 accuracy and 0.983 spam F1 on a held-out SpamAssassin split; it is not a phishing or malware detector.
Jev Models & Reimplementations#community#open-models#local
Community
A specialist full fine-tune of mistralai/Ministral-3-3B-Reasoning-2512 by rony for the LocalLLaMA/typed-decisions benchmark: it reads a case, question and lettered options and softmaxes next-token logits, reporting 0.808 accuracy and 0.0773 KL on all/test.
Jev Models & Reimplementations#community#open-models#local
Community
ActBro's 2B calibrated System 1 decision model on Ollama, built on OpenBMB's MiniCPM5-2B: 78.79% on JevBench among sub-2B open-weight models, 86.41% on the 10-source decision-v7 set, and one output token per decision via /v1/systemone.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.