Article
CommunityJev Guides & Articles
HiddenLayer: Jev Talking - security of decision models
HiddenLayer's insight post walks security teams through its Jev tests: on a simulated checkout page Jev gave the attacker ad's button 91% against 9% for Place order, and TokenBreak edits cut one prompt's toxicity score by more than two thirds so Jev stopped flagging it.
- Format
- Article
- Reading time
- 3 min
- Published by
- Community
- Author
- Kevin Kelleher
- Last verified
- Oct 2026
Highlights
- On a simulated checkout page, Jev's API gave the attacker ad's Confirm Price button 91% and the real Place order button 9%, per HiddenLayer's tests.
- Before the ad, the converted test model gave Place order 99.7%; after, it gave the ad button 79.9% and Place order 19.3%.
- In the safety-filter test both models flagged all 30 toxic prompts before edits; after TokenBreak edits many slipped past both.
- The edits cut one prompt's toxicity score by more than two thirds and Jev stopped flagging it, though the meaning was unchanged.
- HiddenLayer recommends testing decisions in the application where they will be used and monitoring what goes in and out of the model.
Watch out
Copy reviewed against the published insight post on 2026-10-17. HiddenLayer is a vendor selling AI security products, and its test model is a Qwen3.8 27B conversion, not Jev: the post says converted-model results are close to Jev's but not identical, Jev is a closed API tested from outside, and TypeSafe's response is not reported.
Questions about this entry
Short answers drawn from the linked project or page.
It summarizes what HiddenLayer Research found when it tested TypeSafe's Jev, a decision model that returns choices, scores and probabilities instead of text, and explains what security teams can do about the findings.
A checkout-page test, where a planted ad tried to redirect a browser agent's click, and a safety-filter test, where edited toxic prompts were checked against Jev's toxicity scoring.
Among the clickable buttons, Jev's API gave the sponsored ad's Confirm Price button 91 percent and the genuine Place order button 9 percent, so the agent clicked away from the intended purchase.
Before the ad it gave Place order 99.7 percent; after the ad it gave the ad button 79.9 percent and Place order 19.3 percent. The same ad also derailed the text-generating version of the same model.
TokenBreak is a 2025 HiddenLayer attack that adds letters to key words, turning aggressive into Naggressive, so a classifier splits text differently and misses it. After the edits Jev stopped flagging a toxic prompt, and its score fell by more than two thirds.
No. Jev is a closed API, so HiddenLayer built a test model by converting Qwen3.8 27B to read the probability of each allowed option in a single pass. The post says it came close to Jev in decision quality and speed, but close is not identical.
Test decisions in the application where they will be used, monitor what goes in and out of the model, and put independent checks around decisions that trigger real actions. The post lists five questions to bring to a security review.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
YouTube: Kubernetes admission control with a decision model, fail-closed.
Hacker News: SafeDep Gryph plus Jev checks for compromised coding-agent behavior.
2 pointsDetecting Compromised AI Coding Agents with Jev and GryphRead the thread on Hacker NewsX: Jev and Laya on security queries
X: Testing Jev for prompt injection
More Jev resources like this
HiddenLayer researchers tested Jev and an in-house System One-style model: on a simulated checkout page an attacker-controlled ad made the browser agent click away from the purchase, and TokenBreak-style edits lowered toxicity scores for prompts both models had flagged.
Jev Guides & ArticlesArticle#community#press#security
Community
Pillar Security analyzes Jev and System One models: a smart-home demo answered three typed questions in about 200 ms with no language model in the loop, while prompt scanning, output filtering and chain-of-thought monitoring all have nothing to inspect. TypeSafe's docs say state is not treated as hostile by default.
Jev Guides & ArticlesArticle#community#press#security
Community
Rogue Security's post uses Jev and System One models to frame agentic security as a decision problem: the same send_file action should be allowed, blocked or reviewed depending on who requested it and where the data goes, illustrated with policy case files rather than product results.
Jev Guides & ArticlesArticle#community#press#security
Community
VentureBeat's Louis Columbus reports Jev is spreading into agent infrastructure faster than enterprises can audit it. In one Octomind test a planted field cut a block probability from 0.76 to 0.48, and TypeSafe and Pydantic both warn inputs can steer answers.
Jev Guides & ArticlesArticle#community#press#security
Community
A LessWrong pilot testing Jev as a trusted monitor in the ControlArena APPS backdoor setting: one yes/no question reaches AUROC 0.976 and catches 90% of backdoors at a 2% false-positive rate for about $0.04 per thousand submissions.
Jev Guides & ArticlesArticle#community#press#security
Community
Hacker News link to SafeDep's write-up on Gryph telemetry plus narrow Jev yes/no checks against developer policy.
Jev Guides & ArticlesPost#community#security#agents
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.