Screening agent actions with Jev

A first impression with a repo behind it. The author links a harness for evaluating Jev as a fast monitor and action gate, so the setup is inspectable, but the post reports results in one line with no counts, no attack list and no false-positive rate. Nothing here is an independent benchmark.

The source does not give a number. The claim is qualitative in both the post and the thread, and the dataset and method are summarized in the linked repo rather than reported in the post. If false blocks on legitimate tool calls matter to you, read the harness yourself before trusting the framing.

Escalate only the suspicious ones, per the author's own follow-up: Jev filters everything and sends just the suspicious cases to the frontier model. That is the setup described as cutting cost substantially, and it also keeps the monitor off the critical path for benign traffic. A block-everything gate would be a different design with different latency and false-block tradeoffs.

The author's conclusion is no, and the post is careful about it. Fast non-writing models are not going to replace LLMs, but they could make real-time safety checks practical by handling the volume and escalating the rest. That is a latency and cost argument about screening, not a claim that Jev reasons better than the model it filters for.

The author is Nick, a senior research engineer working on technical AI safety, and the linked harness is nican2018/shade-arena-jev-monitor, which evaluates Jev as a fast monitor and action gate. The post is dated 17 September 2026 and has a few hundred views.

It is the author's framing of the model: it does not write an explanation, it returns probabilities for the questions asked, which is what makes it fast enough to sit in front of every action. That framing is why the post treats it as a screening layer rather than a replacement for the reasoning model.

It quote-posts Diogo Almeida's release thread, which says Jev is up to 20-200x faster and 40-400x cheaper than LLMs, so the guard result is being read in that context rather than against an independent baseline.