Question 1

Is the 100% catch rate real, or a corpus written to pass?

Accepted Answer

It is a corpus the project generated itself, and the README calls that a floor rather than proof against a real adversary. More telling is the ablation: a single 'is this dangerous?' question also caught 100% of attacks but blocked 39.2% of legitimate hard negatives, which is the number the author says matters most.

Question 2

Why not just use one prompt asking whether the tool call is dangerous?

Accepted Answer

Because one scalar cannot separate catastrophic-looking from explicitly requested catastrophic. Five independent typed questions, one per hazard, let the policy table reason about which hazard fired and combine that with scope and blast radius. Both versions cost one batched call, so decomposition adds token count rather than round trips.

Question 3

Two gates failed in your own table. Does that mean it is not production ready?

Accepted Answer

The README says so and points at report.md for what that does and does not mean. Noul calibration error came out at 0.156 against a 0.10 target, and p95 added latency at 595 ms against 500 ms. The headline gate, block rate on legitimate hard negatives, passed at 0% on the held-out split.

Question 4

How do I re-tune thresholds without paying for inference again?

Accepted Answer

Raw judgments are logged per record rather than just the final action, so eval/replay.py re-scores an existing decision log under a changed policy.py offline in milliseconds. The run itself is pip install, an SDK smoke check, corpus generation, a stubbed wiring check with no spend, then the full shadow run.

Question 5

What happens if I strip the context out of the state?

Accepted Answer

The catch rate collapses. Ablation C ran the same five-question battery on tool name and args only, without user_request, retrieved_content or tenant_policy, and attack catch fell from 100% to 61.6% while the benign side barely moved, at $0.0000296 per call. The README's conclusion is that the situational state is the moat, not the model call.

Question 6

What did a full 600-record run cost?

Accepted Answer

$0.054 in total across 600 records, which works out to $0.0000365 per gated call at list price. The approval-prompt rate on benign traffic read 20.9%, but 88% of those prompts exactly matched ground truth because the corpus deliberately contains legitimate-but-scary requests, leaving real friction at 1.9%.

Question 7

Which number would you watch in production?

Accepted Answer

Block rate on legitimate hard negatives, and the repo puts it alone on that basis: a firewall that blocks legitimate agent actions gets switched off in week two and then protects nothing. That gate passed at 0% on the held-out split of 320 records, while calibration error and p95 latency, both declared gates, did not.

TypeSafe AI Firewall

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev community