A tool-using agent does not need a better prompt. It needs a policy that decides which calls may run, which need a human, and which are refused before anything is executed. Approval gates are where that policy becomes code. The question is where in the policy the model is allowed to speak.
The policy problem
A flat allowlist works for ten tools. It fails when an agent carries 50 to 200 MCP tools, as the jev-tool-permissions README describes, and the same tool is safe in one context and destructive in another. rm -rf build after a request to clean the build directory is routine. rm -rf ~/.ssh is not. A rule that only sees the command cannot tell them apart; a rule that only sees the conversation cannot enforce a hard limit.
The policy needs two layers: a deterministic floor that never calls a model, and a contextual layer that reads the request, the tool call, and the documents in scope. The model answers narrow questions inside that structure; it does not invent it.
Route before you classify
Jev-Guard is the clearest published version of the routing idea. It navigates a policy tree to select the applicable audit policy, then classifies safety conditioned on the retrieved policies. The tree spans moderation, cybersecurity, privacy, and legal compliance.
It reports 93.72% safety accuracy and 93.10% hierarchical routing accuracy across 7,033 cases, with 58 ms mean classification latency and 240 ms for the full pipeline. As a separate permission gate in the Hermes harness, it cut mean review time per task from 4.80 seconds to 1.33 seconds, with no policy violations for either guard.
Those numbers are self-reported. The abstract does not say whether the weights, code, or policy tree are released, and the harness comparison covers one integration. Treat the architecture as the transferable part and the percentages as a claim to reproduce.
What a gate must never do
The central property of a gate is that model output is evidence, not authority.
ActionGate makes this explicit. It asks six narrow questions in one request: alignment, target match, policy conflict, sensitive-data exposure, scope expansion, and missing intent. Its README states that a model score never overrides an RBAC, schema, limit, or duplicate failure. The permit is bound to the exact approved action, and expired, changed, or replayed permits are refused. The caveats are candid: early public release, no external security review, and an embedded mode whose grants live in memory and do not survive a restart or span replicas.
The Kinde starter kit shows the same division with identity in front. It verifies Kinde tokens, permissions, and flags before any Jev call or workspace write, then sends the call, the request, and the read documents to Jev for calibrated signals. Policy code decides allow, step-up approval, or deny. Writes are recorded to a ledger first, so a failed write never runs. On 300 labeled calls run three times, it reports 0.0% false-allow at 83.7% accuracy, against 12.4% for a single Jev verdict and 7.9% for an LLM judge, at 205 ms and $0.029 per 1,000 calls versus 2,073 ms and $0.931. The labels are author-written; re-run the benchmark before quoting it.
| Design | Who decides | Good for | Failure mode |
|---|---|---|---|
| Deterministic only | Policy code | Hard limits, credentials, paths | Cannot read intent |
| Model-assisted | Code, with model evidence | Contextual allow, step-up, deny | Model evidence treated as permission |
| Model-as-authority | The model | Prototypes, low-stakes routing | Prompt injection becomes authorization |
Fail closed is a budget decision
typesafe-gates turns the pattern into a small TypeScript library. Each gate declares its questions, state, an interpret function, and a fail-open or fail-closed policy. createJudge sends one request per item, runs eight concurrently by default, and returns answers in input order. The behavior is all-or-nothing: if any request fails, the rest abort and the invocation returns unavailable, with a reason such as no_client, request_failed, over_budget, or aborted. A budgetMs setting aborts in-flight requests for request-path gates.
That all-or-nothing default is right for approval: it is fail-closed by construction, and a partial batch would invite the caller to ignore the rest.
jev-tool-permissions applies the same rule to the Vercel AI SDK. Both the pre-flight gate and tool-list pruning sit on a deterministic allow/deny floor; deny rules run before anything reaches Jev. If the call errors or times out, the gate defaults to ask-human, never auto-approve, and pruning keeps every tool definition. The README prices Jev at roughly $0.04 per million input tokens and argues an LLM judge would cost more than the call it gates. That is a claim, not a measurement.
Gates must not widen permissions
A gate that can turn a prompt into an allow is worse than no gate, because it hides the decision.
opencode-jev-guard is careful about this. It asks six questions per shell command: verdict, host litter, global install, global config, harmful, and privacy. A command is safe only if the verdict is run with confidence at or above minConfidence and every risk is below its threshold; anything else asks. If Jev is unreachable, times out, or no key is set, the command asks. The guard only ever adds prompts; it never turns a prompt your existing settings require into an allow. It covers local and FarHand remote-shell commands.
guardrails.md states the same rule for its bash gate: named exceptions in the policy file do not override a destructive verdict. A human must run the command or disable the gate for the session. The gate scores each command in about 100 ms and blocks at 0.7 destructiveness, with the README's example at 0.98. It covers bash only and publishes no false-positive rate.
Version and test the policy
A policy tree is a software artifact: it needs an owner, a version, and tests. The Jev-Guard paper describes the policies as editable; that is a strength only if changes are reviewed like code.
Keep the policy in the repository, not a prompt, and test it against recorded cases, including prior approvals and denials. The jaggedness notes are relevant here: the model reads literally, negations confuse it, and structural invariants do not hold. A policy that depends on the model inferring an unstated exception fails in a way that looks like a bad model but is a bad policy.
Two more checks belong in the suite. jev-guard shows the value of an audit mode that logs would-be blocks without interrupting anything. Jev Consensus Gate shows the value of asking the same action three ways: safety, legitimacy, and adversarial, with all three required to allow and any escalate routed to a human. Three calls cost about $0.00006 per action at $0.042 per million input tokens, roughly 3x the input cost of one call. Use consensus where framing blind spots matter; use a single call where latency dominates.
A reference flow
- Authenticate the caller and check permissions before the model sees anything.
- Apply deterministic deny rules for credentials, paths, limits, and schema violations.
- Route through the policy tree to the applicable policy.
- Ask a small set of narrow questions in one request.
- Apply thresholds in code and fail closed on errors or timeouts.
- Issue a single-use grant bound to the exact action.
- Log the policy version, question text, state hash, answers, and decision.
- Send step-up cases to a human queue with a fresh sign-in and an expiry.
The decision-gates article covers how this fits into agent frameworks, and the guardrails article covers the input-side boundary. The policy tree is the part that scales. The gate is the part that makes it real.
