User repository on GitHub
Judge every LLM response before the user sees it. AI SDK middleware and OpenAI-compatible proxy.
- Forks
- 1
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.Homepage
CommunityJev Integrations & Adapters
Tripwire
TypeScript guardrail that puts seven Jev checks in front of an AI SDK model or an OpenAI-compatible endpoint, blocks on high-confidence hard checks, downgrades low confidence to a flag, and logs every verdict as a PII-free JSONL row you can turn into an eval set.
- Stars
- 3
- Language
- TypeScript
- License
- MIT
- Published by
- Community
- Author
- noelzappy
- Last verified
- Oct 2026
About this project
Judge every LLM response before the user sees it. Seven checks in one 100 ms call to TypeSafe's Jev, cheap enough to run on 100% of traffic instead of sampling 1% with a frontier judge. AI SDK middleware: wrapLanguageModel({ model, middleware: tripwire(...) }). Zero infra. OpenAI-compatible proxy: point any client's baseURL at it. Policy as YAML: which checks run, how confident Jev must be, what a block does.
Quoted from the project's README. Read the full README.
- Languages
- TypeScript
Highlights
- Seven checks covering PII leak, complied injection, tone, forbidden category, on-topic, system adherence and hallucination risk, each mapped to a Jev primitive.
- Two drop-in shapes: wrapLanguageModel middleware with no infrastructure, or an OpenAI-compatible proxy that any client's baseURL can point at.
- Policy is YAML, including which checks run, per-check confidence thresholds and whether a block replaces, throws or annotates.
- Eval CLI prints per-check precision and recall, a verdict confusion matrix and hard-check coverage across thresholds 0.6 to 0.95.
Watch out
Version 0.1. The middleware and proxy work end to end against a mock judge with 29 tests, and the README states there are no accuracy numbers against real Jev yet; it recommends replacing the 27-item seed set with 300 to 500 rows from your own logs before drawing conclusions. It fails open: if Jev is unreachable the response passes with verdict flag and an empty checks map unless you override onJudgeError. Streaming is passthrough by default, meaning the user has already seen the response before the judge runs, while the proxy serves a streaming request as one SSE chunk after judging. Responses with no text, such as tool-call-only turns, are not judged and tool-call arguments are not inspected. State is trimmed to roughly a 32k token budget, oldest first. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
Not against real Jev yet. The README is explicit: middleware and proxy pass their end-to-end tests against a deterministic mock judge, and the eval harness plus a 27-item seed set exist so the first live run is one command. Its own advice is not to put this in front of users until a real run shows block precision above 95% on your data.
It fails open. The response is returned unchanged with verdict flag and an empty checks map, and the error is logged. Override onJudgeError if you want fail-closed; in sync mode an error thrown from it propagates to the caller. Decision log and onVerdict are telemetry and cannot affect the response.
Not in the default mode. With stream: passthrough, tokens stream unchanged and the judge runs when the stream ends, so on block the block message is only appended as a final text part. Use buffer to hold every part until the judge returns. Through the proxy, a streaming request is answered as a single SSE chunk after judging.
No by default. The proxy forwards the client's own Authorization header upstream, so it holds no secrets. If you set UPSTREAM_API_KEY so it spends its own key you must also set TRIPWIRE_PROXY_KEY, and the proxy refuses to start with an upstream key and no proxy key. Meanwhile the judged state is fitted to roughly a 32k token budget by trimming oldest first, so long system prompts and histories are the first thing to lose.
pii_leak and injection_complied are boolean, tone and category are choice, on_topic is boolean, follows_system is a 0-3 score and hallucination_risk is a 0-2 score. Any hard check above its threshold blocks; a soft check or a below-threshold hard check flags.
Jev sees text only within roughly a 32k token budget; the state builder trims long system prompts and histories oldest first before judging. Responses with no text, such as tool-call-only turns, are not judged at all.
npm install @noelzappy/tripwire ai, or bun add, then run bun run eval against eval/dataset.jsonl. The README says not to put it in front of users until block precision exceeds 95% on your own data, replacing the 27-item seed set with 300 to 500 rows.
More Jev resources like this
3GitHub stars
Fifteen independent local plugins for the Paseo desktop app, two of which use Jev: an evaluator exposed to Codex and Claude agents over MCP, and an orchestrator that routes a task to one model profile and effort level.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
HiepPP/hiep-paseo-pluginTypeScript
HiepPP/hiep-paseo-pluginTypeScript69GitHub stars
Provider-neutral TypeScript runtime for System One decisions: write choice, noul, and score code once, then run it on TypeSafe Jev, local Reflex, any compatible /v1/systemone endpoint, or a custom provider.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
iamaamir/system-oneTypeScript
iamaamir/system-oneTypeScript38GitHub stars
File-first project notebook where many agents share context in .context/ files while Jev only judges the next step and records it as JSON.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
pedrocivita/tocketTypeScript
pedrocivita/tocketTypeScript15GitHub stars
Uncertainty-aware semantic control flow and routable procedures for Effect, with type-safe patterns and policies over Effect Decision and DecisionModel where uncertain answers stay explicit branches.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
doeixd/discernTypeScript
doeixd/discernTypeScript12GitHub stars
Command-line checker that asks Jev whether each selected source file violates a plain-English convention, returning file-level probabilities for an agent or CI step to act on.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
iamtoomas/JevLintTypeScript
iamtoomas/JevLintTypeScript21GitHub stars
Language and VM for agent programs that wait for approval, survive restarts, and replay portable receipts without recalling the model.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
hraness/algalTypeScript
hraness/algalTypeScriptFrom the Jev community
Posts from builders shipping with Jev right now.