How is this different from a normal guardrail wrapper?

Post-hoc guardrails buffer a sentence, call a second model taking 850-1600ms, and by then 18-35 sensitive tokens are already on screen. Tokengate verifies batches before release and drops the buffer on violation, so leakage stays at zero across the measured engines even though evaluator latency spans five orders of magnitude.

What does a slow evaluator cost me in practice?

Mostly stutter, not leakage. Leakage is independent of evaluator latency by design, but a slow engine still delays release. Measured cost on a 400-token response is about +0.11s with Jev versus +2.78s with a much slower engine, after batching and pipelining.

How much app code has to change to try it?

Change one line to point your OpenAI client at localhost:8787. Upstream chunks are replayed verbatim so id, usage, finish_reason and tool_calls survive, and violations return finish_reason content_filter which existing SDKs already handle.

Does the proxy itself handle authentication?

No. The proxy does not authenticate its own callers, so put it behind your existing API gateway. Without UPSTREAM_KEY it forwards the client Authorization header, which lets it serve multiple tenants while holding no keys.

What engines can evaluate the stream?

One gate with swappable engines, all returning five 0 to 1 scores compared against a threshold. local is a regex fallback at a 0.03 ms median, jev is TypeSafe Jev with one call for all five criteria at about 308 ms, and claude uses Haiku 4.5 or Opus 5 at roughly 1,554 or 2,746 ms. Every engine leaked zero tokens in the scenario set.

How do I define my own policy?

Copy tokengate.config.example.json to tokengate.config.json, or point TOKENGATE_CONFIG at your own file. Each criterion needs id and when; name defaults to id, threshold to 0.8, and action to block (block, replace or block+log), with optional regex patterns for the local engine. A config file replaces the defaults entirely, and a bad config refuses to start the server.

What is the known weak spot?

The false-positive rate is unmeasured, since five scenarios are not an eval set, and the README lists that under known limits. A fixed evaluation window also means violations only visible across the whole response can slip through; raising LOOKBACK catches more at the cost of throughput.

CommunityJev Practices & Patterns

Jev Model Tokengate

OpenAI-compatible proxy that holds streamed tokens in a sliding buffer and scores the full criteria matrix before release, cutting the stream before a violating token reaches the client.

See highlights

Open github.com

Stars: 4
Language: JavaScript
License: MIT
Published by: Community
Author: Thanh-Mathieu95
Last verified: Oct 2026

Thanh-Mathieu95/jev-model-tokengate

User repository on GitHub

View on GitHub

An OpenAI-compatible proxy that sits between your LLM and your users. It evaluates each sliding window of tokens while the response is still streaming and cuts the stream before a violating token can reach the screen.

Forks: 0
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

An OpenAI-compatible proxy that sits between your LLM and your users. It evaluates each sliding window of tokens while the response is still streaming and cuts the stream before a violating token can reach the screen. Same attack prompt, two architectures side by side. Left (post-hoc): the API key sk-proj-... and DBPASSWORD render in full, and the "content removed" notice arrives 2.98s later — 173 characters already leaked. Right (tokengate): the stream is cut inside the buffer. 0 characters leaked.

Quoted from the project's README. Read the full README.

Tagscommunity awesomejev catalog-import

Works forJev for Classification Jev for Evaluation

Used inJev for Software & DevTools

Languages: JavaScript

Highlights

Zero leaked characters measured across 5 scenarios on 3 engines, versus 173 leaked by post-hoc redaction.
Swappable local, Jev, and Claude engines returning five 0-1 scores against a shared threshold.
Evaluates only LOOKBACK recent tokens plus current batch with ordered pipelined passes to avoid quadratic cost.
Policies live outside source in a config file, and a malformed criterion refuses startup rather than failing open.

Watch out

The 35ms interception target is missed at about 300ms over the public internet, and fixed-window evaluation can miss violations spanning a whole response. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

ego-jev-ultrafast

A single-file port of jev-ultrafast that lets Jev drive an Ego Lite browser on macOS: an in-page script indexes visible controls and one typed-choice request returns both the operation and the target index.

Jev Practices & Patterns#community#awesomejev#catalog-import

Community

shikaizhong-design/ego-jev-ultrafastJavaScript

cairn-jev-lab

Memory admission evaluator that asks Jev three typed Choice questions on support, durability and commitment, then applies a versioned policy to recommend save, skip or defer.

Jev Practices & Patterns#community#awesomejev#catalog-import

Community

Cairn-ink/cairn-jev-labJavaScript

jev-shadcn-lint-eval

An eval that puts Jev two questions about shadcn-ui/lint: whether each finding is a real rule violation, and whether the linter's message tells you what to change. It runs the real linter over 131 rule test cases and scores both the detections and the messages.

Jev Practices & Patterns#community#jevbest#catalog-import

Community

blas0/jev-shadcn-lint-evalJavaScript

jev-on-a-laptop

Unofficial laptop study of parallel constrained decoding: prefill once, share the KV cache across fields, and decide all schema fields in one batched forward pass without writing JSON.

Jev Practices & Patterns#community#jevbest#catalog-import

Community

rorshopping/jev-on-a-laptopPython

jev-column-race

Live race rig that labels 1,000 real app reviews with Jev typed questions against Gemini JSON output on identical batches, then replays recordings and re-ranks rows locally without extra model calls.

Jev Practices & Patterns#community#jevbest#catalog-import

Community

goodrahstar/jev-column-raceJavaScript

jev-rag-benchmark

Vendor-neutral harness that tests whether Jev reranking improves a small RAG stack, using frozen hybrid BM25 plus BGE-M3 candidates and isolating retrieval, rerank, and generation stages.

Jev Practices & Patterns#community#jevbest#catalog-import

Community

erendikmenn/jev-rag-benchmarkPython

From the Jev guides

Original write-ups on what this entry is for.

Designing the State: What to Include and What to Drop

Oct 12, 2026

Most Jev accuracy problems are state problems. What belongs in a state, what is a pre-made judgment to leave out, and why naming the slice you ask about matters more than the wording.

Jev Practices & Patterns#state#design#primitives

Read article

Noul, End to End: One Probability and a Dozen Thresholds

Oct 12, 2026

A Noul returns one number and no separate confidence. How to read the value, pin its boundary with criteria, split compound questions, and set a per-action threshold that matches the cost of being wrong.

Jev Practices & Patterns#noul#primitives#confidence

Read article

Scoring Without Measuring: Bands, Rubrics, and Composite Triage

Oct 12, 2026

A Score is a weighted position, not a ruler. Why the number can land between levels, why levels are judged separately, and how to combine several atomic scores in code instead of one crowded rubric.

Jev Practices & Patterns#score#primitives#scoring

Read article

Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai