User repository on GitHub
Open source implementation of Poke
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Practices & Patterns
Openpoke Meets Jev
Fork of OpenPoke that replaces three yes/no LLM calls with batched Jev questions: important-email screening, a tool-call guardrail for the execution agents, and a relevance re-check on search results, with the old classifier left as the fallback.
- Stars
- 1
- Language
- Python
- License
- MIT
- Published by
- Community
- Author
- 0xShin0221
- Last verified
- Oct 2026
About this project
A fork of OpenPoke that stops asking a chat model to make decisions.
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
- Languages
- Python
Highlights
- One Jev call screens each email on four things — urgency, security code, bulk mail, and whether the body is trying to instruct an AI — before any LLM sees it.
- The tool guardrail holds a call only when it contradicts the assignment and cannot be undone; it returns to the agent as a tool error, not a refusal.
- The contamination harness ran 43,776 requests with zero failures: eight mechanisms and 80 templates surfaced nothing, with an upper bound of 0.51%.
- With no key set, every entry point returns undecided or allow and the fork behaves like upstream OpenPoke byte for byte.
Watch out
The comparison table is per decision, not per pipeline, and the author flags the limits himself: the cost ratio is list-price arithmetic over unequal token counts because the LLM arm carries a tool schema, and the latency figures come from one machine in one afternoon with one provider per arm, which he would read as an order of magnitude and not much more. There is no labelled set in the repository, so the harness reports agreement, latency, cost and distribution and refuses to print accuracy or Brier — agreement is not accuracy. The most interesting finding is a hole in the author's own defence: silently dropping a message flagged as injection is exactly what a suppression attacker wants, so flagged bodies are now quarantined with a code-assembled notice. How much the screening saves depends on your inbox and he declines to quote a number. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
Per decision, and the author says so. The cost ratio is arithmetic on list rates over unequal token counts — 55,554 against 41,739 — because the LLM arm carries a tool schema, and the latency numbers come from one machine in one afternoon with one provider per arm. He would read 6× as an order of magnitude and not much more.
You cannot, from this repository. There is no labelled set, so the harness reports agreement, latency, cost and the probability distribution, and deliberately refuses to print accuracy or Brier scores. The 33 of 36 agreement at the 0.75 bar means both arms agreed, not that either one was right.
Yes, and finding it is the more interesting half of the write-up. Flagging a body as an injection attempt originally dropped the message, which is precisely what a suppression attacker wants: appended override text tripped that gate on 95.3% of emails across all twelve carriers, including one-time security codes. Flagged bodies are now quarantined — withheld from the agent, but reported with sender and subject clipped in a notice assembled in code.
Only one case stops anything, and reads are never blocked. There are three rungs above allow, but a call is held only when it reads as contradicting its assignment and the effect cannot be undone, and then it comes back to the agent as a tool error so the agent can correct itself. Holding on off-task alone was dropped after a calibration replay found that signal weakest and behind most holds with no user complaints.
Everything is optional. With no TYPESAFE_API_KEY set, every entry point returns undecided or allow and the fork behaves like upstream OpenPoke byte for byte, because the old classifier is kept intact rather than rewritten. Every decision is written to a bounded log in server/data/ with probabilities, verdict and reason but not the email body, so thresholds can be swept offline; JEV_DECISION_LOG=0 turns that off.
One Jev call asks four things about each message before any LLM sees it: does the recipient need it promptly, is it a security code, is it bulk mail, and is the body trying to give instructions to an AI assistant. Confidently unimportant mail is dropped with no LLM call, confidently important mail skips to a summary, and only the uncertain middle near 0.5 pays for the full tool-calling classifier.
43,776 requests, zero failures, on jev-1.13.0 for $2.04. Nothing could force an email to be surfaced: eight mechanisms, 80 templates and 720 eligible trials each, with an exact upper bound of 0.51%. Position mattered, in the opposite order to AgentDojo's finding for tool output: top of body 21.6%, middle 16.8%, end 10.9%.
More Jev resources like this
38GitHub stars
Local unofficial /v1/systemone adapter that serves Choice, Score, and Noul probabilities via first-token LLM judgments or a fine-tuned ModernBERT scorer, including Japanese docs.
Jev Practices & Patterns#community#awesomejev#catalog-import
Community
Argos1111/jev_localPython
Argos1111/jev_localPython24GitHub stars
Unofficial laptop study of parallel constrained decoding: prefill once, share the KV cache across fields, and decide all schema fields in one batched forward pass without writing JSON.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
rorshopping/jev-on-a-laptopPython
rorshopping/jev-on-a-laptopPython22GitHub stars
Offline evaluator that checks whether classifier confidences match reality and converts mistake costs into a deployable threshold, emitting an HTML report plus machine-readable threshold files.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
rlaope/jevalPython
rlaope/jevalPython16GitHub stars
Vendor-neutral harness that tests whether Jev reranking improves a small RAG stack, using frozen hybrid BM25 plus BGE-M3 candidates and isolating retrieval, rerank, and generation stages.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
erendikmenn/jev-rag-benchmarkPython
erendikmenn/jev-rag-benchmarkPython37GitHub stars
Minimal Python recipe that turns any causal LLM into a Jev-style classifier by prefilling choice_index and constraining sampling to legal indexes in one forward pass.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
sgoedecke/system-onePython
sgoedecke/system-onePython39GitHub stars
Open replica of the typed decision API on openly available base models, answering choice, score, and true-probability questions in one forward pass with no decoding.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
mithalouni/system-one-openPython
mithalouni/system-one-openPythonFrom the Jev guides
Original write-ups on what this entry is for.
Most Jev accuracy problems are state problems. What belongs in a state, what is a pre-made judgment to leave out, and why naming the slice you ask about matters more than the wording.
Jev Practices & Patterns#state#design#primitives
Read article
A Noul returns one number and no separate confidence. How to read the value, pin its boundary with criteria, split compound questions, and set a per-action threshold that matches the cost of being wrong.
Jev Practices & Patterns#noul#primitives#confidence
Read article
A Score is a weighted position, not a ruler. Why the number can land between levels, why levels are judged separately, and how to combine several atomic scores in code instead of one crowded rubric.
Jev Practices & Patterns#score#primitives#scoring
Read article
From the Jev community
Posts from builders shipping with Jev right now.