User repository on GitHub
LLM-authored task-specific Jev harnesses with optional full-trajectory reward reflection and GEPA evolution.
- Forks
- 21
- Last pushed
- Updated Oct 2026
Repo stats from the GitHub API, cached Oct 2026.Homepage
Jev directory
CommunityJev Tools & CLIs
JevHarness
A Python framework where an authoring LLM writes a task-specific Jev harness, then optionally evolves it using reward reflection and GEPA over full trajectories.
- Stars
- 546
- Language
- Python
- Reading time
- 3 min
- Published by
- Community
- Author
- TianyuCodings
- Last verified
- Sep 2026
About this project
Let an LLM write a task-specific harness for Jev. Run it, inspect its decisions, and optionally improve it using rewards and complete execution traces. A harness turns task observations into useful features, constructs Jev questions and criteria, and combines the structured answers into actions. The authoring LLM can change the code, questions, graph, and memory. Once the harness is fixed, execution uses that code and its Jev calls; it does not need the authoring LLM on every decision.
Quoted from the project's README. Read the full README.
Used inJev for Gaming
Highlights
- On a Pokemon task the selection eval win rate improved from 25% to 75% after five reflection rounds.
- The selected harness averaged 568 ms per full decision and 269 ms per Jev request across 12 games.
- Supports choice, score, and noul questions with multiple questions per request and concurrent graph nodes.
- Optional GEPA integration selects parents from an instance frontier and records rejected proposals' ancestry.
Quickstart
bash
git clone https://github.com/TianyuCodings/JevHarness.git
cd JevHarness
node website/build.mjs
node website/preview.mjs --port 8768Watch out
Needs an AI Gateway or TypeSafe key, Python 3.11 or newer, and a macOS native sandbox for functional Python nodes. The reported gain is on the selection eval set, not unseen games, and the repo ships no license.
Questions about this entry
Short answers drawn from the linked project or page.
Jev decides at runtime. An authoring LLM writes the harness during development: the Python and expression code, the Jev questions, the graph, and the memory. Once the harness is frozen, that code and its Jev calls execute without the authoring LLM on every decision, and the frozen artifact still needs Jev whenever it contains Jev nodes.
It is a real measurement on one small set, and the README says so: 12 Eval games per harness, 3/12 to 9/12 after five reflection rounds, with the search itself retaining the round-3 candidate as best using that same Eval set. It is not an estimate on unseen games.
No. They are recorded successful timings from 12 Eval games, with every included Jev call explicitly marked as missing the local response cache, and the README calls them non-controlled observations. It states the experiment did not benchmark an LLM making every runtime decision, so do not read it as a head-to-head.
That boundary is the design. The task adapter owns observations, legal actions, side effects, and scoring; the harness owns feature construction, Jev judgments, and decision logic. The README presents this split as what makes it possible to change the harness without letting it rewrite its own reward or read hidden state.
Clone the repo, run node website/build.mjs, then node website/preview.mjs --port 8768 and open localhost:8768. The archived Pokemon run, the selected harness, the browsable evolution tree including rejected proposals, and complete decision records with their Jev answers are all there; browsing the archive makes no model requests and starts no games.
It selects parents from an instance frontier, compares each parent and proposal on the same training batch, and fully evaluates accepted proposals on Eval, recording real ancestry including rejected proposals. Reflection is optional: a harness can be authored, run, and frozen with no search at all. The whole loop needs an AI Gateway or TypeSafe key, Python 3.11 or newer, and a macOS native sandbox for functional Python nodes.
The harness writes the Jev input itself: a task-specific state, named questions, an answer type, and the instructions and criteria used to judge the legal actions. choice, score, and noul are supported, several questions can share one request, and independent graph nodes run concurrently. The probabilities coming back are action-choice probabilities, not calibrated confidence that the choice is right.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Decision outputs in Grep
Hacker News: jevmem keeps Claude Code project memory by asking Jev which turns are worth saving.
avinashjetwani61 pointsJevmem – automatic project memory for Claude Code, built on JevRead the thread on Hacker NewsX: A usage router for Grok Bot
X: The compaction plugin, in Chinese
More Jev resources like this
A collection of 26 production-ready agent skills for Claude Code, Cursor, Kiro, Windsurf, and OpenCode; four skills call TypeSafe Jev for calibrated Score and Noul judgments and fall back to heuristics when it is unavailable.
Jev Tools & CLIs#community#python#skills
Community
fabricioctelles/skillsPython
fabricioctelles/skillsPythonA collection of Jev demos, workflows, and agent skills for coding agents: five installable skills and 108 scenarios, installed by pointing Codex, Claude Code, or OpenCode at an agent prompt.
Jev Tools & CLIs#community#python#skills
Community
wuyoscar/jev-skillPython
wuyoscar/jev-skillPythonA Hermes skill that calls Jev as a typed tiebreaker: the agent submits a state and a question, and Choice, Score, or Noul answers with probabilities let it act or escalate.
Jev Tools & CLIs#community#python#skills
Community
shantanugoel/ask-jev-skillPython
shantanugoel/ask-jev-skillPythonA Stop hook that keeps coding agents from quitting early: rules written in plain language are scored by Jev in about 0.7 seconds, and a violated rule sends the agent back to work instead of letting it stop.
Jev Tools & CLIs#community#python#claude-code
Community
noplan-inc/limpetPython
noplan-inc/limpetPythonA code reviewer built as AI-powered software rather than an LLM agent: code slices the diff, Jev answers narrow typed questions per hunk, and code composes the verdict and findings.
Jev Tools & CLIs#community#python#coding-agent
Community
rbalch/typesafeai-reviewPython
rbalch/typesafeai-reviewPythonAn experimental Sutro CLI that evaluates CSV, Parquet, and JSONL data with Jev, asks you to label ambiguous and random audit rows, and uses GEPA to propose improved function definitions.
Jev Tools & CLIs#community#python#cli
Community
sutro-sh/jev-alignPython
sutro-sh/jev-alignPythonFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.