AstroHan's Jev Harness

A minimal coding agent that filters every tool result through a small relevance model, Jev, before the main model sees it, plus an A/B harness that runs the same tasks with filtering on and off. The question it asked: can a clean, filtered context plus a cheap model match a conventional harness at lower cost?

The TB4 follow-up, September 22, 2026, closed the line: on 49 scored pairs, Jev on passed 2 tasks versus 5 for off, used 11.78% more cumulative input tokens and incurred 8.17% more estimated model cost; mean per-task test pass fractions were 44.27% versus 48.04%. The README states the experiment did not demonstrate enough value to justify default adoption.

With deepseek-flash as the main model: 25/30 pass@1 with filtering versus 22/30 without, $2.82 versus $2.80 total ($0.113 vs $0.127 per passing task). The gap was entirely on DeepSWE (6/9 vs 3/9); Terminal-Bench tied at 19/21, and at test granularity the filtered arm was never worse: 6 wins, 0 losses, 3 ties (sign test p≈0.031).

Every tool result except `read` is split into roughly 2k-character chunks on line boundaries; each chunk gets one boolean question about whether it contains information the agent needs this step; chunks are kept at p > 0.5. Dropped runs collapse into a visible elision marker naming the full output's path.

No — the raw output is always stored at .jev-store/<call_id>.txt and the `read` tool can retrieve it. The README describes filtering as a routing decision, never destruction, and Jev failures fail open with the unfiltered output returned and the error logged.

The README is careful: it does not establish that Jev is generally ineffective or that filtering caused the success-rate difference. It does establish that this implementation has not demonstrated an end-to-end benefit sufficient to justify its complexity; trajectory inspection found unbounded-output failures, environment limitations and divergent execution paths.

RESULTS.md has per-task numbers and caveats, TB4-CONCLUSION-2026-09-22.md has the TB4 report and trajectory analysis, and tb4-results-2026-09-22.json is the per-task metric snapshot. The experiment VM was deleted and automatic reruns disabled; earlier numbers are kept as history and not pooled with TB4.