Jev Arena

No. DeepSeek Flash leads on accuracy in that run: all three labels were correct together for 62.69% of Jev rows against 67.26%, with DeepSeek also ahead on relevance, sentiment, and intent individually. Where Jev wins is time and money, at 203.2 seconds and about $0.84 for 10,000 reviews versus 823.5 seconds and an estimated $1.50.

It is AI-reviewed rather than a human gold standard, and the README states that outright. The review model looked only at the original text and title, with 400 rows re-labelled as a sample, and under a strict rule that accepts only the first choice the all-three-correct rates fall to 50.58% for Jev and 55.45% for DeepSeek.

Not for a duel. A run needs an OpenRouter key and a DeepSeek key, and the default pairing is Jev against DeepSeek, though the other side can be pointed at any OpenAI-compatible chat model. Node.js 22 or newer is required, and Docker is the alternative path.

No. Results are saved under runs/<run-id>/ and replay never calls a model, and the HTML report is generated offline with no keys at all, defaulting to a factual draft. Uploads above 30 rows prompt for confirmation first, and the default dataset is 20 synthetic comments.

Intent, in that run. Across the 10,000 comments the figures were relevance 94.70% against 96.26%, sentiment 82.91% against 84.54%, and intent 77.90% against 80.33% for Jev against DeepSeek Flash. That ordering holds on both sides, and the gap is widest in the aggregate, where 62.69% of Jev rows had all three correct against 67.26% for DeepSeek.

MIT covers the code. The demo comments are explicitly not part of it: they belong to their original authors and platforms and sit outside the license scope, so a copy of examples/comments.csv or examples/comments.xlsx is not yours to redistribute even though the harness around it is.

docs/CONTRACT.md holds the label contract, docs/usage.md covers configuration, the import format and how cost is accounted, and docs/report-generation.md covers agent-written analysis. The published accuracy run ships as a report plus a per-row check CSV under audit/accuracy-0919-124001/, and examples/demo/README.md carries the raw data with both sides' labels, so every percentage can be traced back to a row.