Jev Reward Model Evaluation

RewardBench v1 at 92.58% and RewardBench 2 at 81.15% are close to leaders, with safety above 95% and strong aggregate ranking correlation. Precise instruction following at 50.63% and Olympiad-level verification lag instead.

It nearly matches Athene-RM-8B per-example but beats it on aggregate ranking correlation, so the digest positions it for large-sample model ranking rather than treating each label as ground truth.

Yes for reranking, Best-of-N, A/B ranking, safety review, and rubric-guided eval where rubrics exist. Add deterministic checks for exact following and code, plus calibration, tie policy, and private holdouts.

Summary, baseline, capability, and 54-row comparison CSVs plus the interactive report with difficulty curves and heatmaps. The 76M input tokens ran with zero API errors at about $3.19 input cost.

Eight tracks across seven benchmark families, 40,940 successful records, 76.02M input tokens and an estimated $3.1927 at $0.042 per million input tokens, excluding any unreported output or platform charges. Observed API error count is zero and every completed record returned model version jev-1.13.0.

Scoring every response independently before rebuilding the comparison matrix lifts RM-Bench from 81.29% to 83.79%, a 2.50 point gain that the study reads as reduced within-call coupling or ordering effects. Pointwise also lifts Code from 67.64% to 75.93% while keeping Safety above 93%, so the two protocols are scored separately rather than one replacing the other.

PRMBench scores 66.38%, within 0.42 points of GPT-4o, but the split matters: correct-step recall is 80.12% while wrong-step recall is only 63.46%, meaning it tends to accept flawed reasoning steps. ProcessBench tells the same story from the other side, falling from 74.63% on GSM8K to 66.01% on OlympiadBench.