Jev Benchmark — YidiDev

The headline core-classification table reports Jev 98.84% against Haiku 96.46%, Sonnet 96.18% and OpenJev 96.28%, with per-1k-call prices from $0.031 to $3.599. Advantages concentrate where batching and small-step execution favor Jev.

Part 1 uses an adversarial shuffle control to separate rubric conditioning from label-text matching, with NLI and embedding baselines included specifically to prove the difference.

razorback16/openjev speaks the exact Jev wire API and ran at $0.00 real spend via a free hosted tier, with a separate self-hosted rented-GPU estimate so it is not compared as if free to deploy.

CT9 walks a 10-layer decision tree in 1/2/5/10-step chunks and CT10 grades partial-credit AP World History answers. Sonnet led narrow single-step tasks but trailed most on broad batched versions.

Eight classification suites, from simple type sort through thresholds, lookup exceptions, negation, arithmetic without a running total, date comparison, chained lookups and a long-context distractor, plus a chained decision-tree walk in one, two, five and ten-step chunks, and a partial-credit grading suite over AP World History answers.

The core suites used $110.11 of metered Anthropic spend, with separate GPU rental of about $1.03 and $2.11 for the later open-weight parts, across 152 passing tests. The per-model spend table puts Jev at $0.75, Haiku at $29.67, Sonnet at $68.10 and OpenJev at $0.00 real spend on a free hosted tier.

Every arm saw the same rubric on the same grid, and the Claude models were driven through forced tool-use, so the comparison rests on identical inputs rather than each model's preferred prompt shape. The methodology notes also document where an initial approach was found wrong and replaced.