Does winning the vote mean the answer was right?

No. Models, speed, and cost stay hidden until you vote, and your vote records preference, not verified correctness. It is a blind comparison, not a correctness proof.

Do I hand my API key to the site?

Your key stays in the current tab memory and vanishes on refresh, with browser-to-OpenRouter calls direct. Use a dedicated key with a small provider-side limit or run locally.

What gets sent where when I run a battle?

Only to the selected providers when you start a run. Public contribution is a separate visible opt-out choice, so do not submit secrets or private conversations.

Are the JevJudge-Bench scores official leaderboard results?

The Python package converts pinned JudgeBench, RM-Bench, and RewardBench 2 data into one diagnostic pairwise protocol. Scores are not the upstream leaderboards, and mock output is marked.

How do I run the site myself, and what does it use in the browser?

cd web, npm ci and npm run dev. The checks are npm run typecheck, npm test, npm run build and CI=true npm run test:e2e. Public live comparisons go through OpenRouter, while Vercel AI Gateway is available for local use and stays disabled on the hosted site pending its production relay gate. The browser tests use synthetic responses and never spend model credits, so the e2e suite is safe to run in CI.

How do I reproduce the JevJudge-Bench run myself?

The Python package is the research harness. uv sync, then uv run jevjudge fetch, then prepare with --groups-per-domain 2 --partition test --out data/pilot.jsonl, then run with --data data/pilot.jsonl --config configs/mock.json --out runs/smoke, then report runs/smoke. Mock output is marked MOCK / PIPELINE TEST so it can never be mistaken for a real result. Read PROTOCOL.md before making model claims and RESEARCH_PLAN.md before paid runs; actual runs require an explicit call and cost limit, and the runner has no hidden retries.

Who runs it, and what licence covers the cases?

Maintained by Richard Tang, and independent: not affiliated with Arena, TypeSafe, or the model providers. The code is Apache-2.0, original case contributions are CC BY 4.0, and third-party material keeps its original licence and attribution. Production changes stay migration-led, and the project-scoped Supabase MCP in .mcp.json is read-only. Provider behaviour, release gates and data handling live in docs/ rather than in the product introduction.

CommunityJev Practices & Patterns

Jevarena — chenmingtang830

Open arena pitting Jev against a chosen opponent on your question, hiding identities until you vote, then revealing latency, cost, and confidence data.

See highlights

Open github.com

Stars: 5
Language: TypeScript
License: APACHE-2.0
Published by: Community
Author: chenmingtang830
Last verified: Oct 2026

chenmingtang830/jevarena

User repository on GitHub

View on GitHub

Open-source BYOK arena for Jev and other AI judges. Find failures, compare quality, cost, and latency.

Forks: 0
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.Homepage

About this project

JevArena is an open-source arena for testing Jev against another judge. Ask a question, choose possible answers and an opponent, then vote before the models, speed, and cost are revealed. Open JevArena · Try examples · Browse community questions · Submit a question. Ask — enter a question and its possible answers, or start from an example. Choose — connect OpenRouter and select any supported model to face Jev.

Quoted from the project's README. Read the full README.

Tagscommunity awesomejev catalog-import

Works forJev for Classification Jev for Evaluation

Used inJev for Software & DevTools

Languages: TypeScript

Highlights

Every live battle includes Jev against any supported OpenRouter opponent model.
Ask with custom answers or start from examples, then submit questions to the community.
Browser tests use synthetic responses and never spend model credits.
Code is Apache-2.0 with migration-led production changes and read-only tooling.

Watch out

Votes record preference between hidden decisions, not verified correctness, and community questions stay unverified until independently reviewed. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

Other listings of this project

Forks and copies of this project in the directory. This entry is the canonical one.

jevarena

Next.js spectator demo in which two independent Jev agents drive separate Snake browsers through same-origin iframes, clicking indexed direction buttons only, with each board state streamed to a server-only endpoint that returns the chosen target.

Jev Demos & Experiments#community#jevbest#catalog-import

Community

raihankhan-rk/jevarenaTypeScript

JevArena

A Next.js spectator app where two independent Jev agents play Snake against each other, each spectator page running its own match. Jev is reached only through Vercel AI Gateway, and an empty key gives you a labelled demo-policy match instead.

Jev Demos & Experiments#community#jevbest#catalog-import

Community

rolki-png/JevArenaTypeScript

More Jev resources like this

jev-lm

Word-level language model experiment that uses Jev as its output layer, with explicit tokenizer, sampler, repetition mask, stop rules, and chunk verification.

Jev Practices & Patterns#community#jevbest#catalog-import

Community

y0usaf/jev-lmTypeScript

jev-chess

TypeScript chess library where Jev resolves natural-language move intent against verified legal moves, scores sharpness, strategic theme and king risk in one call, and drives persona opponents.