What is JevPokerBench?

ProphetLab's Texas Hold'em benchmark and playground for decision models: watch cash and SNG leaderboards, replay hands, ask the advisor for a decision analysis, or play against models at tables with virtual chips.

How does Jev appear in JevPokerBench?

Jev is the default advisor for decision analysis, and official Jev runs on a separate 128-request pool from the seven local model routes; each pool reserves two slots exclusively for formal benchmarks.

How do I play against the models?

Open the hosted platform, register or sign in, choose Play with models (自己组局), pick cash or SNG, configure the seats and virtual-chip settings, create a table, and press Start game. Registration needs no invitation.

Can I add my own agent?

Yes, through the frontend Add Agent card: choose official Jev or DeepSeek and enter your key, or a custom OpenAI-compatible or Claude agent with its model name, public HTTPS endpoint, and key. No operator API access is needed for this personal setup.

What does it cost to play?

Chips are virtual throughout. Registered players can use the hosted local models and official Jev for free, an invitation adds a lifetime CNY 5 hosted DeepSeek allowance per account, and bring-your-own-key agents are billed by their provider.

Are the shown probabilities win rates?

No. The README says model action probabilities and confidence are not calibrated win probabilities, and all-in equity adjustments cover eligible runout luck rather than complete decision quality or GTO strength.

Who runs it and what are the limits?

It is a ProphetLab project hosted at 123.56.23.73/pokerbench/. Leaderboards and replays are read-only, cash and SNG results remain separate, and a displayed model name alone does not establish the weights or provider behind a service.

Jev directory

CommunityJev Benchmarks & Evaluations

JevPokerBench

ProphetLab's Texas Hold'em benchmark and playground for decision models: separate cash and sit-and-go leaderboards, hand replays, a Jev-backed advisor, and bring-your-own-agent tables, all with virtual chips.

See highlights Open github.com

Stars: 12
Language: Python
License: MIT
Reading time: 2 min
Published by: Community
Author: ProphetLab
Last verified: Oct 2026

prophetlab/jevpokerbench

Organization repository on GitHub

View on GitHub

ProphetLab's Texas Hold'em benchmark and playground for decision models: cash and SNG leaderboards, live replays, and bring-your-own-agent tables.

Forks: 6
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

Tagscommunity catalog-import madewithjev poker benchmark leaderboard

Works forJev for Evaluation Jev for Ranking

Used inJev for Gaming

Languages: Python
Platforms: Browser API
Capabilities: Open source

Highlights

Cash and sit-and-go results stay separate, and model action probabilities and confidence are not calibrated win probabilities.
Seven local model routes (semif, laya, openjev, jeff, nimble, verdict, nanojev) share a 128-request concurrency pool; official Jev has its own.
Registration needs no invitation; an invitation adds a lifetime CNY 5 hosted DeepSeek allowance per account.
BYOK agents need a model name, public HTTPS endpoint, and key; keys use browser sessionStorage and server memory, not server databases or logs.
Human rooms time out model requests after 15 seconds and the seat folds if the action is illegal; a DeepSeek seat retires after three cumulative seat timeouts.

Watch out

Copy reviewed against the repository README on 2026-10-15; it was truncated before the full add-your-own-agent guidance. Chips are virtual, the public interface provides no benchmark or model-registry administration, and the README warns that a displayed model name alone does not establish the weights or provider behind a service.

Questions about this entry

Short answers drawn from the linked project or page.

From the Jev guides

Original write-ups on what this entry is for.

The trust ledger: measuring confidence your domain has earned

Oct 16, 2026

ASSAY-001 calibrated Jev on CLINC150 but found it overconfident on Banking77 — one audit, two verdicts. How calibration research measures the model-and-domain pair, discounts stated confidence, and signs the evidence.

Jev Benchmarks & Evaluations#calibration#evaluation#open-models

Read article

From the Jev AI model community

Posts from builders shipping with Jev right now.

Follow @typesafeai