Does Stockfish leak into the model's state and make the numbers flattering?

Only through the opt-in lesson levels, which the README calls one deliberate exception. Otherwise its evaluations never reach Jev's state, and the lessons toggle sits at 0 by default. Everything else is computed after the pick: the decision panel shows Jev's probability for each move next to Stockfish's evaluation and the centipawn loss, and the blue arrow marks Stockfish's best.

Are the Elo numbers comparable to real chess ratings?

No. They come from a calibration run of Stockfish against itself, kept in bench/elo-calibration.json, not from FIDE or lichess. Any setting that calibration does not cover is displayed as unrated, and its games are left out of performance Elo entirely — so a game count in the dashboard is not the same thing as a rating you could quote.

What does a full run cost, and can I look around without an API key?

Without a key the app starts in mock mode with a fake Jev behind an amber MOCK badge, which is enough to walk through the interface but the answers are meaningless. Live decisions are quoted at about $0.00007 each, and the full M5 bench run of roughly 5,000–6,000 decisions is well under a dollar. npm run bench -- --mock is the offline smoke test.

Can I compare Jev's prompting variants on the same position?

Yes. Compare setups asks all four setups about the current position and grades them together, and Shadow-run the other setups does that on every Jev turn while a game runs. The variants are raw versus assisted facts, choice versus noul framing, foresight 0–3, lessons 0–2, and taking the top move versus sampling from the distribution.

How do I give the app my API key?

Either write it to a gitignored .typesafe-api-key file in the repo root (the README shows a hidden prompt that avoids shell history) or set TYPESAFE_API_KEY, which wins over the file; TYPESAFE_MOCK=1 forces mock mode even with a key.

What is the Stockfish ladder?

After each Jev-versus-Stockfish game the engine goes up if Jev won and down if Jev lost, moving one setting of the mode you chose (Elo 1320-3190, or skill level 0-20) without switching difficulty modes, so games cluster where the result is informative.

Can I ask Jev about a position without the UI?

Yes. node scripts/ask.js --fen " " --setups assisted-choice,assisted-noul prints each setup's distribution, with --history for prior moves and --json for machine-readable output; it runs live when a key exists.

CommunityJev Practices & Patterns

TypeSafe Chess — Dimesio

A local chess workbench for measuring Jev as a move chooser: the code enumerates legal moves with chess.js, Jev picks from them, and Stockfish grades every pick without its evaluations reaching the model.

See highlights

Open github.com

Stars: 2
Language: JavaScript
Published by: Community
Author: Dimesio
Last verified: Oct 2026

Dimesio/typesafe-chess

User repository on GitHub

View on GitHub

FUn little experiment with Typesafe AI Jev Model playing chess against stockfish :)

Forks: 0
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

A local app for testing TypeSafe Jev (a System One model) as a chess decision-maker. Jev never generates a move: the code lists the legal moves, Jev picks from them, and Stockfish grades every pick. The point is to find out how well Jev chooses, so nothing here is tuned to make it look good or bad. chess.js owns the rules. Jev only chooses. Stockfish only grades. Its evaluations never reach Jev's state, with one deliberate exception: the opt-in lesson levels, which feed Jev's own graded mistakes back to it.

Quoted from the project's README. Read the full README.

Tagscommunity awesomejev catalog-import

Works forJev for Classification

Used inJev for Software & DevTools

Languages: JavaScript

Highlights

Stockfish grades but never informs Jev, apart from the opt-in lesson levels that feed Jev's own graded mistakes back to it.
Raw or assisted facts, choice or noul framing, foresight 0–3 and lessons 0–2 are switchable, and setups can be compared on one position.
A Stockfish ladder adjusts engine strength after each game, moving one setting and refusing to switch difficulty modes.
Runs without a key in mock mode behind an amber badge, and a failed live call surfaces the error instead of quietly falling back.

Watch out

The app is explicit that Jev never generates a move, so this measures choice quality inside a small legal-move set rather than chess strength. Answers vary between identical requests — the Ask again button exists to show it — so one position is not a stable measurement and the intended unit is a run. Estimated Elo comes from a Stockfish-versus-Stockfish calibration run stored in bench/elo-calibration.json, not FIDE or lichess, and settings the calibration does not cover are shown as unrated and excluded from performance Elo. The lessons levels are the single place Stockfish output reaches Jev, a deliberate exception to the separation. Re-calibrating takes about an hour on 10 workers, and the full bench run is 5,000–6,000 decisions. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

Other listings of this project

Forks and copies of this project in the directory. This entry is the canonical one.

typesafe-chess

Two Jev players playing chess with no engine: code owns the rules and the tactical facts, and the model returns one Choice per legal move with a probability for each, plus a Score for the position and a Noul for sharpness.

Jev Demos & Experiments#community#jevbest#catalog-import

Community

TholeG/typesafe-chessJavaScript

More Jev resources like this

jev-column-race

Live race rig that labels 1,000 real app reviews with Jev typed questions against Gemini JSON output on identical batches, then replays recordings and re-ranks rows locally without extra model calls.

Jev Practices & Patterns#community#jevbest#catalog-import

Community

goodrahstar/jev-column-raceJavaScript

hermes-and-jev-play-minecraft

Minecraft setup where Hermes plans at milestones, Jev picks one bounded harness action per step, and Mineflayer code owns validity and execution with no screenshots.