CommunityJev Demos & Experiments
Tetris JEV Laya ZTC
A static page that races three LLM judging gates — Laya, JEV, and zero-token ZTC — on the same Tetris board with identical pieces, proposals, and noise, publishing paired-match results and the measured constants behind them.
- Format
- Playground
- Published by
- Community
- Author
- FINAL-Bench
- Last verified
- Oct 2026
- Platforms
- Browser
- Capabilities
- Open sourceRuns locally
Highlights
- Races Laya, JEV, and zero-token ZTC gates on one Tetris board with identical piece order, proposal, and noise across 1,000 paired matches.
- Reported lines-cleared gaps (95% interval): ZTC-JEV -0.07 [-0.14, +0.01], a tie; ZTC-Laya +1.03 [0.79, 1.26]; ZTC-no gate +1.37 [1.16, 1.59].
- On 1,939 Claude Haiku 4.5 answers the card reports ZTC-Judge-27B v2 at 0.7752 against JEV's 0.7521.
- Measured constants: JEV AUC 0.7350 vs ZTC 0.7289 on 2,018 questions; gate latency JEV 0.591 s vs ZTC 0.636 s; calibration error ZTC 0.0245.
- Ships in Korean, English, and Chinese, follows your browser language, and reads local model state, so it can run in an air-gapped network.
Watch out
A static, self-published comparison: the 2026-09-21 corrected set, the leave-one-domain-out protocol, and the wired constants come from the page itself rather than an independent rerun. JEV is billed at 20 output tokens per decision, and Laya's root checkpoint scores AUC 0.4904 on the corrected set, where the page says its gate reduces to always re-rolling.
Questions about this entry
Short answers drawn from the linked project or page.
A generation-free way to judge an answer: read the model's own internal representation and score every candidate in one forward pass, writing zero output tokens, so it cannot name an option that does not exist.
Over 1,000 paired matches the reported gaps in lines cleared are ZTC minus JEV -0.07, a tie; ZTC minus Laya +1.03; and ZTC minus no gate +1.37.
On the corrected set the root checkpoint scores AUC 0.4904, so the page says its gate reduces to always re-rolling; Laya-Multilingual's calibration error is 0.4985.
The page contrasts a 0.033 s readout against 3.206 s for a generative judge on the same 4,444-token, 200-candidate screen, and JEV is billed at 20 output tokens per decision.
No. The single-retry measurement says re-rolling a bad move helps 38% of the time, while re-rolling a good move breaks it 30% of the time, so a gate is worth what it leaves alone.
Yes, by construction: reading internal state requires local weights, so it cannot be offered as a hosted API against someone else's model, unlike the JEV API path.
Darwin-397B-ZTC ships a self-readout probe, and ZTC-Judge-27B, ZTC-Judge-9B, and ZTC-Judge-4B judge other models' answers; this page races the judging gates on Tetris.
Related terms
Glossary definitions related to this entry.
More Jev resources like this
An interactive ZeroGPU demo of AutoTrust's JEV-9B: System 1 answers noul, choice, and score questions in one prefill pass with calibrated probabilities, while an Escalate tab hands low-confidence cases to the untouched Qwen3.5-9B base.
Jev Demos & ExperimentsPlayground#community#playground#demo
Community
AutoTrust's live game-arena demo: watch JEV-27B play Mario, a 3D Rubik's Cube, and Tetris while a side panel reports action probabilities and per-decision inference time, alongside a decision lab and a ViZDoom arena.
Jev Demos & ExperimentsPlayground#community#playground#demo
Community
A ZeroGPU demo of AutoTrust's JEV-27B-VL: System 1 returns calibrated noul, choice, and score probabilities from one prefill pass over text and images, with System 2 on the untouched Qwen3.8-27B for escalation.
Jev Demos & ExperimentsPlayground#community#playground#demo
Community
A three-lane demo of pngwn's System One 4B scorer: one prefill plus a batched forward for every option branch, a naive per-option equivalence check, and a Qwen3.5-4B instruct baseline generating JSON.
Jev Demos & ExperimentsPlayground#community#playground#demo
Community
A Gradio playground for Jev-Omni: give a situation plus a question with options and get one probability per option from a single forward pass, with no generated text, from a Gemma 4 12B IT fine-tune.
Jev Demos & ExperimentsPlayground#community#playground#demo
Community
Benchmark Heaven's static snapshot of the JevBench v1.6.1 decision-model board: capability and composite scoring over typed-decision accuracy, calibration, latency, and cost, readable without JavaScript.
Jev Demos & ExperimentsPlayground#community#playground#demo
Community
From the Jev guides
Original write-ups on what this entry is for.
More than eighty preprints now evaluate, attack, and reproduce the typed-decision class. The broadest board, the coherence gap, the rejection bottleneck, and the attacks that still work.
Jev Practices & Patterns#research#evaluation#calibration
Read article
Base decision checkpoints sit near chance on purpose. The typed-decisions loop, the corpora behind the students, and the temperature fit that turns probabilities into thresholds.
Jev Practices & Patterns#fine-tuning#calibration#open-models
Read article
TypeSafe publishes evals, but the numbers that matter are the ones measured on your task. A tour of the independent benchmarks, field reports, and the recurring finding that question design dominates.
Jev Practices & Patterns#benchmarks#evaluation#calibration
Read article
From the Jev community
Posts from builders shipping with Jev right now.