User repository on GitHub
Challenge Atari with Jev: structured decisions, value questions, and replayable experiments
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Demos & Experiments
Jev Atari Lab
A research repository that asks how a teacher should rewrite the structured questions controlling an Atari player, publishing the negative results alongside the one that worked. Jev weights stay fixed throughout.
- Stars
- 0
- Language
- Python
- License
- GPL-2.0
- Published by
- Community
- Author
- memorysaver
- Last verified
- Oct 2026
About this project
How should a teacher learn to improve the structured questions that control an Atari player? We have two goals: work toward mastering every single-agent Atari game exposed by the pinned Arcade Learning Environment with TypeSafe Jev, and discover effective teacher-driven optimizers for structured question policies. We publish the evidence behind each attempt: observations, model inputs and outputs, actions, rewards, videos, and every teacher-authored change to the question program.
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
- Languages
- Python
Highlights
- Direct policy and value-based tracks are kept separate; the videos use direct policy, and action probabilities are explicitly not Q values.
- The Seaquest ten-round study is the one that worked: development mean 80 to 370 and final mean 80 to 290 with no seed regression.
- Every run's observations, model inputs and outputs, actions, rewards, videos and each teacher-authored question edit are published.
- The pinned Arcade Learning Environment registers 104 discrete single-agent ALE/*-v5 games, all through a local boot and action check.
Watch out
The README states plainly that no repeatable optimization benefit has been established and that the first automated teacher study stopped in round two, so most of the content here is failed experiments. The Freeway ten-round study found no revision improved held-out return — generic and selected Jev both scored 23 out of 26 on final seeds while predictive local controls scored 29 out of 29 — and the Seaquest execution follow-up saw the selected wording revision regress macro agreement from 68.75% to 60.94% and mean return from 300 to 240. An earlier Seaquest pilot produced 1,108 decisions that all fit DOWN when objects were active and NOOP otherwise. Two studies were cut off by provider errors rather than finishing: HTTP 402 stopped a criteria-edit study after 15,093 attempts, and HTTP 502 stopped a Pong study in round two. Reported costs are small, around a dollar to low single digits per study, and isolated teacher calls were zero in several of them because the teacher is the coordinator rather than a model. Setup needs Python 3.12, uv, and Git LFS for videos and archives; ROMs and credentials are never committed, and the 128 raw RAM bytes the runner exposes are not decoded objects. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
One narrow result, and the README is careful about the framing. In the Seaquest ten-round study a training-selected question passed both held-out gates, taking development mean from 80 to 370 and final mean from 80 to 290 with no seed regression. Everywhere else it reports no benefit established, and it explicitly makes no mastery or causal feedback-benefit claim.
They are recordings of the direct policy selecting the highest-probability action from Jev, and the README pre-empts the obvious misreading twice: it says the videos do not show a strategy discovered from scratch, and that action probabilities are not Q values. So a video is evidence that a question produced those actions, not evidence that the model learned the game.
Because the stated goal is finding repeatable optimization patterns under a measured budget, and most candidates did not meet it. Freeway: no revision improved held-out return, with local controls outscoring Jev. Seaquest execution: the selected wording regressed. Pong: an HTTP 502 in round two, and an earlier HTTP 402 after 15,093 attempts. Only the Seaquest ten-round study cleared its gates.
The README reports per-study API costs in the low single-digit dollar range — US$0.979 for the Freeway ten-round study, US$0.573 for the Seaquest execution follow-up, US$2.24 for the Seaquest ten-round study, US$0.119 for the first Seaquest pilot. Expect the mechanical overhead to dominate: Python 3.12, uv with a locked dependency set, Git LFS pulls for videos and archives, and long-running episodes.
Clone the repository, run git lfs install --local and git lfs pull for videos and archives, then uv sync --locked --dev. Python 3.12 is required, dependencies are pinned in uv.lock, and ROMs and credentials are never committed.
The pinned Arcade Learning Environment registers 104 discrete single-agent ALE/*-v5 games, all through a local boot and action smoke check. uv run jev-atari games discovers them with no API calls, and games --check exercises every registered game rather than acting as a gameplay benchmark. All means that versioned scope, not every cartridge or mode.
The direct policy asks one Choice question over available actions and picks the highest probability, and the videos use this track. The value-based track asks each action's first scoring outcome within 240 raw frames and ranks by P(gain) minus P(loss) with a fixed heuristic continuation. Action probabilities are not Q values.
More Jev resources like this
46GitHub stars
Open 150M encoder that reproduces the typed noul, choice, and score interface in one forward pass, trainable on a free GPU and runnable locally for nothing.
Jev Demos & Experiments#community#jevbest#catalog-import
941GitHub stars
Library that adds schema-guaranteed typed outputs to existing autoregressive LLMs without changing weights, served through SGLang or a hosted API.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
TypeLLM/TypeLLMPython
TypeLLM/TypeLLMPython45GitHub stars
Drop-in service that exposes Upstage Solar models through the System One shape, accepting state plus noul, choice, or score questions.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
hunkim/solar-mini4-jevPython
hunkim/solar-mini4-jevPython38GitHub stars
Research recipe that fuses two frozen open readers into Jev-style yes/no judgments with a fit-free router, confidence gate, and candidate sets.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
FeiLiuEM/open-medical-jevPython
FeiLiuEM/open-medical-jevPython62GitHub stars
Simulation study that drives one xArm7 apple-to-plate task with Jev, a large reasoning model, and a small model through identical incremental controls.
Jev Demos & Experiments#community#jevbest#catalog-import
40GitHub stars
Open research toolkit that encodes an image once and answers multiple typed choice, noul, and score questions from a shared distribution, plus a local 0.6B text-decision model.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
IamBusy/OpenJev-VisionPython
IamBusy/OpenJev-VisionPythonFrom the Jev guides
Original write-ups on what this entry is for.
A game tick is a state plus a legal action list. How Clash Royale, Minecraft, and the earlier Mario, Pokémon, StarCraft, and drone loops ask Jev one step at a time.
Jev Demos & Experiments#gaming#agents#control
Read article
From the Jev community
Posts from builders shipping with Jev right now.