Jev + Gymnasium

A harness for evaluating TypeSafe Jev as a policy in Gymnasium and Atari environments. Game-specific adapters convert observations into structured JSON, and Jev selects a legal action through a single Choice question; it records rewards and decisions without training or updating any weights.

CartPole uses named position, velocity, angle and angular velocity observations; Pong, Breakout and Ms. Pac-Man have RAM-based adapters, and Pong also supports RGB object detection. Montezuma's Revenge is experimental with development paused.

uv with the committed uv.lock and Python 3.14; Atari runs use the Arcade Learning Environment, and no ROM files or credentials ship in the repo. A TYPESAFE_API_KEY is needed for Jev runs, while the seeded random baseline needs no credentials at all.

Set TYPESAFE_API_KEY in a local .env, then run the trial wrapper with a bounded decision limit, for example a breakout trial with --max-steps 64 and a fresh output directory. The wrapper uses RAM observations, one episode, no HTTP retries and exports a GIF automatically.

Model jev-1.13.0, two emulator frames per Atari decision, sticky-action probability 0.25, seed 7, one episode, a 500-decision general CLI limit and 64 decisions per game in the trial wrapper, with two HTTP retries in the general CLI and none in the wrapper.

No. The project evaluates a fixed model: it records rewards and decisions — RAM, frames, requests, responses and rewards to NPY or JSONL, plus GIF renders — and only provides a seeded random policy as a local baseline.

RAM and known room geometry give the adapters privileged information compared with pixel-only Atari benchmarks, and the README documents adapter limitations plus the Montezuma prototype. Existing outputs are never overwritten, so each trial needs a new output directory.