Question 1

Is the calibration score real, or is that number still missing?

Accepted Answer

Deliberately missing. The endpoint's free tier serves about five calls per window, so the longest labelled run so far is n=5 turns with a confidence interval covering everything, including the constant predictor. The README says publishing a Brier there would be decoration, and the measure command prints whatever the run actually supports, a bad number included, and says when there is nothing to say.

Question 2

Is the model playing the game, or following a hardcoded route?

Accepted Answer

Mostly the route. It cannot plan and never sees more than the current decision. Every route, threshold and piece of arithmetic is ordinary Python, and the model only picks among actions the code already proved legal. It is also not asked often: code handles routine ticks and calls Jev only where the game branches, so between the bedroom and Oak's lab it is not asked anything at all.

Question 3

Will it get past Route 1, or does it stop at a ledge?

Accepted Answer

It stops at a ledge. v0.1 targets Viridian City on a hardcoded route: Pallet Town's exit is measured and works, but Route 1 still walks into a ledge at (10, 28) and stops, because one waypoint per map cannot express a way around it. Without Jev the code default loses the rival battle, so win_lab_rival never completes and nothing downstream of it runs.

Question 4

Do I have to supply the ROM myself?

Accepted Answer

Yes. You pass your own Pokemon Red dump through POKEMON_ROM, and the repo contains no game data and will not help you find any. Save states hold copyrighted memory and stay out of git. Tests run offline against fixtures with no key or network, JEV_BASE_URL can point at a gateway shim, a recording proxy or a local fake, and the ROM smoke test only runs locally.

Question 5

How fast is it, and what does a decision actually cost?

Accepted Answer

Measured over the six calls that were answered on a cartridge, 479 to 1,470 ms, median 621 and mean 759, so about 1.3 decisions per second and $0.14 an hour at roughly 725 input tokens a call, all through a Vercel AI Gateway shim rather than the direct API. That is call latency only: playback pacing and rate-limit retries are kept out of it.

Question 6

How do I know the memory addresses it reads are correct?

Accepted Answer

The addresses come from walking pret/pokered's ram/wram.asm from the WRAM0 origin and summing the struct macros, not from copying hex, and nine of them match independently published values byte for byte. The synthetic RAM fixtures are too tidy to catch a wrong address, so the ROM smoke test is what does that, and it only runs locally against a cartridge.

Question 7

Can I see the actual request it sends without a ROM?

Accepted Answer

Yes. jpp state --ram fixtures/ram_battle.bin prints the exact request body, jpp overlay --replay fixtures/runs/sample.jsonl renders the overlay from a recorded run with no ROM, and jpp probe walks the route by hand while printing the decoded state. One caveat: sample.jsonl is a fixture, with 20 of its 40 rows real answers and the rest tagged source fake, which measure excludes.

Jev Plays Pokemon Red

About this project

Highlights

Quickstart

Questions about this entry

Reactions & coverage

From the Jev guides

From the Jev AI model community