Skip to content

Jev Demos & Experiments

Gaming: per-tick decisions, NPC behaviour, and matchmaking

Per-tick decisions, NPC behaviour, and matchmaking run at a different clock. What the game demos show about latency and action gating.
Published
Category
Jev Demos & Experiments
Resources cited
9
Reading time
6 min
Tags#gaming#demos#latency#agents

Games are the natural home of the per-tick decision. The state is already structured, the legal actions are already enumerated, and the loop already has a latency budget. What games also have is a brutal feedback signal: a policy that looks clever in a demo can lose to a three-line rule.

Per-tick does not mean per-frame

Jev-Chess 1 Small is a 6.5M-parameter model that predicts the next best move with one parallel forward pass. The board state goes in, and a calibrated distribution over legal moves comes out. It was trained for 30,000 steps on 5,000,000 Lichess positions labeled with Stockfish's best move, and it reports a held-out top-1 of 36.5%, calibration ECE of 0.036 and an Elo of 1239 plus or minus 51 over 200 games against Stockfish UCI_Elo 1320 and 1400.

Those numbers are self-reported, and the model is an independent reimplementation from Jev's public description, not TypeSafe's weights. It is a pure policy network with no search. The useful lesson is the scale: a 6.5M-parameter model can return a distribution over up to 218 legal moves in one pass, fast enough to sit inside a chess loop. A game does not need a per-frame model; it needs a decision at the cadence the game can absorb. The per-tick decision glossary entry describes the same contract.

sys1npc is a mechanics-free decision core for game NPCs. The boundary is battle.action: the adapter supplies an observation plus a complete, opaque set of legal actions, and a provider picks exactly one. The shared coordinator validates membership and correlation and bounds how long a provider may take. It knows no Pokémon mechanics, never reads the observation and never decodes an action id.

That is the correct division of responsibility. The game owns legality and state decoding; the model owns the choice. The project ships a deterministic offline provider and a Jev provider whose wire format is verified against TypeSafe's first-party docs. But it is a foundation only: the FireRed hook, patch and bridge are not implemented, there is no shipped CLI, and nothing can run against a real game yet. It is MIT licensed and was created in October 2026.

Wide action spaces break naive policies

Jev on SMAC is the most instructive negative result in this group. It tests whether Jev can act as the policy in StarCraft multi-agent micro, in four steps. Step 1 gave Jev every unit's raw action each 0.5-second step: 0 wins in 35 games, the same as random, while attack-move won 25. Step 2 let Jev pick 2 to 6 named tactics per group and won 91 of 115 on official maps, but four hand-written rules won 92. Step 3 on unseen battles reached 196 of 400, tied with the hand rules at 196 and with SMAC's three-line focus-fire rule at 193. Step 4 layered choice on systems built to generalize, improving unseen wins from 475 to 720 per 1,000 battles.

The pattern is consistent: Jev helped when the choice was a small set of named tactics, and the tactical knowledge came from code and rules rather than the model. Giving it every raw action was worse than a simple baseline. The repository declares no license, and the step-4 results table is truncated before its conclusion. The result is still useful: a decision model is a chooser, not a planner, and a wide action space is a planning problem.

Vision agents still need a controller

mario-decisions-agent has no memory hacks and no hidden game state: pixels in, button out. Its brain is a remote vision model, either the OpenAI Decisions API with gpt-6-luna or TypeSafe's Jev with jev-preview, and the only question it is ever asked is about a picture of the screen. It is experimental, Python 3.12, macOS and Linux, MIT licensed, and reproducing the recorded runs needs your own API key. The README does not document the decision cadence, emulator requirements or cost for the Jev path.

Jev Factorio Agent is the more architectural version. Jev chooses goals and actions as typed questions, while deterministic code owns the game rules and actuation. The quick start is an offline mock loop that needs no key and no game copy; live Factorio and a hierarchical controller are documented as additional paths. The README badges claim MIT and experimental status, but the digest provides no dates, language or license field, and the excerpt truncates in the shipped-features note.

Both projects point at the same seam: the model can choose, but the code must translate the choice into an action the game can validate. The agent action gating discussion makes the same argument for browser agents.

Benchmarks are simulators with caveats

JevPokerBench is a Texas Hold'em benchmark and playground. Cash and sit-and-go results stay separate, and model action probabilities and confidence are explicitly not calibrated win probabilities. Seven local model routes share a 128-request concurrency pool, official Jev has its own pool, and human rooms time out model requests after 15 seconds and fold the seat if the action is illegal. Chips are virtual, registration needs no invitation, and BYOK agents need a model name, a public HTTPS endpoint and a key. The README warns that a displayed model name alone does not establish the weights or provider behind a service.

Decision Blackjack is a cleaner controlled comparison. Three players receive identical card sequences from a shared shoe pool, Jev gets dynamically generated prompts with the table situation, legal actions and hand values, and an oracle scores every move against Basic Strategy. Fast batch mode runs up to 100,000 rounds headless. Mock mode simulates both AI players without a key. The README shows no results from actual benchmarking runs, so its comparison claims are untested, and Laya needs a separate local inference server the project does not ship.

The methodological point is worth copying: identical deals, a known optimal policy and an error taxonomy by hard totals, soft totals, pairs and surrender. That is how a game benchmark becomes a decision benchmark rather than a highlight reel.

Arcade as a harness

System One Arcade is a Bun workspace with a typed API over five decision-model providers, including TypeSafe. The sample calls OpenRouter with ~typesafe/jev-latest and boolean, choice and score questions over text, JSON, image or audio state. A React arcade ranks models, and every call is stored in SQLite with tokens, cost, latency and error codes. The default budget is 1 USD per UTC day, configurable per provider, and over-cap calls return HTTP 429.

The games are Arena on Ring and Plane fields in lockstep or real-time clocks, plus Attention with a 9x9 curiosity grid. The latency and provider comparisons come from the author's own arcade runs rather than an external benchmark, and live calls need provider keys. The budget cap is the most practical feature: it makes a model comparison cheap enough to run repeatedly without a surprise bill.

The trading and markets roundup and framing sensitivity articles cover the transferable parts. The use-case map lists gaming among its example domains, but the map is a catalog of decision shapes, not evidence that any of these policies is good.

What the games show is that per-tick decisions work when the action set is small and the rules stay in code. When the action set is wide, or the task needs several moves of lookahead, a chooser is the wrong tool. The best results in this group come from layering a model choice on top of a system that already knows how to play.

From the Jev directory

The resources behind this article.

A 6.5M-parameter independent chess policy model built from Jev's public description: it returns a calibrated distribution over legal moves in one forward pass, with a self-reported held-out top-1 of 36.5%, ECE 0.036 and Elo 1239 plus or minus 51.
Jev Models & Reimplementations#community#open-models#chess
Community
0GitHub stars
A small, mechanics-free System One decision core for game NPCs: the adapter supplies an observation and opaque legal actions, and a scripted or Jev provider picks one, with a deterministic offline provider for tests.
Jev Repos & SDKs#community#github#python
3GitHub stars
A four-step experiment asking whether Jev, a fast model that picks among named options, can act as the policy in StarCraft multi-agent micro (SMAC), comparing every step against baselines that had the same information but no Jev.
Jev Benchmarks & Evaluations#community#github#benchmark
A Super Mario Bros agent whose only brain is a remote vision model on the other side of an API: screenshots in, button out, and no memory peeking. It runs on the OpenAI Decisions API by default and supports TypeSafe's Jev (jev-preview) as an alternative.
Jev Demos & Experiments#community#github#mario
4GitHub stars
An experimental Factorio agent in which Jev chooses goals and actions as typed questions while deterministic code owns the game rules and actuation; an offline mock loop runs with no API key and no game copy, with live and hierarchical modes documented.
Jev Agents & MCP#community#github#python
12GitHub stars
ProphetLab's Texas Hold'em benchmark and playground for decision models: separate cash and sit-and-go leaderboards, hand replays, a Jev-backed advisor, and bring-your-own-agent tables, all with virtual chips.
Jev Benchmarks & Evaluations#community#catalog-import#madewithjev
3GitHub stars
A benchmarking studio and real-time simulator that pits Jev (remote TypeSafe System One), a local Laya model and the optimal blackjack Basic Strategy against each other on identical deterministic shoes. It scores every deviation turn by turn and runs headless batches of up to 100,000 rounds.
Jev Benchmarks & Evaluations#community#github#blackjack
1GitHub stars
A Bun workspace pairing a typed decision API over System One providers with a React arcade where decision models play games and every call is stored and ranked. The sample code calls Jev through OpenRouter with boolean, choice and score questions.
Jev Demos & Experiments#community#github#typescript
Explore TypeSafe use cases by industry and turn promising ideas into concrete software workflows.
Jev Practices & PatternsDocs#official#use-cases
Official
Back to all articles

Related Jev articles

A game tick is a state plus a legal action list. How Clash Royale, Minecraft, and the earlier Mario, Pokémon, StarCraft, and drone loops ask Jev one step at a time.
Jev Demos & Experiments#gaming#agents#control
Read article
Voice agents need decisions in milliseconds. How turn detection, intent, and escalation work in real time, and where a fast decision layer fits.
Jev Practices & Patterns#voice#realtime#agents
Read article

From the Jev AI model community

Posts from builders shipping with Jev right now.

Follow @typesafeai