User repository on GitHub
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
Jev directory
CommunityJev Practices & Patterns
Jev Playground — hegargarcia
A Next.js playground that benchmarks Jev against GPT-5.6 Luna, Claude Haiku 4.5, Gemini 3.5 Flash Lite, and GPT-6 Astra in tic-tac-toe and Connect Four, giving every model the same legal-move Choice options through the Vercel AI Gateway.
- Stars
- 2
- Language
- TypeScript
- Reading time
- 4 min
- Published by
- Community
- Author
- hegargarcia
- Last verified
- Sep 2026
This is one of several Jev listings for the same project. The canonical entry is Jev Playground (MoonBit).
About this project
A playground for benchmarking TypeSafe AI’s Jev against other evaluation models in games with explicit states, legal actions, and measurable outcomes. The core question: how well does each model choose the next action when the rules and available choices are clearly defined? Games provide a small, inspectable environment for exploring decision quality, tactical reasoning, and consistency across a sequence of moves. Each game acts as a state machine:
Quoted from the project's README. Read the full README.
Works forJev for Evaluation
Highlights
- Tic-tac-toe and Connect Four pit Jev against four frontier models on identical legal-move options and shared tactical instructions.
- Jev uses a native evaluation model while Astra's structured output is restricted to legal squares or columns by a Zod schema.
- Each decision log shows chosen moves, board snapshots, option weights when returned, and provider confidence when available.
- Aggregate scores are not published yet: automated tournaments and controlled runs are listed as future work.
- Tests use mocked evaluations and make no paid model calls.
Quickstart
bash
bun install
echo 'AI_GATEWAY_KEY=YOUR_API_KEY' > .env.local
bun run devWatch out
No license file, so reuse terms are unclear. Needs Bun and a Vercel AI Gateway key; it compares interactive games only and publishes no aggregate results yet.
Questions about this entry
Short answers drawn from the linked project or page.
Just the boards. Automated tournaments and aggregate scores are listed as future work, and the README is blunt that interactive play is useful for finding interesting positions while controlled runs are what broader performance claims need. It does name what such a run should track: decision quality, consistency across repeated evaluations of the same state, reliability, and latency and cost.
Not as win probabilities. Option weights describe the model's returned Choice distribution, provider confidence is a separate statistic that is not assumed comparable across models, and missing values are shown as unavailable rather than inferred. Astra is the clearest limit: its structured response contains only the chosen square, so its weights and confidence are never populated.
The adapters are not equivalent, so read latency with that in mind. Jev uses a native evaluation model, Luna, Haiku and Gemini go through the SDK's experimental evaluation adapter around Gateway chat models, and Astra uses generateText with Output.object and a Zod schema restricting choice to legal option names. Each board's log shows the choice that changed the game, which is the part you can actually audit.
Tests use mocked evaluations and make no paid model calls, so bun run test, lint and build are free of API spend. The dev server is different: it needs Bun and a Vercel AI Gateway key in .env.local, and the key is read only on the server while environment files are ignored by Git. Note also that there is no license file, so reuse terms are unclear.
Five, all through the Vercel AI Gateway: Jev as typesafe-ai/jev, GPT-5.6 Luna as openai/gpt-5.6-luna, Claude Haiku 4.5 as anthropic/claude-haiku-4.5, Gemini 3.5 Flash Lite as google/gemini-3.5-flash-lite, and GPT-6 Astra as openai/gpt-6-astra. The list lives in src/lib/models.ts, and these are Gateway identifiers rather than provider-native model names.
The game code owns the rules, legal actions, state transitions and terminal conditions; the model only chooses among the options it is handed. Tic-tac-toe supplies every legal square as a Choice option with a descriptive name, coordinates and the resulting board, under shared instructions covering win, block, forks and holding a draw. Connect Four does the same per column with its landing row, and the server validates gravity, turn counts and terminal states.
Next.js, React, TypeScript, Bun and the Vercel AI SDK, with rules in src/lib/game.ts and src/lib/connect-four.ts, prompts in the matching ai.ts files, and endpoints at /api/move and /api/connect-four/move. Each board keeps its own decision log with chosen moves, board snapshots, option weights and provider confidence when supplied, plus retry and restart that affect only that board.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
Hacker News: Software Bombe and live Enigma demo where Jev ranks cribs and judges plaintext.
ag_212 pointsShow HN: Jev as the Bletchley Park Analyst: A Software Bombe on EnigmaRead the thread on Hacker NewsHacker News: Show HN for public Minecraft PvP where a diffusion LM picks moves from structured state.
2 pointsShow HN: Minecraft PvP against DiffusionGemma-as-Jev, 24 ms per decisionRead the thread on Hacker NewsHacker News: Show HN: Jev finished a Pokémon Red run, with the Hall of Fame stats on the landing page.
pancomplex272 pointsShow HN: Jev Plays Pokémon RedRead the thread on Hacker NewsHacker News: Show HN: a mouse colony where each next move is Jev or a fixed rule.
1 pointShow HN: Jev-mice – mouse colony simulation using Jev and deterministic engineRead the thread on Hacker NewsMore Jev resources like this
A personal-assistant agent with 100 mocked tools that measures how many steps a task takes when the LLM picks the tool versus when Jev picks it before every model step.
Jev Practices & Patterns#community#typescript#agents
Community
vinilana/jev-eval-agentHTML
vinilana/jev-eval-agentHTMLSide-by-side benchmark of TypeSafe Jev, Qwen 3.8 27B on Cerebras, and a local Needle 3 across seven synthetic workloads, recording validated outputs, mistakes, latency, tokens, and estimated cost. Raw exports and per-scene limitations are published.
Jev Practices & Patterns#community#typescript#benchmarks
An independent CUDA and vLLM take on parallel typed decisions: it scores verified single-token labels with the model's own head and assembles JSON in code.
Jev Practices & Patterns#community#python#javascript
Community
kikoncuo/jevfireJavaScript
kikoncuo/jevfireJavaScriptScott Chacon runs hosted Jev against two local decision models, laya and kev, in a Tetris match and a GitHub Settings filter, with a public test repo.
Jev Practices & PatternsVideo#community#video#open-models
Community
A self-hosted implementation of TypeSafe's Jev System One API powered by the 400M-parameter GLiFormer encoder: it serves choice, score, and noul and drops into the official typesafe-sdk via TYPESAFE_BASE_URL, but trails Jev on reasoning-heavy tasks.
Jev Practices & Patterns#community#python#open-models
Community
logan-markewich/jeffPython
logan-markewich/jeffPythonA comparison arena that runs the same batch of review comments through Jev and DeepSeek, showing processing time, cost, and per-label results with CSV/Excel import, replay, and offline reports.
Jev Practices & Patterns#community#javascript#benchmarks
Community
NanmiCoder/jev-arenaJavaScript
NanmiCoder/jev-arenaJavaScriptFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.