User repository on GitHub
- Forks
- 0
- Last pushed
- Updated Oct 2026
Repo stats from the GitHub API, cached Oct 2026.
Jev directory
CommunityJev Benchmarks & Evaluations
Jev on SMAC
A four-step experiment asking whether Jev, a fast model that picks among named options, can act as the policy in StarCraft multi-agent micro (SMAC), comparing every step against baselines that had the same information but no Jev.
- Stars
- 3
- Language
- Python
- Reading time
- 2 min
- Published by
- Community
- Author
- wang90063
- Last verified
- Oct 2026
Used inJev for Gaming
- Languages
- Python
- Capabilities
- Open source
Highlights
- A four-step experiment testing whether Jev can act as the policy in StarCraft multi-agent micro (SMAC).
- Step 1 gave Jev every unit's raw action each 0.5 s step: 0 wins in 35 games, the same as random; attack-move won 25.
- Step 2 let Jev pick 2-6 named tactics per group and won 91 of 115 on official maps, but four hand-written rules won 92.
- Step 3 on unseen battles reached 196 of 400, tied with the hand rules (196) and SMAC's three-line focus-fire rule (193).
- Step 4 layered choice on systems built to generalize, improving unseen wins from 475 to 720 per 1,000 battles.
Watch out
README and GitHub metadata reviewed 2026-10-17; the digest truncates the step-4 results table mid-sentence and the repo declares no license, so the final conclusions, setup instructions and code structure are not evidenced here.
Questions about this entry
Short answers drawn from the linked project or page.
A write-up and code repository testing whether Jev, a fast model that picks among named options, can control units in StarCraft multi-agent micro (SMAC). It records four steps, each compared with baselines that had the same information but no Jev.
Giving Jev every unit's raw action every 0.5 s step won 0 of 35 games, the same as random actions, while attack-move won 25. The README presents that failure as the prompt for the next step.
Step 2 let Jev choose 2-6 named tactics per unit group and won 91 of 115 games on the official maps, but random answers to the same questions won 74 and four hand-written rules built from the same knowledge won 92.
On unseen battles Jev's choices reached 196 of 400, tied with the hand rules at 196 and with SMAC's three-line focus-fire rule at 193, so the README attributes the tactical knowledge to the code and rules rather than to Jev.
The digest shows the author crediting code and hand rules rather than Jev: random answers and simple rules performed comparably, and the tactical knowledge appears fitted to the official maps rather than generalized.
The digest does not include setup or run instructions; it shows only the results narrative. The repository is Python, has 3 stars, was created 2026-10-08 and last pushed 2026-10-09, and declares no license.
The wang90063 GitHub account. The repo title frames it as a question ('Can a Fast Choice Model Play StarCraft Micro?'), and the digest's step-4 table is truncated before its conclusion.
More Jev resources like this
A benchmarking studio and real-time simulator that pits Jev (remote TypeSafe System One), a local Laya model and the optimal blackjack Basic Strategy against each other on identical deterministic shoes. It scores every deviation turn by turn and runs headless batches of up to 100,000 rounds.
Jev Benchmarks & Evaluations#community#github#blackjack
Community
mrbanana55/decision_blackjackPython
mrbanana55/decision_blackjackPythonA deterministic gridworld that benchmarks System One decision backends, typed calibrated decisions over a declared option set with one forward pass and no decoding, in a sequential loop, with a scripted oracle scoring agreement without labels.
Jev Benchmarks & Evaluations#community#github#benchmark
A Python benchmark that evaluates LLMs' reasoning and instruction-following by making them play chess against a random player or the Komodo Dragon engine in multi-turn dialogs, separating chess skill from protocol durability.
Jev Benchmarks & Evaluations#community#catalog-import#madewithjev
Community
maxim-saplin/llm_chessPython
maxim-saplin/llm_chessPythonA timestamped benchmark and trading arena that replays resolved prediction markets: models commit to probabilities using only information available at the time, and realized P&L is the headline metric with Brier, log-loss and ECE as checks.
Jev Benchmarks & Evaluations#community#github#python
Community
ruyianry/JevGymPython
ruyianry/JevGymPythonGTM workflow experiments with Jev from deepdave98: weekly datasets, prompts, and benchmark results that test Jev against Claude on lead triage and routing, deal risk, reply triage, MEDDPICC from call summaries, duplicate accounts, and closed-lost reactivation.
Jev Benchmarks & Evaluations#community#github#gtm
Community
deepdave98/jev-playgroundPython
deepdave98/jev-playgroundPythonA benchmark of Jev and other decision models on trolley-problem states, with runners, raw answers and analysis in one repository. The v1 snapshot covers 1,098 states and the October 2026 v2 run has 14 models on 5,240 frozen states.
Jev Benchmarks & Evaluations#community#github#benchmark
Community
zachlandes/trolley-benchHTML
zachlandes/trolley-benchHTMLFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.