Systemone Gridworld

A Python benchmark that evaluates LLMs' reasoning and instruction-following by making them play chess against a random player or the Komodo Dragon engine in multi-turn dialogs, separating chess skill from protocol durability.

A CLI from Pinecone's DevRel team for testing Agent Skills and docs against tasks: run tasks across Claude, Copilot and Gemini in local or Modal sandboxes, compare with-skill, without-skill and with-docs variants, and grade with Claude or a TypeSafe System One backend.

A benchmark of Jev and other decision models on trolley-problem states, with runners, raw answers and analysis in one repository. The v1 snapshot covers 1,098 states and the October 2026 v2 run has 14 models on 5,240 frozen states.

The code and data behind ProbabilityBench: text-based simulated worlds with exact class probabilities, so a decision model's reported probabilities can be compared directly with ground truth. An arXiv paper and a Hugging Face dataset accompany it.

A reproducible experimentation framework and report series for Jev: it runs shared tasks at scale across benchmarks, routing and numeric prediction, and publishes bilingual findings on where the model holds up and where it does not.

AkitaOnRails' study documentation and benchmark harness for Jev-like decision models: frozen inputs, inspectable expected answers, raw response archives and reproducible scoring compare TypeSafe Jev with Decision-1, OpenAI Decisions, Clef, Decision 3.0, Laya, Kev, Eikos, Jebadiah, Mapika Decider and GLiNER.