Is this a benchmark with accuracy scores?

No. Prompts and records are original synthetic teaching examples, not a copy of workflow evals or papers, with no automatic pass-or-fail grading. One captured Monty Hall mistake is an example to investigate, not an aggregate accuracy score, and open dilemmas have no right-answer key.

Will browsing examples spend my API key?

No. Selecting examples never calls TypeSafe; only Run example and Run A/B do, with A/B making two requests. The readme asks you to use the shared community key considerately since there is no bulk runner.

Can one A/B pair prove bias or its absence?

No. Each Bias and consistency pair changes exactly one field while keeping operational facts and policy identical, and differences report in percentage points. The readme says one pair cannot establish bias; use repeated runs, multiple examples, controls, and documented criteria.

Where do my key, edits, and results actually go?

The key stays in the server process, not saved to a file or sent to the browser, with env key reused without prompting. Edits persist per browser and address including port, results stay only while the page is open, and Export run includes request, answers, model, timing, and token usage.

How many examples and categories does the playground start with?

110 runnable examples in 22 categories, including 41 A/B comparisons. They are grouped into four collections: Fun & games, Dilemmas & debates, Model challenges, and Use cases.

What setup requirements does the playground have?

Python 3.10 or newer, a modern browser, and a TypeSafe API key. There are no packages to install and no frontend build step; you run python3 run.py.

Are Test notes or example selections sent to TypeSafe?

No. Test notes are never sent to TypeSafe, and selecting examples does not call TypeSafe. Only Run example and Run A/B make requests, and Run A/B makes two.

CommunityJev Demos & Experiments

TypeSafe AI Playground — nickthompson480

Local three-panel playground for trying TypeSafe judgments, with 110 editable examples across 22 categories including A/B comparisons, running one request per Run without bulk automation.

See highlights

Open github.com

Stars: 3
Language: JavaScript
License: MIT
Published by: Community
Author: nickthompson480
Last verified: Oct 2026

nickthompson480/typesafe-ai-playground

User repository on GitHub

View on GitHub

Community TypeSafe AI playground: 110 use cases, games, dilemmas and model challenges, with editable prompts, A/B comparisons and a mobile-friendly UI.

Forks: 2
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

This is one of several Jev listings for the same project. The canonical entry is TypeSafe AI Playground — BunsDev.

About this project

A community playground for exploring TypeSafe AI with practical use cases, party games, dilemmas, and reasoning challenges. Pick an example, inspect its input and questions, and run it through the API. The library starts with 110 runnable examples in 22 categories, including 41 A/B comparisons. It is meant to grow through shared experiments and contributions. This is an independent community project, not an official TypeSafe product.

Quoted from the project's README. Read the full README.

Tagscommunity jevbest catalog-import awesomejev

Works forJev for Classification

Used inJev for Software & DevTools

Languages: JavaScript

Highlights

Desktop Examples, Test setup, Results layout suits screen-share; phone flow becomes Browse, Build, Results.
Selecting examples never calls the API; Run example evaluates selected questions in one request, Run A/B makes two.
Test notes stay local and are never sent; edits persist per browser with reset, save-as-new, and export support.
Bias checks change one field like name or age while keeping facts identical, reporting gaps in percentage points.

Watch out

Independent community project, not official. Needs Python 3.10 or newer, a modern browser, and a key kept only in the server process, with no .env autoload and loopback-only serving; results live only until reload and exports may contain your input. Reference answers are teaching notes, not a validated benchmark. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

laya-vs-jev-arena

Side-by-side game harness where Laya running locally and Jev over API answer the same typed questions for snake, kombat, runner, flappy, tetris, and human-versus-model matches.