brandonbryant12/transcript-scorecard
User repository on GitHub
ACME live support-call scoring demo with TypeSafe AI, Effect, SQLite, React, Vite, and Turborepo
- Forks
- 0
- Last pushed
- Updated Oct 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Tools & CLIs
Transcript Scorecard
A Turborepo proof of concept that replays a fictional ACME support call about once per second and schedules a real TypeSafe evaluation for every new turn, coalescing turns that arrive while a request is in flight and persisting evidence, confidence and score history to SQLite.
- Stars
- 0
- Language
- TypeScript
- Published by
- Community
- Author
- brandonbryant12
- Last verified
- Oct 2026
About this project
Transcript Scorecard is a runnable proof of concept for evaluating ACME support calls against a weighted employee scorecard. Its live replay reveals a realistic transcript at a configurable pace and evaluates batches with the selected provider. By default, one turn appears every 0.5 seconds and scoring runs every five turns. When an evaluation is still running, incoming turns coalesce into the next request using the newest revealed transcript prefix. The weighted score recalculates as the conversation develops,…
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
- Languages
- TypeScript
Highlights
- While an evaluation is still running, incoming turns coalesce into the next request using the newest revealed transcript prefix.
- The weighted score recalculates as the conversation develops; the final evaluation, evidence, confidence and history persist in SQLite.
- JEV_API_KEY stays in the API development task and never reaches the Vite client; the studio shows round-trip, average and P95.
- Two featured calls show opposite shapes: an accurate opening that drops off, and a rough opening that recovers.
Watch out
The README calls this a runnable proof of concept. The calls are fictional, but the scores come from real TypeSafe API requests, so they can vary between runs, and replaying a call consumes API usage. The local demo has no authentication. Reported latency is the server-measured TypeSafe SDK round trip for each successful scoring request, with browser polling delay excluded and SDK retries included. Running it needs Node 22 or newer and pnpm 10 or newer; the SQLite file is created automatically at apps/api/data/scorecard.sqlite and ignored by Git, and JEV_API_KEY must not be prefixed with VITE_. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
Real calls: a TypeSafe AI evaluation is scheduled for every new turn the replay reveals. The transcripts themselves are fictional and the scores come from real API requests, so they can vary run to run, and replays consume API usage. Two featured calls are provided - Cannot connect smart hub for an accurate opening followed by a drop, and Duplicate subscription charge for a rough opening that recovers.
It schedules one evaluation per new turn, but when an evaluation is still in flight, incoming turns coalesce into the next request using the newest revealed transcript prefix, so a burst of turns does not become a burst of separate paid calls. The replay reveals a realistic transcript roughly once per second, which is exactly the case the coalescing exists for. Nothing in the repo caps total spend, so the API account is your only backstop.
No. The browser talks to the API over HTTP, and JEV_API_KEY is available only to the API development task and never included in the Vite client environment. The README warns explicitly against prefixing it with VITE_, because Vite intentionally exposes variables carrying that prefix to browser code. The web workspace runs on localhost:5173 and the API on localhost:3001, with the key read from .env or an exported shell variable, the shell taking precedence.
The server-measured TypeSafe SDK round-trip duration for each successful scoring request, displayed as recent request timings, an average and a P95 latency, with a running timer when an in-flight request is observed. Browser polling delay is excluded from the completed timings, while SDK retries, if any, are included. It is a TypeScript stack: pnpm workspaces and Turborepo, React 19 with Vite, Effect for typed application and service logic, and SQLite for persistence.
The studio defaults to OpenAI Decisions on gpt-6-luna and switches to JEV through the existing jev-latest adapter; each session keeps the provider it started with, and a missing key is reported without blocking saved sessions. Cost estimates use published input rates only, $0.10 per million for OpenAI Decisions against $0.042 for JEV, with no output charges, regional processing premiums and long-context multipliers excluded.
Node.js 22 or newer and pnpm 10 or newer. The workspace pins its runtime to Node 22.22.0, which pnpm downloads and uses for project scripts even if your shell is on another version. Then corepack enable, pnpm install --frozen-lockfile, cp .env.example .env with the keys filled in, and pnpm dev. If SQLite complains about NODE_MODULE_VERSION after a version switch, stop the watcher and run pnpm rebuild better-sqlite3; existing data survives.
None. The README says outright that the local demo has no authentication, so it should stay on a machine you control rather than a shared host. The SQLite database is created automatically at apps/api/data/scorecard.sqlite, resolved from apps/api and overridable with SCORECARD_DB_PATH, and it is ignored by Git. POST /api/health reports which providers are configured without exposing keys.
More Jev resources like this
38GitHub stars
Python memory filter that judges every stored memory against the current request with one calibrated yes/no per memory in a single Jev request, keeping everything above a threshold.
Jev Tools & CLIs#community#jevbest#catalog-import
Community
samdotmak/jev-recallTypeScript
samdotmak/jev-recallTypeScript36GitHub stars
Stock decision terminal that gathers Valyu market evidence across five categories and uses Jev for a typed 30-day outlook with probability, signals, and timing.
Jev Tools & CLIs#community#jevbest#catalog-import
Community
unicodeveloper/jevocksTypeScript
unicodeveloper/jevocksTypeScript4GitHub stars
Playful chat demo that makes Jev hold a conversation one word at a time by picking from a compact 979-word vocabulary through repeated Choice calls.
Jev Tools & CLIs#community#jevbest#catalog-import
Community
mkotlikov/jev-grugTypeScript
mkotlikov/jev-grugTypeScript4GitHub stars
Shell CLI that sends state plus noul, choice, and score questions to Jev in one request and prints probabilities, choices, and rubric positions for pipes and scripts.
Jev Tools & CLIs#community#jevbest#catalog-import
Community
y0usaf/typesafe-cliTypeScript
y0usaf/typesafe-cliTypeScript5GitHub stars
Local-first app that extracts document text on the machine and files it into configured category folders using typed Jev decisions for category, confidentiality, prompt-injection risk and subject, with a per-run audit that previews and undoes every move.
Jev Tools & CLIs#community#jevbest#catalog-import
7GitHub stars
PDF search that asks Jev one noul per line, does this line answer the query, and ranks the lines by the returned probability, so matches highlight without embeddings, a vector index or a chunking strategy. Text is extracted locally with pdf.js and the key stays on the server.
Jev Tools & CLIs#community#jevbest#catalog-import
Community
kylemclaren/jevpdfTypeScript
kylemclaren/jevpdfTypeScriptFrom the Jev guides
Original write-ups on what this entry is for.
The request shape barely changes across surfaces; the surrounding contract does. When to call the API directly and when a gateway, a Worker binding, or a router entry earns its extra hop.
Jev Tools & CLIs#gateway#vercel#cloudflare
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Jev labels a record that already exists. Code moves, flags, or holds it. How the document, mail, and transfer tools keep the action out of the model.
Jev Tools & CLIs#classification#email#security
Read article
From the Jev community
Posts from builders shipping with Jev right now.