Skip to content

Jev MCP — BYK

An MCP server built around measuring questions rather than asking them: jev_ask prototypes wording, jev_eval sweeps thresholds against labeled data, jev_map applies the winner over a workload.

Stars
3
Language
TypeScript
License
MIT
Published by
Community
Author
BYK
Last verified
Oct 2026

BYK/jev-mcp

User repository on GitHub

View on GitHub

An eval-first MCP server for TypeSafe's Jev, a System One model that returns typed judgments (noul, choice, score) with probabilities instead of generated text.

Forks
0
Last pushed
Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

This is one of several Jev listings for the same project. The canonical entry is Jev MCP — jkudish.

About this project

An eval-first MCP server for TypeSafe's Jev, a System One model that returns typed judgments (noul, choice, score) with probabilities instead of generated text. Most Jev integrations expose "ask the model a question". The hard part in practice is not asking — it is knowing whether a question works and where to put the threshold. This server is built around that:

Quoted from the project's README. Read the full README.

Highlights

  • jev_eval reports accuracy, calibration, a threshold sweep and the worst misses, so the threshold comes from a table.
  • jev_map runs a question set over many items, one compact row each, with save_path to keep bulk results out of context.
  • Every question in one call is answered against the same state in a single request; ids are local labels, never sent to the model.
  • The tool descriptions are themselves evaluated, with a labeled request set comparing them against a terse variant.

Watch out

You need Node 20 or newer and a TypeSafe API key, and the quickstart runs the server through npx -y, which fetches the published package rather than anything you have read; build from a clone and point your client at dist/index.js if you would rather pin the code. Question ids are yours alone and are not transmitted, so all of the meaning has to live in the instructions string or the criteria. The evaluation tools are only as good as the labeled data you supply, and thresholds are tied to a specific model version, which is why jev_eval prints the version it resolved. The sample run in the README uses 16 labeled items and two variants, which demonstrates the output format rather than establishing accuracy on your data. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

36GitHub stars
Pi extension that strips the full Agent Skills catalog from every request and loads only the skills a Jev ranking call judges relevant.
Jev Agents & MCP#community#jevbest#catalog-import
102GitHub stars
Behavioral code search that asks Jev to score repository fragments for a plain-language question and returns exact excerpts with paths and lines.
Jev Agents & MCP#community#jevbest#catalog-import
17GitHub stars
Command Code mod that asks Jev at every agent stop whether a nudge would help, continuing unfinished work while leaving waits for the user alone.
Jev Agents & MCP#community#jevbest#catalog-import
11GitHub stars
TypeScript evaluation utilities that test full AI agent conversations as end-to-end scenarios with binary assertions, without imposing a test runner.
Jev Agents & MCP#community#jevbest#catalog-import
7GitHub stars
MCP server exposing Jev Choice, Score, Noul, batched ask, file-reading triage, and model listing as typed tools with full probability distributions.
Jev Agents & MCP#community#jevbest#catalog-import
5GitHub stars
A Claude Code function hook that sends AskUserQuestion dialogs to Jev and draws a probability per option above the dialog, so you still choose while seeing how confident the model is.
Jev Agents & MCP#community#jevbest#catalog-import
Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai