What is JEVals and what does it evaluate?

An open-source playground for evaluating System One and JEV-compatible models: it validates a structured judgment request, runs it through an emulator and real providers, compares the outputs and records every execution.

Which providers can JEVals compare?

A JEV baseline via typesafe.ai, an independent answer and a judge through any OpenAI-compatible endpoint, plus an emulator for offline comparison, with streaming progress shown during runs.

What do the three JEVals screens do?

Principal lets you edit, validate and run a request; Investigation shows the why behind an execution, including evidence, payloads and the full LLM exchange per question; Operation browses the persisted execution history.

What is the NOUL investigation view in JEVals?

It analyses probabilistic judgments on the 0-1 scale, comparing Emulator and JEV probabilities and reviewing delta and midpoint behaviour when predictions diverge; the digest truncates this section.

How is JEV fidelity measured in JEVals?

The execution summary reports JEV fidelity, question alignment, semantic divergence and independent verification, along with the models involved in the run. The digest does not include formulas or calibration numbers.

The digest does not show setup commands; the repo is TypeScript with Next.js and FastAPI topics and an MIT license. The README excerpt covers the three screens and the summary views only.

Who maintains JEVals?

The duberblock GitHub account; metadata shows 2 stars and 2 forks, created and last pushed 2026-09-28. It is a distinct project from other tools that share the jevals name.

Jev directory

CommunityJev Benchmarks & Evaluations

JEVals — duberblock

A self-contained playground for evaluating structured judgment requests against LLM providers: validate a SystemOneRequest, run it through an emulator, a JEV baseline via typesafe.ai and an OpenAI-compatible judge, then compare results on three web screens.

See highlights Open github.com

Stars: 2
Language: TypeScript
License: MIT
Reading time: 2 min
Published by: Community
Author: duberblock
Last verified: Oct 2026

duberblock/jevals

Organization repository on GitHub

View on GitHub

Open-source playground for evaluating System One and JEV-compatible models with deterministic fidelity, semantic judging, and independent verification.

Forks: 2
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

This is one of several Jev listings for the same project. The canonical entry is Jevals — openlayer-ai.

Tagscommunity github benchmark nextjs system-one

Works forJev for Evaluation Jev for Verification

Used inJev for Software & DevTools

Languages: TypeScript
Platforms: Next.js
Capabilities: Open source

Highlights

Validates a SystemOneRequest payload and runs it through an emulator, a JEV baseline via typesafe.ai and an OpenAI-compatible judge.
Three screens: Principal (edit, validate, run), Investigation (evidence and the full LLM exchange) and Operation (persisted history browser).
Reports JEV fidelity, question alignment, semantic divergence and independent verification per run.
The NOUL investigation view compares Emulator and JEV probabilities on the 0-1 scale, including delta and midpoint behaviour.
MIT-licensed TypeScript/Next.js repo, 2 stars, created and pushed 2026-09-28.

Watch out

README and GitHub metadata reviewed 2026-10-17; the digest truncates the NOUL Investigation section, so setup instructions, the emulator's logic and any benchmark results are not evidenced here.

Questions about this entry

Short answers drawn from the linked project or page.

From the Jev guides

Original write-ups on what this entry is for.

The trust ledger: measuring confidence your domain has earned

Oct 16, 2026

ASSAY-001 calibrated Jev on CLINC150 but found it overconfident on Banking77 — one audit, two verdicts. How calibration research measures the model-and-domain pair, discounts stated confidence, and signs the evidence.

Jev Benchmarks & Evaluations#calibration#evaluation#open-models

Read article

From the Jev AI model community

Posts from builders shipping with Jev right now.

Follow @typesafeai