Evaluation & TrainingJev Bench
JevBench
An independent benchmark for Jev-class decision models, scored as the harmonic mean of intelligence, calibration, speed, and cost.
- Category
- Evaluation & Training
- Also known as
- Jev Bench
- Related terms
- 4
- Directory entries
- 1
- Docs
- —
- Added
- 2026-10-11
Definition
Run outside TypeSafe, JevBench ranks systems that answer the Jev question shape, including open reproductions and hosted rivals, on one composite score built from four axes. Several open models have been reported above Jev 1.13 on its board.
The benchmark itself cautions that the speed axis for a self-hosted model is an assumption — measured latency doubled plus a fixed offset — rather than a live measurement, so the composite should be read alongside the per-axis numbers.
Tagsevaluationbenchmarks
Related terms
Definitions that connect to this one.
LeaderboardA public ranking of decision models on a shared task set, where the selection rules decide what the numbers mean.EcosystemBenchmark claimsTypeSafe's headline multipliers — 193.6x faster, 444.6x cheaper, a 238x lower input price — come from its own workflow comparisons.Evaluation & TrainingDecision IndexA Hugging Face leaderboard that scores open reproductions of Jev on the same frozen decision suite.Evaluation & TrainingOpen weightsModel checkpoints published for anyone to download, run, and fine-tune, as opposed to a hosted closed API like Jev.Ecosystem
From the Jev directory
253GitHub stars
Independent cross-model benchmark for Jev-class decision models, running 534 frozen cases per complete entrant with scoring code and a four-axis score of accuracy, calibration, latency and cost.
Jev Practices & Patterns#community#python#benchmarks
Community
fstandhartinger/jevbenchPython
fstandhartinger/jevbenchPythonFrom the Jev community
Posts from builders shipping with Jev right now.