Jev Uncertainty

A benchmark providing exact instance-level class probabilities across three simulated text domains: peer review, covert information transfer, and credit risk. Because the data-generating process is defined, exact probabilities and Bayes error are computable, enabling KL-divergence scoring of reported probabilities.

An audit of whether Jev's confidence reflects missing knowledge rather than chance, using over 15 public datasets and 6 generated task families with paired interventions. Jev's confidence is calibrated on familiar closed-choice tasks but fails as an indicator of missing knowledge.

Introduces Sys1Cal-v1, a dataset of True/False questions about propositions whose exact probability is known by construction, queried through Jev's three primitives and scored by total variation from ground truth. Recovering a suppressed third value raises median soft accuracy from 0.771 to 0.978.

KITE queries a typed behavioral kernel once per unique state, then runs populations of any size from the table, reserving an expensive flagship model for sparse paired anchors. On Epstein experiments with 9,070 participants, anchors covering 1.7% of states cut effect error by 41%.

An exploratory study of JevForest, a budgeted feature-acquisition policy that aggregates path-dependent proposals from bootstrapped trees and queries Jev for semantic answers. On small held-out samples it beats static and random orderings on AG News but not on TREC.