Jev directory
CommunityJev Repos & SDKs
Jev Trust
Trust middleware for Jev decision APIs: it logs every call, measures calibration in your domain as outcomes arrive, discounts stated confidence to what it is worth there, and signs the log for outside recomputation.
- Reading time
- 2 min
- Published by
- Community
- Last verified
- Oct 2026
Highlights
- Every call logged to append-only JSONL with question, answer, stated and effective confidence, usage, and timestamp.
- record_outcome() builds your domain's calibration record: accuracy, Brier, ECE, and calibration currency C = 1 − ECE.
- effective_confidence reports what a stated confidence is worth where you run, with bin-observed, C-adjusted, or insufficient-n bases.
- Overconfidence alerts fire when a decision is confident enough to act on but the domain has not earned that confidence.
- One ed25519 signature covers the whole log so anyone can recompute the numbers; zero dependencies, Python 3.8 and later.
Quickstart
bash
pip install jev-trustWatch out
Copy reviewed against the PyPI package description on 2026-10-15. The page cites the author's linked Assay research: hosted Jev 1.13 at ECE 0.041 on closed deterministic tasks, and 50 percent accuracy while stating 91 percent confidence on synthetic email triage. No licence or repository is stated on the package page.
Questions about this entry
Short answers drawn from the linked project or page.
Calibration is a property of model and domain pairs, not models alone, and nobody has measured it for your domain. jev-trust wraps the Jev API and measures calibration where you actually run, so confidence is spent at its verified exchange rate.
stated_confidence is what Jev claims on an answer; effective_confidence is what that stated number is worth in your domain so far, taken from the observed accuracy of the matching confidence bin once it has at least five samples, otherwise C-adjusted, otherwise reported as insufficient evidence.
As ground truth arrives, record_outcome() accumulates the domain's record: number of outcomes, accuracy, Brier score, ECE, and a calibration currency C defined as 1 minus ECE, with a verdict such as FACE_VALUE, DISCOUNT, DOWNGRADE, or UNVERIFIED.
When a decision is confident enough to act on, say stated confidence at or above 0.90, but the domain has not yet earned that level of trust. You supply the threshold and a callback, and the alert can route the decision to a human instead.
One call signs the whole log with ed25519. You can publish the log, its signature, and the public key, and anyone, including the Nautilus Assay wall, can independently recompute your accuracy, Brier, and ECE numbers from the artifacts.
Python 3.8 or later with zero dependencies and a pure stdlib implementation, plus a Jev API key for the calls it wraps. The package itself adds no third-party dependencies to your environment.
No. It is third-party middleware published on PyPI that wraps the Jev decision APIs and measures their calibration in your domain; the package page links its own Assay research and states no licence or repository, so treat support and reuse terms as unset.
More Jev resources like this
A Python library that turns Jev-style decision models into ordinary function calls: check returns a bool, classify picks one option, label picks every option that applies, and score rates on a scale, with probabilities behind every answer.
Jev Repos & SDKs#community#github#python
Community
jlowin/vibecheckPython
jlowin/vibecheckPythonA fast, typed Python framework from TrustifAI for evaluating LLMs, RAG pipelines and agents with decision models: Jev by default, Microsoft Decision-1 on OpenRouter or OpenAI's Decisions API through the same backend. It returns typed scores, thresholds and status, with optional human-label calibration.
Jev Benchmarks & Evaluations#community#github#python
Community
trustifai/typed_evalsPython
trustifai/typed_evalsPythonA pluggable decision substrate for RAG pipelines: explicit state → Decision → confidence → action gates with Jev as the first swappable backend, plus a calibration-first evaluation harness for five decision primitives.
Jev Repos & SDKs#community#catalog-import#logicrw
Community
ajanm007/jevragPython
ajanm007/jevragPythonAn open-source, locally executed decision engine in the spirit of Jev (TypeSafe AI): state becomes features, and Noul, Choice and Score heads return typed, calibrated probabilities without generating text. It ships a cheap-to-strong cascade and a full train-calibrate-evaluate-deploy cycle.
Jev Repos & SDKs#community#github#python
Community
slavabars/decision-cascadePython
slavabars/decision-cascadePythonProbabilistic control flow for TypeScript: Hunch asks a System One model such as Jev yes/no questions, thresholded booleans, picks and ratings, and returns typed answers you can branch on in ordinary code.
Jev Repos & SDKs#community#catalog-import#cobanov
Community
carldaws/hunch-tsTypeScript
carldaws/hunch-tsTypeScriptA Python library that wraps TypeSafe System One with a deterministic local layer: canonical JSON sorting, SHA-256 fingerprinting with volatile key masking, SQLite caching, neutral escape injection and calibration checks, all on the standard library, with an official MCP server for coding agents.
Jev Repos & SDKs#community#catalog-import#madewithjev
Community
seb4ez/jevguardPython
seb4ez/jevguardPythonFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.