User repository on GitHub
Eval framework library to evaluate AI hallucinations, correctness, and instruction following, powered by Jev AI.
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Integrations & Adapters
Ground Zero
An alpha Python library that uses Jev to grade model responses for hallucination, correctness and instruction following, returning Noul probabilities and five-level severity Scores.
- Stars
- 0
- Language
- Python
- Published by
- Community
- Author
- zavocc
- Last verified
- Oct 2026
About this project
Ground Zero is a library to evaluate model's hallucinations, correctness, and instruction following drift from user and tool or source material prompts against the model's responses.
Quoted from the project's README. Read the full README.
- Languages
- Python
Highlights
- Three modes with separate docs: hallucination against source material, correctness against expected output, and instruction following.
- Hallucination severity is a five-level Score whose number is the probability-weighted mean of the per-level probabilities.
- Noul criteria report one probability each, with 0.5 as the natural true/false split, and you choose the thresholds.
- The README says confidence describes how concentrated the probabilities are and does not guarantee the evaluation is correct.
Watch out
The README leads with two warnings: alpha status where schemas and API shapes may change at any time, and an explicit instruction that because LLM-as-a-Judge is still evolving you must manually validate the results, including score discrepancies and contradicting outputs. Installation is not a normal dependency add - the instructions are pip install of a .whl or .tar.gz from a releases page, and that URL points at a GitHub organization different from this repository's, so check where the artifact actually comes from before installing it. An OpenRouter account with $1 balance and an API key is listed as a requirement, and the README does not explain which calls use it. There are no evaluation numbers, no benchmark results and no test suite described in what we read; the roadmap lists documentation and schema as still in progress and asks for PRs to improve evals. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
The README does not explain it. The installation section states you must have an OpenRouter account with $1 balance and tells you where to get the key, while the decision-making model is described as Jev by TypeSafe via its Decisions API. Nothing in the digest maps the OpenRouter credential to a specific call, so you would be installing on trust rather than on documentation.
None in the README. There are no accuracy figures, no eval set, no comparison against another judge and no test suite described. The roadmap explicitly lists documentation and schema as work in progress and invites PRs to improve evals, and the caution block says score discrepancies and contradicting results are expected and that you must validate results manually.
You set the thresholds yourself - the README says you need a threshold based on how you interpret a probability, and for Noul roughly below 0.5 is false and above 0.5 is true. For Score, do not read the number alone: it is the probability-weighted arithmetic mean of the legend levels, so different probability distributions can produce the same score, which is why the README tells you to inspect probabilities alongside it.
Two. It is self-declared alpha, with schemas and API shapes liable to change at any time, and installation means pip installing a wheel or tarball from a releases page rather than adding a package from a normal index - a URL that points at a different GitHub organization than this repository. Given that, read the artifact before installing it and pin your expectations accordingly.
Match the mode to the question. Hallucination checks faithfulness to the prompt and source material and deliberately ignores world knowledge, correctness compares the output against an expected reference to probe parametric knowledge, and instruction following checks strict adherence to system, multi-turn and user instructions. For grounding a retrieval answer, hallucination is the natural first choice.
Two of the primitive shapes. A Noul is a yes/no probability where roughly below 0.5 is false and above 0.5 true, and you choose the threshold. A Score is one of a fixed legend of levels, and the number returned is the probability-weighted arithmetic mean of those level positions, so different distributions can produce the same score. Inspect probabilities alongside the score; confidence describes concentration, not correctness.
Jev by TypeSafe is the decision model at the core, reached through its Decisions API, and it does not generate text: it answers typed questions about a state and returns calibrated probabilities. The README's OpenRouter account with a $1 balance is an install requirement it names but never maps to a specific call, so the credential's role stays undocumented.
More Jev resources like this
41GitHub stars
Typed Python planner runtime that turns batched Jev action distributions into executable probability trees and graphs for finite decision tasks.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
Chuf-H/jev-treePython
Chuf-H/jev-treePython25GitHub stars
Agent runtime with a terminal UI and a messaging gateway that keeps memory, self-built skills, session search, scheduling, and subagents across Telegram, Discord, Slack, WhatsApp, Signal, and CLI.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
wundercorp/lokiPython
wundercorp/lokiPython51GitHub stars
GraphRAG layer that runs a 4-phase ingest-to-answer pipeline over Neo4j, Memgraph, AGE or Kuzu using Laya or Jev decisions instead of per-hop LLM calls.
Jev Integrations & Adapters#community#jevbest#catalog-import
23GitHub stars
Memory-maintenance library that checks every stored memory against each new event with Jev yes-or-no votes, then marks stale facts superseded or contradicted instead of editing them.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
chopratejas/invalidatePython
chopratejas/invalidatePython14GitHub stars
Python library for schema-enforced JSON extraction into Pydantic models, with provider-native JSON mode fallback, auto-repair, token and cost tracking, plus agents, RAG, and eval helpers.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
jhd3197/PrompturePython
jhd3197/PrompturePython30GitHub stars
Experimental language where questions and methods are composable values, with standalone .jpp sources run by a native Rust kernel and a Python implementation kept as reference.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
Towow-ai/jppRust
Towow-ai/jppRustFrom the Jev community
Posts from builders shipping with Jev right now.