When should I call jev_evaluate instead of the single-purpose tools?

Use jev_evaluate when several judgments share one state in a single call, capped at 20 questions by default. Use jev_check, jev_route or jev_score when you need one atomic Noul, Choice or Score with its own schema.

Do I pass levels or criteria for a Score question?

Score questions accept levels on the tool and map it to the API criteria list automatically; if both spellings are present, criteria wins.

What setup and failure behavior should I expect?

Set TYPESAFE_API_KEY in the Hermes env or your own environment, with optional model, timeout and max-questions settings. Handlers always return JSON strings, errors included.

Can I use it without learning question design?

No. The bundled jev-questions skill covers atomic question design, state preparation and confidence gating, and the recipes show triage, review-gate, routing and severity patterns.

What tools does the plugin add?

Four tools: jev_evaluate runs any mix of questions in one call, jev_check is a Noul yes/no, jev_route is a Choice pick, and jev_score rates an ordered rubric. A bundled jev:jev-questions skill covers question design.

How do I install and configure it?

Run hermes plugins install ajensenwaud/hermes-jev-plugin, then answer the TYPESAFE_API_KEY prompt (saved to ~/.hermes/.env) or set it yourself. Settings live under plugins.entries.jev.settings: model, timeout_seconds, and max_questions, defaulting to jev-latest, 30, and 20.

What license is it and how is it tested?

MIT. tests/agent_test.py is a stdlib-only harness that drives real hermes -z sessions and checks the JSON each tool returns; probabilities drift about 0.02 run to run, so assertions use tolerance bands.

CommunityJev Agents & MCP

Hermes Jev Plugin

Hermes Agent plugin exposing Jev as four coding-workflow tools for parallel judgments, binary checks, option routing and rubric scoring, with a bundled question-design skill.

See highlights

Open github.com

Stars: 9
Language: Python
License: MIT
Published by: Community
Author: ajensenwaud
Last verified: Oct 2026

ajensenwaud/hermes-jev-plugin

User repository on GitHub

View on GitHub

TypeSafe Jev (System One) decision tools for Hermes Agent: jev_check / jev_route / jev_score / jev_evaluate

Forks: 3
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

TypeSafe Jev is a "System One" decision model: it evaluates typed questions (Noul / Choice / Score) against a state and returns structured answers with calibrated probabilities. This plugin exposes it to Hermes Agent as four tools, focused on coding-agent workflows. The bundled skill jev:jev-questions (load with skillview("jev:jev-questions")) covers atomic question design, state preparation, and confidence gating. (Or clone the repo into /.hermes/plugins/jev/ and run hermes plugins enable jev.)

Quoted from the project's README. Read the full README.

Tagscommunity jevbest catalog-import awesomejev

Works forJev for Classification Jev for Routing Jev for Ranking Jev for Evaluation

Used inJev for Software & DevTools

Languages: Python

Highlights

Four tools cover mixed evaluation, Noul checks, Choice routing and Score rubrics.
Bundled skill documents atomic questions, state preparation and confidence gating.
Recipes map raw errors and diffs to retry, escalate, block-merge and severity calls.
Stdlib-only harness drives real Hermes sessions and asserts calibrated bands.

Watch out

Probabilities drift slightly run to run, so the harness asserts tolerance bands rather than exact values; a live TypeSafe key and an installed, enabled plugin are required for the tests. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

langchain-skill-router

LangChain deepagents middleware that replaces SkillsMiddleware with per-turn judging, loading only the SKILL.md files a turn needs so large catalogs stay out of the prompt.