Question 1

How is this different from calling the Jev API in a script?

Accepted Answer

The signature file separates the reusable judgment definition from the data: state mapping, questions, and output columns live in JSON you can share, while run-dataset handles per-row calls, Excel output, and low-confidence flags.

Question 2

Can it score full documents instead of spreadsheet rows?

Accepted Answer

Yes. Point the input at one file for a single result row or at a directory for one row per text or Markdown file, using the file-content placeholder for text and the filename placeholder for the source label.

Question 3

What trips people up on the first run?

Accepted Answer

Column placeholders must match input headers exactly, Python with requests and openpyxl must be on PATH, and the key must be exported before running. A Summary sheet then reports rows processed, errors, and flagged rows.

Question 4

Can I trust the batch output blindly?

Accepted Answer

No. Output appends confidence columns and a flag-for-review marker with a threshold you define, so low-confidence rows route to a human instead of passing silently into downstream decisions.

Question 5

How do I install and load the plugin?

Accepted Answer

Run /plugin install with the path to make it permanent, or pass cc --plugin-dir for a single session, then open /plugin to confirm it lists with two skills, create-signature and run-dataset. The host can be Claude Code, Claude or Cowork; Codex support is described as coming soon rather than shipped.

Question 6

Can choice criteria be more than a single sentence?

Accepted Answer

Yes. A choice criterion can be a plain string or a structured object with what, not_for and examples, which is how the travel-review signature separates praise from balanced reviews without wording collisions. A noul takes true and false keys, a score takes an ordered list of level descriptions, and all three question types run in parallel for each row.

Question 7

Which signatures ship as starting points?

Accepted Answer

Four in examples/: travel_review for column-based sentiment, aspect, traveller type and recommendation strength; llm_guardrail for jailbreak detection, which rule was violated and severity; support_agent_review for text files, auditing agent traces for resolution, policy adherence and CSAT; and document_review for papers and reports across methodology, argument, evidence and originality.

TypeSafe Jev Plugin

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev community