Question 1

It scored its own README against eight rewrites. Are those numbers real or self-serving?

Accepted Answer

They are real runs, and the losing drafts are pinned to commits so you can read them side by side: nine drafts through one group of 12 questions, each scored 3 to 6 times, with the top README at a 92.0 median against 90.8 for the previous version. Two details keep it honest. The ranges across six runs each did not overlap, and the README says plainly that a gap that small should be treated as a modest edge because Jev is probabilistic. It also states the numbers describe an earlier README, since the later install, agent-loop, app and MCP sections were never re-scored.

Question 2

Is a one-point score change signal or noise?

Accepted Answer

Run-to-run variation is the yardstick, and the README is blunt about it: a reworded sentence moves scores by less than that variation, so each revision should target the weakest questions, change something substantial, and be scored once. When the top two drafts sit within a point or two, the suggested move is to score each a few more times and switch the workspace to median ranking, which uses the middle run rather than the best one.

Question 3

Why does this need an OpenRouter key instead of a TypeSafe one?

Accepted Answer

The stated install requirement is an OpenRouter API key with access to Jev. The tool reads .env and .env.local from the directory you run it in, or the environment, and OPENROUTER_JEV_MODEL overrides the model, which defaults to typesafe/jev-1.13. A doctor command checks Node, the database, the key and OpenRouter before you start. The web app then serves locally on 127.0.0.1:4317 unless you pass a different port.

Question 4

Does it only work if an agent writes every draft?

Accepted Answer

No. It is aimed at people who let Codex or Claude Code edit their writing, but it ships its own editor for when you would rather write the next draft yourself. The pairing is a context file plus questions you wrote, such as a resume against a job posting, and the loop is the same either way: save a revision with a summary of what changed and its parent, score it, then rank the drafts overall or by a single question.

Question 5

What do the concepts mean before the loop makes sense?

Accepted Answer

A context is the text a document must satisfy, such as a job posting. A workspace is that context plus every revision of the document and all their scores. A document is one stored draft, numbered from #0 for the original, and numbers are never reused. An evaluation group is a reusable list of questions plus the scorer that folds them into one number: the default scorer is a mean, and a lower-is-better question counts as 100 minus the score.

Question 6

What happens if I edit the context or the questions after scoring?

Accepted Answer

Runs scored against an earlier context version are kept but marked stale, and rankings count only runs against the current version. Questions stay editable until a draft has been scored with the group; after that you fork the group to change them. Every reported change is signed, so positive means better even for lower-is-better questions, and the output names the three weakest questions along with the change against the parent draft and against the best so far.

Question 7

Where do the starter rubrics come from?

Accepted Answer

jev-score group create --template resume starts from a built-in evaluation group instead of a questions file, and jev-score group templates lists all eight: resumes, cover letters, college essays, cold emails, READMEs, landing pages, product specs and blog posts. The pairing is the same either way, so a template is a starting point you edit against your own context rather than a fixed rubric.

Jev Score

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community