Question 1

Is this just asking one big LLM to review the document in a single prompt?

Accepted Answer

That is the failure it names: generative models given a whole document and a rule list skip lines, miss rules and report wrong line numbers. Because Jev answers discrete semantic questions with typed probabilities instead of generating prose, every rule runs against every line independently. Lines are grouped into batches of up to 255 with plus or minus 10 lines of surrounding context, so boundary lines and sparse edits keep their neighbors.

Question 2

What happens if the model silently drops a question mid-batch?

Accepted Answer

Execution halts immediately rather than caching a partial answer. Every batch response is verified for complete answer-to-question parity, and any dropped question stops the run before anything is cached. Gaps between distant regions are marked with an omitted-lines marker so line structure stays uniform for the model, and padding lines are non-evaluated context that add no question cost.

Question 3

How much will grading a long article actually cost me?

Accepted Answer

Cost scales with rules multiplied by non-empty lines. The 16-line example against 53 rules is about $0.0053, 773 questions in 49 calls. A 2,100-word article at 300 lines against 42 rules is about $0.076, 12,600 questions in 84 calls. Re-runs after editing cost only for changed lines, since unchanged lines resolve from the content-hash cache with zero API calls.

Question 4

Will it flag innocent lines and cost me hours fixing phantoms?

Accepted Answer

It flags candidates, not verdicts. The documented workflow is a two-step loop: grade with the CLI, then paste the output to an agent that distinguishes real violations from false positives and writes concrete replacements. The entry caveats expect a human or agent pass to filter results. Custom rulesets are plain Markdown or JSON and validate offline with --check before you spend anything.

Question 5

How does incremental caching keep repeat runs cheap?

Accepted Answer

Line evaluations are cached by a hash of the line's content and the rule's criteria. Unchanged lines resolve from cache with zero API calls and zero cost, and only new or modified lines are batched to the model. Changing a rule's criteria invalidates its cache; --no-cache bypasses it and --cache-dir moves it.

Question 6

How does context-window padding work?

Accepted Answer

Each evaluated line is padded with up to 10 lines of surrounding text. Targets within 21 lines of each other merge into one region, gaps between distant regions are marked with an omitted-lines placeholder, and padding lines are non-evaluated context so they add no question fees.

Question 7

How do I write custom rules and validate them offline?

Accepted Answer

Write rules in Markdown with -r ./my-rules.md or as JSON. Rules ask plain-text questions about a single line or the whole document and are evaluated against criteria you define. Run with --check to validate a ruleset offline before spending any API calls.

Slop Grader

About this project

Highlights

Quickstart

Questions about this entry

From the Jev guides

From the Jev AI model community