Question 1

If Jev decides nothing, what is left of the plugin?

Accepted Answer

The deterministic layer, which is where the author puts the value. It derives facts in code: test command exit codes, which files changed and whether an untracked file was created, and whether the agent claimed a verification it never ran. The highest-value check is unverified_claims, where the agent says all tests pass and the recorded commands contain no test command at all. That contradiction is caught for free with no model involved. Jev is still called once per evaluable turn and logged, as the only record that could justify promoting a probability back into a decision.

Question 2

The README admits the probabilities swing by 0.04-0.11. Why call it at all?

Accepted Answer

Because the call is cheap and the author is honest about the noise: roughly 566-850 ms and about $0.000022 per request, with seven questions in one request. That is what makes the logging decision defensible, not a threshold decision. The README argues that thresholds placed inside the observed spread are partly deciding on noise, and that a before-and-after difference smaller than the spread is not evidence, which is why an apparent four-task improvement only survived on one task. Under policy_v 3, sampling determines which label a row gets rather than whether a turn is interrupted.

Question 3

Can this stop a turn that was actually fine?

Accepted Answer

Under policy_v 3, not on probability grounds. The README records that the only intervention that reached a user was logged as a false_block, that in eleven probability-triggered turns retrospective review found none of them necessary, and that the good_block verdict path has never been walked. The asymmetry is a documented design choice: finishing has a high bar and blocking a low one, with a wide default-pass band between them, on the reasoning that a false pass costs the user nothing they would not have noticed. The signal to watch is a false block on an obviously finished task.

Question 4

Why is this the opposite of Foreman, which stops and calls a human?

Accepted Answer

Different failure costs. Foreman is an unattended factory scheduler, so when its evaluation fails it must halt and escalate, because continuing blindly could damage a repository. This is assistant augmentation, so the correct outcome when its judgement fails is that DSH behaves exactly as if the plugin were not installed. That choice also explains the shape of the logs: most rows in early runs were blocking, which the author reads as evidence that the state was incomplete enough to judge and the questions were about things that did not exist, rather than as a threshold that needed moving.

Question 5

What are the seven questions Jev is asked?

Accepted Answer

requirements_satisfied, implementation_complete, verification_sufficient, blocking_issue_remaining, evidence_matches_claim, needs_more_verification, and ready_to_finish, each a noul question answered in one request.

Question 6

How is the call count kept to one to three per task?

Accepted Answer

Three mechanisms: a hard maxAssessments of three per user task, an evidence fingerprint that skips Jev when the material facts did not change, and skipping non-coding turns that make no tool call.

Question 7

What happens when Jev is unavailable?

Accepted Answer

Every failure is fail-open: a missing key, timeout, HTTP error, malformed response, or over-budget state is logged and does not intervene, and the deterministic checks still run, so Jev is never a single point of failure.

Dsh Completion Supervisor

Highlights

Questions about this entry

More Jev resources like this

From the Jev community