Question 1

If advisory is the default, what actually stops a destructive bash command?

Accepted Answer

Nothing by default, and the README is explicit that advisory feedback arrives after execution and cannot prevent side effects. Blocking is the opt-in answer: set PI_JEV_CONFIG to {"mode":"blocking"} and qualifying hazards are refused with { block: true, reason }. Blocking still does not set terminate, so the agent gets a chance to revise rather than dying, and shadow mode records assessments without changing anything model-visible.

Question 2

A 750 ms budget per handler feels tight - what happens when a check times out?

Accepted Answer

The tool call is left untouched. Missing or blank key, bad configuration, authentication errors, rate limits, connection errors, malformed responses, unavailable context, timeouts and cancellation all return undefined, and a failure while recording an assessment or logging cannot block execution either. Each handler gets its own 750 ms best-effort budget (allowed range 1-5000 ms), propagates ctx.signal and disables SDK retries; a write can therefore cost two requests, and these costs accumulate.

Question 3

What is this extension running on my machine to grade my code?

Accepted Answer

Repository discovery spawns a child process, and the README documents the teardown: it explicitly sends SIGTERM, escalates to SIGKILL after a bounded grace period and waits for child exit where the OS can deliver those signals, though it cannot force termination of a process stuck in uninterruptible kernel I/O. That discovery work happens inside the same 750 ms handler budget, and session shutdown or reload cancels outstanding work.

Question 4

How much would this add to my per-step token bill in Pi?

Accepted Answer

The mechanics are documented and the cost is not. A write can trigger two requests, one pre-flight and one critic, Pi preflights sibling tools sequentially, and every phase shares one request across its questions - but the README publishes no token counts, no price figures and no measured end-to-end cost, so you would be estimating before enabling it across a whole session.

Question 5

How is it configured, and what are the defaults?

Accepted Answer

Set PI_JEV_CONFIG to a JSON object; omitted fields default to mode advisory, model from TYPESAFE_DEFAULT_MODEL then jev-latest, and a 750 ms total budget per handler. Unknown fields, invalid JSON, invalid modes and out-of-range values disable the judge instead of guessing a policy, and configuration is read at load so a change needs a reload.

Question 6

What do the eight checks actually ask?

Accepted Answer

Pre-flight uses Choice with true/false labels for destructive, credentials, outsideRepository and weakensTests, where a hazard qualifies only when the label is true and confidence meets the threshold (0.95, 0.95, 0.95, 0.90). The critic uses a Score rubric over style, implementation, errorHandling and taskScope, defaulting to confidence at least 0.80 and score below 1.00.

Question 7

What happens on a judge failure?

Accepted Answer

Missing or blank key, bad configuration, authentication errors, rate limits, connection errors, malformed responses, unavailable context, timeouts and cancellation all return undefined and leave the tool call untouched. A failure while recording an assessment or logging also cannot block execution, and missing credentials or any judge failure always leaves the tool unmodified.

Pi TypeSafe — twilwa

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev community