Question 1

Is this a library I can depend on, or code I have to copy?

Accepted Answer

Code you have to copy. There is no package to import here, just four folders, each with one small file holding the Jev call, a demo.py that wires it in, and a README with the snippet and run command. The guard bodies are deliberately short and the README's own framing is that a guard is a dataclass overriding one hook and asking one question, with only three things to decide. Copying one file into your project is the intended usage.

Question 2

Will putting a Jev check on every tool call be slow or expensive?

Accepted Answer

The README's position is that the call is cheap enough to do per prompt, per tool call or per game tick, which a chat model could not support. It puts the cost at about a thousandth of a cent and the latency at a few hundred milliseconds, and those are the author's figures rather than a measured table. Structurally the guard runs where you put it: the input guard is marked parallel so it does not serialize behind the model request, and the shell guard acts before the command executes.

Question 3

What happens when the guard is not sure?

Accepted Answer

The shell guard makes uncertainty an explicit outcome rather than a coin flip: the question is run it, block it, or ask a human, and deferring is handled by DeferredToolRequests so a run can pause and hand you a command. On the input side the threshold is a plain dataclass field defaulting to 0.75, and the answer is acted on directly. Flappy Bird pushes the same idea further, asking on every tick whether the bird should flap.

Question 4

Will these run unmodified in my own project?

Accepted Answer

Probably not, and the README gives you the run recipe rather than a compatibility promise: clone, export TYPESAFE_API_KEY, then uv run demo.py. Expect two edits. The snippets import InputGuardrail, ToolGuardrail and Coder from pydantic_ai_harness, which the README never explains installing, and they name 'anthropic:claude-fable-5' as the model, so you will need a model id your provider account can reach. Flappy Bird can drop Anthropic entirely with --offline.

Question 5

Where in a run can a Jev question actually go?

Accepted Answer

Six places. before_model_request, where you raise SkipModelRequest so no tokens are spent. before_tool_execute, where you block or pause for a human and the harness ToolGuardrail does the raising. after_run, where you send it back to the model and OutputGuardrail raises. after_node_run for loop detection, which the README lists as not here yet. Your own loop, which is what flappy_bird asks from. And a Pydantic Evals evaluator, which is what jev_judge uses to return a pass or fail number instead of a reason.

Question 6

What keys does each example need?

Accepted Answer

input_guard needs TypeSafe alone. shell_guard needs TypeSafe and Anthropic. flappy_bird needs TypeSafe and Anthropic, or nothing but TypeSafe with --offline, and the README is candid that it is the author shoehorning Jev into a game loop rather than a realistic integration. jev_judge needs TypeSafe, plus Anthropic too if you pass --compare. Every folder has the same layout: one small file with the Jev part, a demo.py that plugs it in, and a README.

Question 7

What does a guard actually look like underneath?

Accepted Answer

A dataclass overriding one hook and asking one question. The input guard has a threshold field defaulting to 0.75, reads the prompt out of the run context, calls system_one with a Noul, asserts the answer type, and raises SkipModelRequest carrying a short decline message when the probability reaches the threshold. Change the hook and the question and you have an output check or a loop detector. The question text is the whole program.

Pydantic Jev Examples

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev community