Calibrating Jev as a Code Reviewer

Not safely. The README's central claim is that thresholds do not transfer when unrelated comments share a batch without file context, so a cutoff tuned in one repository can be meaningless in another. Every measurement behind the notes comes from a single Unity game repo.

No. The captured discussion is a couple of comments with no technical back and forth, and the substance lives in the linked GitHub repo. The thread does not document methodology, prompt wording, or any accuracy figure.

Context leakage. When unrelated comments land in one batch without file context, nothing ties a finding back to the code it came from, and that is what the author blames for the calibration loss. Which files he batches, and how, the thread does not spell out.

The post is framed against ordering.py, but the thread never names the baseline, the ranking metric, or a winner. Read it as an argument about prompt and batch shaping, not as evidence that another ranker was beaten.

In the repository, not the thread. The story URL points at github.com/Selmar/typesafe-jev-calibrate-for-code-review and the post sits at 2 points with no technical discussion attached, so the calibration notes themselves have to be read on GitHub.

Only your review code. The work is calibration notes on Jev comment rules, batching behavior, and a comparison against ordering.py, all of it sitting in a personal repository. Nothing in the capture suggests a change to the Jev API or a proposed upstream patch, so the thresholds are ones you tune for your own review setup.

Two rules. Keep comments about unrelated files out of the same batch, since the README blames the loss of calibration on exactly that, and treat any cutoff you tune as specific to the codebase you measured. Every number behind the notes comes from a single Unity game repository, so re-measure before reusing a value anywhere else.