Sniff Test

That is the cheap half. Countable rules are regular expressions that run locally and send nothing: an en or em dash, three or more colons in a paragraph, uniform sentence rhythm across long paragraphs, and twelve words that turn up far more often in generated text than in human prose. The judgment rules are the other half, and those need a key.

One paragraph at a time, plus the wording of the judgment rules in force, sent over HTTPS to https://api.typesafe.ai/v1/systemone with your key as a bearer token; what comes back is one probability per rule. File names, file paths, and anything outside the text you pointed at are never sent. Nothing leaves before you answer yes, and countable rules never leave even when you decline.

No, and it does not try. The judgment model returns one probability per question and nothing else, and there is no general model in the loop. Output is one line per flag with file, line, rule id, how sure the checker is, and the message, with --format json for machines and snifftest rules to show which file each rule came from.

Because the judgment model answers about 0.5 on text it cannot read, and a flat middle number is the one failure that looks like a clean draft. Readings in that band are reported as no judgment and never become a flag or a pass. Above it, a judgment rule flags at 0.7 unless your ruleset or --threshold says otherwise, while a countable rule always scores 1.00 and says nothing about severity.

Countable rules are regular expressions: they run on your machine, cost nothing and send nothing. The judgment rules instead send one paragraph at a time to a hosted judgment model, only after you have consented. The countable rules run first whatever happens next.

Yes. A .snifftest.yaml in your project root extends the default ruleset and overrides shipped rules by id. A custom countable rule is a pattern plus a message, while a judgment rule needs a what, a not_for, examples and two criteria. A judgment rule with no not_for will flag things you did not mean.

Answers are cached under ~/.cache/snifftest or $XDG_CACHE_HOME, so an interrupted run does not repay for every paragraph. Each entry is named by a hash of the paragraph, the exact question wording and the model, and holds rule ids, probabilities, model name and date; the paragraph itself is never written. Entries expire after a fortnight, and --no-cache or SNIFFTEST_CACHE_DIR=off disables it.