Tax Doc Classifier — kyotofin

It is a strict one: a page counts as an error if the answer is wrong or its confidence is below 0.95. On 753 blank IRS forms the author reports 0 wrong labels and 38 strict errors, all of them low-confidence instruction pages, deep pages of corporate forms, and single schedules hedging against their parent. Per-page results are not committed.

Yes. The Backend interface is a single method that takes a state and questions and returns a distribution over each question's options, and jevBackend is only the implementation shipped. Anything returning calibrated probabilities over a labelled option set can implement it; the form descriptions are plain data.

It is reported as blank, not sent through OCR. The package is text only, so scans need OCR first, and only the PDF helpers require poppler's pdftotext and pdfinfo - classifyPage itself just takes lines of text. Scope is also federal forms and English only, with state forms detected by page kind but never identified.

The baseline was a Claude Sonnet classifier that sends each page as a PDF with the form registry in the prompt and never shares a prompt cache, so every page pays the full prompt. On the same bench pages and machine it reports $0.039 versus $0.00115 and roughly 3.3 s versus 0.5 s warm, naming 30 forms against 261.

You need Node 20 or newer and pnpm, poppler for pdftotext and pdfinfo (only classifyPage on lines of text works without it), and a TYPESAFE_API_KEY read from the environment by jevBackend() and never written to disk or logged. Then run pnpm install, pnpm typecheck and pnpm test.

pdftotext splits a page into header, body and footer; one request asks a 7-option kind and a 230-option form question, and only if the form is one of five corporate or foreign parents does a second small sub-question run. The result carries form, kind, formConfidence, gated and probabilities, and formConfidence is the minimum over steps.

Text only, so scanned pages need OCR first and a page with no text layer is reported as blank. Federal forms only, with state forms detected by kind but not identified, and English only. The confidence is calibrated on the shipped corpora, so calibrate on your own pages before choosing a gate. The code is Apache-2.0.