Jev: one judge call, or twelve dimension scores? I measured both on three tasks

Comprehension was not the bottleneck, the decision format was. On a 12-option ledger classification the single call scored 0.400, meaning it understood the words and still could not pick one of twelve labels in one shot. Fourteen dimensions with locally fitted weights reached 0.911, and stacking n-gram class probabilities on top reached 0.9695.

Only when the single call is already weak. On Japanese NLI the direct call scored 0.837 and the 14-dimension model with fitted weights reached 0.908. On easier tasks the pipeline lost ground, and the general rule is that splitting costs 1.6 to 2.3x the tokens for a worse error profile: on hard-benign guardrail text it flagged 37.2% against 1.5% for the single call.

The run was 25,174 calls, 5,477 test rows and 34.1M input tokens for $1.43, extracted in 9 minutes 14 seconds with no failed calls. But output tokens are left unpriced, so the stated cost is input-only. Budget for output before repeating this at volume, and write a free local baseline first: a character-bigram naive Bayes tied the dimension model on one task with zero API calls.

Not reliably. On one 300-row task the direct call reported confidence of 0.9 or higher on 126 rows, 42% of the set, and was only 72.2% accurate on exactly those rows. Confidence and correctness came apart once the task got hard, and feeding the probability back as a thirteenth input did not help, scoring 73.0% against 74.0% without it.

Differences used McNemar exact tests plus a 20,000-sample paired bootstrap, across random 5-fold and grouped template-family folds. The author recommends designing the split before the dataset and printing the majority baseline first.

A direct choice call with 12 account options scored 0.400, fourteen dimensions with fitted weights scored 0.911, and stacking 12 n-gram class probabilities beside those dimensions reached 0.9695.

Character 2-gram, character 3-gram, word-bigram naive Bayes, and majority - all free and local. On one task a character-bigram naive Bayes tied the full pipeline at 74.0% with zero API calls.