Back-office decisions have a counterparty. A wrong label is not just a bad answer; it is a payment sent, a claim delayed, an invoice filed against the wrong supplier, or a patient routed to the wrong queue. That changes the design target: the model should narrow the choice, and ordinary code should own the action.
The four systems below cover desktop bookkeeping, procurement screening, insurance claims, and clinical intake.
The accounting clerk: the model chooses, code acts
jev-clerk works a real Mac desktop as a bookkeeping clerk. It opens a supplier invoice in Preview, reads the screen as OCR lines, types the invoice into Frappe Books, saves, submits, and files the PDF. Jev answers two multiple-choice questions per step: which kind of action from a closed list of twelve, and which line to click.
A System 2 model never touches the screen. After every ten invoices, Claude Fable 5.1 reads the decision log and rewrites the playbook (question wording, lessons, typing formats, waits, and reflexes), and code validates every rewrite so the action list stays closed. In the recorded run, 30 real invoices worked through three times used 1,141 Jev calls for $0.24 plus 978 seconds of System 2 thinking. Correct bookings on USD invoices went from 0/9 under playbook v0 to 9/9 by v6 and v8. Block 4 regressed to 0/9 after a rewrite, and two block-8 misses were an amount-parser bug reading the year in a due-date string, not a Jev error.
The referee is the accounting database, opened read-only: an invoice counts only when a submitted Purchase Invoice with the right supplier, date, and amount exists. A decision cost about $0.0002 at 0.35 seconds. The caveats: one recorded run on one Mac, an engine that changed mid-run, and two restarted blocks. The human review boundary is enforced by code, not intentions.
Procurement: indicators in code, probabilities in the model
sap-s4-antifraude-jev flags suspicious purchase orders in SAP S/4HANA. ABAP computes risk indicators, a Fiori Elements app sends five Noul questions, one Score, and one Choice per order, and ABAP (not the model) assigns the final class. The thresholds are a documented uncalibrated hypothesis: maximum risk below 0.20 releases, up to 0.60 routes to approval, and above that blocks.
The measured cost was $0.0000913 per seven-question evaluation, about $0.09 per thousand orders. It runs in shadow mode only: evaluations are logged and no order is ever blocked. Blocked vendors are a hard ABAP rule that bypasses Jev entirely. Supplier names, supplier codes, user IDs, and order numbers never leave the system; only days, ratios, counts, and flags are sent, and an automated test enforces the rule.
This is a proof of concept, not a production control. There is no license file, the Z views lack authorization checks, and the thresholds need calibration on real history before anyone acts on them. The architecture is the transferable part: deterministic indicators and hard vetoes in ABAP, probabilistic judgments in Jev, and a shadow period before the model is allowed to influence a real order.
Claims: screen, don't decide
claim-triage reads a first-notice-of-loss narrative and a document checklist rather than a transaction ledger. It classifies the loss type, scores documentation completeness, and flags fraud indicators in one request. Suspicious claims go to an investigation queue; the tool never denies a claim or sets a payout.
Every decision stores the narrative state, the question set, the answers, the model version, and the thresholds applied, so a later review can reconstruct why a claim was routed the way it was. Because Jev is text-only, attachments are reduced to extracted fields and a short checklist; image inspection is left to a separate system. The caveat is the scope: it is a screening aid for adjusters, and the fraud flag is a routing signal rather than a determination.
Intake: keep the answer space administrative
intake-router routes a patient's reported reason for contact to a care pathway and flags whether the message needs a same-day response, in one request. The answer space is deliberately administrative: no question asks for a condition, a medication, or a dosage. Every same-day flag goes to a triage nurse, and anything below the confidence floor goes to a person.
The repository ships intake messages with embedded instructions so prompt injection can be tested before routing is influenced. It is not a medical device and is not validated for clinical use. The design lesson is broader than healthcare: when a model is near a regulated decision, shrink the question until it is only asking for a queue assignment. A confidence gate is what keeps the model from opening a lane of its own.
What the workflow evals actually measure
The WorkflowEvals code runs four published workflows against Jev and other providers: invoice processing with 150 cases, customer service with 204, agent traces with 111, and security incidents with 240. Inputs and reference labels download from Hugging Face, the default model is typesafe:jev-1.13.0, and the harness is Apache-2.0, Python 3.13, and uv. It reports exact_actions and primary_action, and can score a run against another run with --reference. The README says the workflow code is assumed correct: it compares models, not the questions.
The published workflow evals report Jev averaging 67.8% accuracy at $0.0004 and 0.4 seconds per case across the four workflows, against consensus labels from GPT-6 Astra and Claude Fable 5.1 at high thinking. Every model is more accurate, cheaper, and faster in a workflow than with the same policy as a prompt, on average. Invoice processing is Jev's weakest workflow at 61.8%, against 79.1% for "sol."
These are vendor-run evals. The reference labels come from two frontier models, and the harness and label correctness are assumed rather than independently audited. The jev-ai.org audit adds a useful outside read. The authors run a Jev-based playground, so they have an interest, but they note that against similar-accuracy opponents the gap is smaller than the headline multiples: about 76× cheaper and 25× faster than "terra," or 8× cheaper and 32× faster than "luna." Their own six production-shaped calls cost $0.000023–$0.000038 each, with a median round trip of about 310 ms. They also flag invoice processing as the weak spot and treat it as a reminder to test document-heavy workflows on your own data.
Thresholds and the cost of a wrong route
The confidence-gated routing pattern gives the vocabulary. Any intent below 0.6 confidence routes to a person before the action is considered. A balance check is safe at 0.6 because the worst case is a wrong read-out; approving a transfer needs more than 0.85, or the user confirms first. Thresholds are set per action by the consequences of a wrong classification.
The caveat is the one to print above the turquoise: confidence is not permission. A high-confidence answer can still be wrong, so irreversible actions keep hard limits in code beside the model. In the SAP example that is the blocked-vendor rule; in the accounting clerk it is the read-only database referee; in claims it is the rule that the tool never denies or prices.
Audit the whole chain
The jaggedness list supplies the domain guardrails. Dates are read as text, so invoice due dates and claim dates should be extracted as components and compared in code. Counting is unreliable, so do not ask the model how many line items match. Structural invariants do not hold, so do not assume two separate probabilities compose cleanly.
A back-office pipeline should log four things for every decision: the state, the questions, the answers with probabilities, and the action code took. The records article covers the document and inbox versions of the same pattern, and confidence gates in production covers how to set the thresholds. The model is the cheap part; the audit trail is the product.
