Mini-Vibe Check: TypeSafe's Jev Judged Everything I've Written in 0.7 Seconds

They are his own measurements rather than vendor numbers, which is the point of running the experiment. He sent 27 of his Every articles plus 10 deliberately AI-styled counterparts as 37 documents, asked the same 21 questions concurrently across all of them, and got 777 judgments back in under 0.7 seconds for an estimated quarter of a cent. Those numbers measure throughput and cost. They say nothing about whether the judgments were right.

Partly, by his own account. He browsed the articles it classified and says Jev correctly flagged pieces of his own that leaned more on AI, and the questions came from Every's own skill for AI tells, such as whether text repeats an idea without adding evidence or forces a symmetrical both-sides argument. He is equally explicit that he would want a more thorough accuracy check before production, and frames it as an early warning system where the alternative is not checking at all.

In the small defect test the entry records, Jev caught six of seven planted problems at 0.35 seconds per passage while Claude Fable 5.1 caught all seven at 8.8 seconds. Same corpus and same planted defects, so the trade is one missed problem against roughly 25 times the latency. That is a seven-item test from one afternoon, not a benchmark, and he ran eleven experiments overall.

Calibration is the stated training goal, but nothing here measures it. If the model flags 100 customer messages as a 90 percent chance of expressing anger, about 90 should express anger, and output tokens are not billed because they are said to be too cheap to meter. What the write-up lacks is any ECE figure or per-pattern false-positive rate, so a threshold on "probability of AI tells" still needs its own measurement before it gates anything.

A first-person experiment write-up by Mike Taylor, head of evals at Every and co-author of Prompt Engineering for Generative AI, published 2026-09-15 and updated 2026-10-07, with an eight-minute read. He ran eleven experiments to explore how a decision model behaves on everyday text, and reports both what worked and what he would not trust yet.

His framing is background automation: routing a customer service request, checking an invoice against a purchase order, classifying a transaction as fraud. You pose a plain-English question, even a fuzzy or subjective one, and get a probability back your code can branch on. TypeSafe calls the training approach Reinforcement Learning for Calibrated Decisions, so confidence should track how often it is right: of 100 messages flagged 90 percent angry, about 90 should be.

Because System One answers in parallel rather than generating tokens one at a time, Jev answers multiple questions in fractions of a second and for a fraction of an LLM query's cost. TypeSafe prices it at $42 per billion input tokens where most LLMs are priced per million, and does not charge for output tokens at all, described as too cheap to meter. The article's own 777-judgment run came back in under 0.7 seconds for an estimated quarter of a cent.