Local Laya vs hosted Jev

No. The held-out replay pits local Laya against the author's own deterministic router; hosted Jev was never an arm. The author says outright that whether Laya beats Jev in general is a separate question he did not test, so read this as one Mac Studio deployment, not a head-to-head.

It is normalized entropy over the option distribution. The author is explicit that it is not the probability the downstream task will succeed. He picked thresholds on a separate development set, and would want fresh evidence before reusing them on a different model roster.

Not really. About 421M parameters on a ModernBERT-large encoder with a 1,024-token limit, trained on four synthetic workflows: agent observability, customer service, invoice processing and security incidents. Routing is absent from that list, which the author gives as a reason to test rather than assume.

More than the original brief assumed. Plain AutoModel loads only the encoder shape, the typed checkpoint is its own repository, and the custom decision head has to be reconstructed from the published runtime. You also inherit accelerator compatibility, startup time and process supervision that a hosted API absorbs.

A Mac Studio with an M3 Ultra and 256 GB of unified memory, and he says that hardware is part of the result, so a smaller machine needs its own memory and latency tests. He did not calculate a break-even and offers no savings number: privacy and control were the decisive reasons and cost was a side effect.

It renders the question, each option and the state into a single token sequence with a marker before each option. A bidirectional encoder reads the whole sequence, a small decision head scores the marker positions, and a softmax turns those scores into a distribution over options. Several questions against one state batch into one call, and there is no autoregressive output anywhere to parse.

TypeSafe lists $0.042 per million input tokens with no output-token charge, and says Jev handles choices with up to 255 options. Hosted also means nobody babysats model files, accelerator compatibility, startup time or process supervision, which the author counts as real work he chose to own rather than a free saving.