Systemone — infinitylogesh

Mostly. The team measured its own rows on 2,000 typed decisions plus public suites, and Laya zero-shot reproduced its published figure, which cross-checks the scorer. The Jev rows are third-party published figures shown as indicative, not remeasured.

No GPU is needed in OpenRouter mode with an API key, serving hosted models alone or beside local upstreams. Local serving needs logprobs enabled in the upstream config, and only providers returning logprobs can serve.

Raw model probabilities run overconfident, so bench fits one temperature per question type and option count on labeled data. Serve applies the calibration files and re-reads them on change, falling back to the nearest option count.

Choice is capped at 26 options and score at 26 levels with a 422 beyond that, question order needs explicit chaining, and images take the chat endpoint path, which slightly changes the prompt shape.

The README picks by constraint. Gemma 4 31B for accuracy at 0.709 typed accuracy, Qwen3.5 for calibration out of the box since its raw ECE is 0.033 with no fitting needed, gpt-oss-20b for throughput at 189 requests per second and 42 ms per case, and gpt-oss with think: 64 when quality matters more than the extra 290 ms. Context matters too: gpt-oss read with no reasoning puts about half its mass on the last option regardless of content.

systemone rlcd prepare builds training data from the serving stack's own tokens and systemone rlcd train fine-tunes the answer-label distribution with LoRA on top of TRL, with --objective rlcd, proper or ce. On Gemma 4 31B it lifts typed-decisions accuracy from 0.702 to 0.79 and Brier from 0.111 to 0.041, and the README is candid that plain cross-entropy did as well as the RL objective. The fitted models are no longer zero-shot, which is why they are footnoted separately.

Four suites behind systemone bench, summarized by systemone report into a markdown table including Jev's published row. features is 31 request cases covering types, state shapes, six languages, depends_on, ask_if, think, images and validation. public is AG News, DAIR Emotion and SST-2. typed is LocalLLaMA/typed-decisions, 400 cases and 2,000 decisions, scored the same way as Jev's published numbers. latency is sequential p50 plus a concurrency sweep. Any endpoint speaking /v1/systemone can be benchmarked, and ,basic narrows it to Jev's core contract.