Question 1

Do I need labelled data before the probabilities are usable?

Accepted Answer

Not to get started. The zero-label L0 level drops the answer-flip rate under reversed option order from 0.230 to 0.073 and lifts accuracy from 0.747 to 0.803. The recommended rotation budget for any K-option choice reads 7.2 rotations instead of 18 at a certified 1% disagreement rate against the project's own full-strength readout, for 2.2x the decisions per second on vLLM with accuracy unchanged.

Question 2

Is the headline table a head-to-head comparison with Jev?

Accepted Answer

No. Those rows are Qwen3-8B on BANKING77 20-way with 300 test items, measuring order-flip rate, accuracy, calibration error and auto-decidable share against raw logits. The separate typed-decisions table puts its cells next to Jev's published 0.727 and fine-tuned Laya's 0.768, using each project's own published numbers, so treat it as a reference point rather than a controlled benchmark.

Question 3

What does the L2 head cost me to maintain?

Accepted Answer

It does not transfer: a new question or a different model needs new labels. Once fitted, only its feature mean and scale keep moving, re-estimated from unlabelled traffic. A reworded question dropped the Qwen3-8B head from 0.77 to 0.65 to 0.70, and 30 unlabelled requests brought it back to 0.74 to 0.75 against 0.77 for a full relabelled refit.

Question 4

Will it run without a patched inference engine?

Accepted Answer

Yes, on stock vLLM, because L2 reads a hidden state, so a pooling embed server plus a few kilobytes of head is the whole serving story; raw, L0 and L1 run against a normal generate server. A head fitted through transformers and served by vLLM answered identically in 99.0% of cases, mean absolute probability difference 0.0011 on BANKING77-20. fp8 quantization is available but not recommended: it buys latency and costs accuracy.

Question 5

What is the difference between the two ways in?

Accepted Answer

Decider is training-free and works on any open causal LM through HF, vLLM, SGLang or llama.cpp backends, with no training and no labels. Tacit uses self-distilled checkpoints from 1.7B to 9B on Hugging Face, trained so the model gives in one forward pass the answers it reaches when it reasons, with no human labels and no other model involved.

Question 6

What does adaptive mode actually do?

Accepted Answer

With adaptive enabled, a decision whose top two options sit less than tau apart in log-probability is answered again with thinking on, then read as a label distribution so it still cannot fail to parse. At most max_cot_share of the last cot_window decisions escalate however hard the traffic gets, each escalation is reported with route set to cot and the first pass kept, and tacit.stats counts them. The HTTP gateway listens on 127.0.0.1 with no authentication.

Question 7

What do the Tacit checkpoints actually score?

Accepted Answer

On the 231-item JevBench public set, Tacit-9B from Qwen3.5-9B scores 0.823 in one forward pass and 0.887 adaptive, and 0.727 to 0.755 on the 46,320-item bev-decision test split. Tacit-4B moves 0.723 to 0.758 and 0.663 to 0.704. Tacit-8B scores 0.736, Tacit-2B 0.671 and Tacit-1.7B 0.619 on JevBench with no adaptive column.

AnyJev

About this project

Highlights

Quickstart

Questions about this entry

Reactions & coverage

From the Jev guides

From the Jev AI model community