Hearim

Prompting returns generated text you must parse. Here one question maps to exactly one next-token scoring call, with label logprobs normalized by conditional softmax and teacher-forced fallbacks when the fast path is unavailable.

It falls back to an inference probe that registers emitable labels, then uses text matching over top-k instead of ID requests. Recovery is bounded by the backend top-logprob cap, and reports carry a tokenizer_unverified warning.

One Jev request fans out to one upstream call per question, grouped by shared prefix on bounded lanes with prefix affinity. The prefix key hashes model, tokenizer, template, layout, and bytes only.

Build with go build, configure hearim.yaml with env-expanded keys, then run the probe command for your provider and model before serving. CPU-only mocks and go tests cover HTTP shapes without hardware.

go build -o hearim ./cmd/hearim, then copy hearim.example.yaml to hearim.yaml and export the keys from your secret store: OLLAMA_API_KEY for the backend and HEARIM_API_KEY for clients. Before trusting a backend, run the probe command for the provider and model, then start with ./hearim serve -config hearim.yaml -addr :8080. API keys are never put in the config file; environment expansion is supported.

The project's Korean name is the nominal form of a verb meaning both to count things one by one and to grasp something by thinking it through, which matches a gateway that turns understanding into counting label logprobs at one decode position. Two papers ship: one explains the conditional softmax reading, the tokenization boundary invariant and the silent failure modes, and one describes how to train a Jev-like model when a dedicated one is actually needed.

Per question the adapters try selected-token-ids first, then top-k logprobs with text or byte matching, then a teacher-forced label, then a constrained vocabulary mask, and finally full choice-text continuation with sum, token-mean or pmi normalization and a per-mode calibration temperature. Top-k N is auto-tuned per model across a ladder from 4 to 64, respecting each server's rejection cap, which is 20 on Ollama. In strict mode an unrecoverable candidate fails with 529 rather than being assumed to have probability zero.