Tinyjev

Use the 0.6B build for in-process MLX or PyTorch inference and the 4B build inside Ollama for longer context. The digest reports the 0.6B near one hundred milliseconds per short question on a base M1.

It scores only the options you supplied and stops; it cannot return anything else. State can be a string, dict or list, with objects flattened keeping field names.

Yes. The server speaks the System One request shape, so existing clients work unchanged. Keep it on loopback since it has no authentication.

Load with 8-bit quantization on Apple Silicon: the digest reports identical accuracy to fp16 with lower latency, while 4-bit costs about two points on the transfer dev set.

Either. TinyJev 4B v2 needs Ollama 0.35.1 or newer, pulled as parable/tinyjev at about 4.5 GB, and is reachable over /v1/systemone on localhost:11434 or through tinyjev.load in Python. TinyJev 0.6B is 596M parameters and 1.2 GB, runs in-process via pip install 'tinyjev[mlx]' on Apple Silicon or 'tinyjev[torch]' elsewhere, and downloads once before working offline.

Choice takes 2 to 255 named options with optional descriptions, noul takes no criteria, and score takes ordered levels from low to high. One forward pass scores all of them, which is where the batching advantage comes from, and the 4B v2 build reads up to 16k tokens per request.

MIT. The project requires Python 3.9 or newer and ships TinyJev-0.6B and TinyJev-4B weights on a Hugging Face collection, with the 4B build also published to Ollama as parable/tinyjev. Everything runs offline once the weights are local.