Jevify — fidecastro

Each answer is read off the next-token distribution with no generation or parsing, one request per question, and returned in Jev API shape so the unmodified typesafe-sdk works. A probe first verifies answer token ids per deployment and writes findings into the recipe.

Yes for serving: serve binds to loopback by default, requires an explicit host flag plus a bearer key for remote access, and every answer carries a block naming the readout method, latency, and recipe hash. Recipes are hashed YAML describing endpoint, template, and readout policy.

The documented arms range from single-digit milliseconds for in-process classifier and embedding kinds to tens of milliseconds for local GPU models and hundreds for remote vLLM setups. The in-process kinds are fastest and least accurate on the author's suites.

Copy the nearest recipe, set the endpoint URL and model name, and run the probe; in-process classifier kinds need the encoder extra. Eval runs a frozen suite with controls and writes a scorecard with command, hashes, and raw digests.

It measures the deployment rather than trusting a config: dialect (vLLM, llama.cpp or generic), context length, which logprob rungs it honours, whether thinking is really off, whether an image genuinely adds tokens, and how much of a warmed prefix a question reuses. What it finds is written into the recipe, so the next run reads a file that matches the server in front of it rather than what its name suggests.

uv tool install jevify or pipx install it, then probe a recipe and ask: jevify probe recipes/gemma-4-e4b-it.llamacpp.yaml, then jevify ask with --state, --choice name=question=opt,opt, --score similarly, and --noul, optionally with --image. State goes once and each question is a short suffix that reuses the server prefix cache, so a 32k-token state costs one warm request and then a few hundred milliseconds per question. The README quotes roughly 30 to 140 ms per question on a local GPU.

endpoint covers any OpenAI-compatible server with dialect extras for vLLM and llama.cpp, proven live with DeepSeek-V4-Flash on vLLM and Ternary Bonsai 2 27B, Gemma 4 E4B and Qwen3-VL-Reranker 8B and 2B on llama.cpp, for text and images. embedding covers a /v1/embeddings route or an in-process model, proven with Qwen3-Embedding-0.6B. encoder covers an in-process NLI sequence-classification head, proven with bart-large-mnli. rerank covers /v1/rerank and is fakes only.