Skip to content

Dohnuts.cpp

Native C++ inference for Dohnuts on llama.cpp that runs the Qwen3.5-based decision model on CPU, scoring candidate markers with a scalar head in one forward pass.

Stars
11
Language
C++
License
APACHE-2.0
Published by
Community
Author
DreamBlooms
Last verified
Oct 2026

About this project

Native C++ inference for Dohnuts, built on llama.cpp. It runs the released Qwen3.5-0.8B base with the merged Dohnuts LoRA, scores candidate markers with the scalar decision head, and returns probabilities from a single forward pass. No GPU, no generation.

Quoted from the project's README. Read the full README.

Highlights

  • Serves /v1/systemone for multi-question decisions plus /predict, /health, and /v1/models with optional API key and CORS controls.
  • Default CPU-only build with optional CUDA, Vulkan, ROCm, and Metal backends plus MinGW cross-compile for Windows.
  • One image per request supported via separate vision file, resized and encoded once then shared across questions.
  • Shared state prefixes decode once across candidates and persist in a bounded LRU shared by side-model profiles.

Watch out

CPU prefill is slow and quantization shifts probabilities, so use higher-precision files when exact numbers matter. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

38GitHub stars
Python memory filter that judges every stored memory against the current request with one calibrated yes/no per memory in a single Jev request, keeping everything above a threshold.
Jev Tools & CLIs#community#jevbest#catalog-import
46GitHub stars
Browser app for systematic-review extraction that opens a trial report with supplements and returns verbatim quotes with file and place, stored locally as projects holding studies.
Jev Tools & CLIs#community#jevbest#catalog-import
34GitHub stars
Pure-HTTP automation for the TypeSafe register, magic-link login, onboarding, and API-key creation flow, with pluggable temp-mail backends, concurrent batch runs, and key verification exports.
Jev Tools & CLIs#community#awesomejev#catalog-import
36GitHub stars
Python toolkit plus 0.8B multimodal model that takes a message, document, or image and returns calibrated choice, yes/no, and score answers in one forward pass for local decision workloads.
Jev Tools & CLIs#community#awesomejev#catalog-import
39GitHub stars
Video tool where Jev scores subtitle segments and code assembles a short highlights cut plus a de-watered full cut with timestamped summaries.
Jev Tools & CLIs#community#jevbest#catalog-import
30GitHub stars
Forkable weekday radar that asks Jev one yes-or-no question per interest over each new arXiv announcement and publishes a page plus RSS feed.
Jev Tools & CLIs#community#jevbest#catalog-import

From the Jev guides

Original write-ups on what this entry is for.

The request shape barely changes across surfaces; the surrounding contract does. When to call the API directly and when a gateway, a Worker binding, or a router entry earns its extra hop.
Jev Tools & CLIs#gateway#vercel#cloudflare
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Jev labels a record that already exists. Code moves, flags, or holds it. How the document, mail, and transfer tools keep the action out of the model.
Jev Tools & CLIs#classification#email#security
Read article
Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai