The hosted price is easy to read: $0.042 per million input tokens, output free. The self-hosted price is a stack of costs that never appear on a pricing page: hardware, utilization, operations, evaluation, and the engineer time to keep the whole thing working. Comparing the two on token price alone is how teams convince themselves a self-hosted model is cheaper when it is not.
This is the TCO model for a decision workload, with the variables that actually move the answer.
Start with the workload
A decision model's cost profile is unusual. The output is tiny (a label, a level, or a probability), so output pricing is negligible. The input is the state, and the state can be large. That makes the input token count the dominant variable, both hosted and self-hosted.
The models guide documents the hosted side: $42 per billion input tokens, 64k context, dynamic rate limits. The jev-ai.org audit measured six production-shaped calls at $0.000023 to $0.000038 each, with a median round trip of about 310 ms. Those are small numbers, which is why the comparison is sensitive to volume.
The first TCO question is not “what does the model cost?” It is “how many calls per month, at what average state size, with what burst pattern?”
The hosted side
Hosted cost is the simplest line: input tokens times price, plus any platform or gateway fee, plus the cost of retries and failed calls. The workflow evals put Jev at $0.0004 per case across four workflows, which is a useful unit for back-of-envelope math.
The hidden hosted costs are operational, not token-based:
- Integration and SDK maintenance.
- Retry and rate-limit handling.
- Evaluation and monitoring.
- Vendor risk and price changes.
- Data retention and compliance review.
The enterprise page lists SOC 2 Type II, zero data retention, and higher rate limits. Those are real costs to verify, but they are usually smaller than the engineering cost of running your own model.
The self-hosted side
Self-hosted cost has five components:
| Component | What drives it |
|---|---|
| Hardware | GPU model, VRAM, utilization, depreciation |
| Serving | Engine, batching, concurrency, failover |
| Operations | Upgrades, monitoring, on-call, capacity |
| Evaluation | Re-certification whenever the model or data changes |
| Opportunity cost | The engineers who could be building the product |
The Soro-Kev-4B card is a useful data point: NF4 4-bit serving at about 4,048 MB of VRAM, fp16 at about 8,200 MB, median prefill latency about 413 ms on an RTX 4050 Laptop GPU or a Kaggle T4. The Train-Jev 27B LoRA needs an 80 GB GPU and reports a single-request p50 of 82 ms on an H20. Those are very different hardware tiers.
The Kev-4B MLX FP16 package is a reminder that the artifact is only part of the stack: it repackages the backbone with the adapter merged, keeps the pointer head in float32, and ships SHA-256 digests for every file. Someone has to own that packaging when the base model updates.
A simple break-even model:
monthly self-host cost =
(GPU hourly cost × 730 × GPU count)
+ (engineer hours × hourly rate)
+ monitoring and storage
+ evaluation time
Then divide by the number of decisions per month. If the result is above the hosted cost per decision, self-hosting is a hobby, not a saving.
The utilization trap
GPUs are cheap per hour and expensive per idle hour. A hosted API bills per token, so it scales to zero. A self-hosted GPU does not. If your workload is bursty (a few thousand decisions in the morning, silence overnight), the self-hosted cost per decision can be many times the hosted cost even when the hardware is modest.
The S1 LLM Auto-Router shows the other end of the spectrum: a CPU-first ONNX model at about 14 ms p50 on four EPYC cores. That is a different TCO shape entirely, and it may be the right choice for a high-volume, low-complexity gate. The model routers article covers the routing side.
The costs that are easy to miss
Three costs dominate the surprises:
Evaluation. A self-hosted model needs a labeled set, a calibration check, and a re-run whenever the checkpoint, the data, or the policy changes. The measuring Jev article and the calibration article cover the work. It is not a one-time cost.
Failure handling. A hosted API gives you retries, rate limits, and a status page. Self-hosted, you own the queue, the circuit breaker, and the on-call. The retries article covers the mechanics.
Lock-in. A hosted model can change its price or deprecate a version. An open model can change its license or stop being maintained. Both are risks; they are just different risks. The reproducibility article covers the pinning side.
When self-hosting wins
Self-hosting wins when at least three of these are true:
- The workload is steady and high-volume.
- The data cannot leave your environment.
- The latency requirement is tighter than the hosted p95.
- The model is small enough to run on hardware you already own.
- You have an evaluation harness and someone to own it.
It loses when the workload is bursty, the model is large, or the team does not have the operational capacity. The hosted-vs-self-hosted decision is a capability question before it is a cost question.
A short checklist
- Model cost per decision, not cost per token.
- Include retries, failed calls, and evaluation in the hosted side.
- Include hardware, utilization, operations, and opportunity cost in the self-hosted side.
- Use your real state size, not a demo prompt.
- Test the burst pattern, not just the average.
- Count the engineer time to keep the model current.
- Re-run the comparison when the hosted price or the model changes.
The hosted price is a number. The self-hosted price is a system. Compare the systems.