Question 1

How is this different from a router with five models pinned in a YAML file?

Accepted Answer

It reads OpenRouter's live catalog — hundreds of models — on every routing decision behind a short cache TTL, and nothing is hardcoded, so a newly listed model is picked up without editing config. Jev only classifies the request: domain, complexity, needs_long_context, needs_vision and latency_sensitive. It never sees the model list, and ranking the surviving candidates is deterministic code in the library.

Question 2

Is the 8/8 result a real benchmark?

Accepted Answer

No. The README calls it a small, directional harness of eight hand-labeled prompts across four domains, and says it demonstrates the pipeline works end to end against real APIs rather than making a rigorous cost-savings claim. Domain confidence ran 0.93 to 1.00 and the two genuinely hard prompts escalated to frontier reasoning models, but nothing there measures whether the cheap choices were good choices.

Question 3

What happens to my prompt text, and what if the classifier call fails?

Accepted Answer

The prompt goes to TypeSafe's Jev API for classification, and to OpenRouter only when you call .complete(), which sends it to whichever model was chosen. Fetching the catalog carries no prompt data at all — it is a plain GET of OpenRouter's public model list. On a classifier failure the router falls back to rank_cheapest_first, a pure cost ranking with no invented classification, configurable as fail_open or fail_closed.

Question 4

How are the weights set, and how do I reproduce the evaluation?

Accepted Answer

Current defaults are weight_complexity_match 0.9, weight_domain 0.1, confidence_threshold 0.5 and a 600-second catalog cache TTL. Running the evaluation needs both keys: pytest alone covers unit tests with no API access, a TypeSafe key adds the real-Jev integration test, and python eval/routing_eval.py is the manual billed sanity check across the labeled prompts.

Question 5

Is it a library or a proxy?

Accepted Answer

A Python library, not a proxy: pip install it, import model_router, and call a function, with no server process to run and no client to repoint at a different base URL.

Question 6

What five things does Jev classify?

Accepted Answer

domain, complexity, needs_long_context, needs_vision, and latency_sensitive, in one call of about 100 ms; Jev never sees the model list, which is ranked by a separate deterministic scoring function.

Question 7

How are hard constraints applied?

Accepted Answer

RoutingConstraints filters the catalog before Jev is called, dropping models that fail needs_vision or min_context_tokens, so an impossible constraint never wastes a classification call.

Jev Model Router — lucianfialho

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev community