Question 1

Is that 89% topic accuracy anything like my traffic?

Accepted Answer

It is 80 labelled prompts on a pinned decision-model version, and the README links a fuller report rather than presenting it as a general result. Topic accuracy came out at 89% and the route matched human labels 72% of the time, which means roughly a quarter of prompts land somewhere a human would not have put them. That matters if you branch on the topic label downstream, so treat the floor as a starting point rather than a settled number.

Question 2

What happens to my prompts, keys and private data?

Accepted Answer

The gateway binds to 127.0.0.1:8787 and expects your OpenRouter key in .env. A pre-check screens for secrets, PII and request size before a request is forwarded, and private: local_only restricts those requests to local models or refuses them. Body text is still sent upstream whenever routing picks a hosted model, so this is a local router with a privacy pre-check, not a boundary against the providers themselves.

Question 3

Will Claude Code and OpenAI clients work without changes?

Accepted Answer

Yes, by environment variable. It serves /v1/chat/completions and /v1/messages, so you point ANTHROPIC_BASE_URL at the local address with ANTHROPIC_MODEL=auto and ANTHROPIC_SMALL_FAST_MODEL=auto for Claude Code, or OPENAI_BASE_URL with model "auto" for any OpenAI SDK. Responses carry X-Router-Model, -Reason, -Topic, -Complexity and -Risk headers so you can see what was picked and why. There is also an MCP server exposing route, delegate and feedback tools.

Question 4

How do I know the quality floor is set right for my workload?

Accepted Answer

You edit config.yaml, and the README documents no automatic tuning beyond cmd/refit, which re-fits model skills from logged decisions, checks and ratings and compares them against peers at the same difficulty. Until you have that data, start conservative and read the dependency-free dashboard, which breaks down spend, savings, routing reasons, latency and escalations per model. Load penalties and daily budgets are part of the cost estimate rather than afterthoughts.

Question 5

What four questions does the decision model answer?

Accepted Answer

One call answers topic, complexity, risk, and private-data questions about the prompt, then plain arithmetic over config.yaml picks the cheapest System Two model that clears your quality floor. There is no LLM-as-router prompt, and the X-Router-Model, -Reason, -Topic, -Complexity and -Risk headers show what was chosen and why.

Question 6

What does the dashboard show?

Accepted Answer

A built-in, dependency-free dashboard breaks down spend, savings, routing reasons, latency, and escalations per model. Decisions, checks, and ratings are logged to decisions.jsonl, and the refit command re-fits per-model skills from that log against peers at the same difficulty.

Question 7

What is check-and-escalate, and what does it cost?

Accepted Answer

It is optional. One yes/no call judges the finished answer, and a weak one is re-sent to a stronger model exactly once. Because it adds a second model call to every turn, the reported cost saving assumes you leave it off.

System One Router

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community