Question 1

If the offline fixtures are hand-written, what does running this actually prove?

Accepted Answer

It proves the shape, not the accuracy. The probability distributions come from files the author wrote by hand, but choice, score and confidence are derived locally by the official formulas and match the worked examples in the docs. Request structure, response structure and the way code consumes an answer are identical in both modes. Whether Jev is right about your data is only answerable with your own key and your own inputs.

Question 2

Is it safe to run before I have an API key?

Accepted Answer

Yes, and that is the default path. With no TYPESAFE_API_KEY found, the demos run against fixtures. Key lookup takes the process environment first, then the project .env, then ~/.typesafe.env, and .env is gitignored. If you want to be sure a run is live rather than mocked, set TYPESAFE_MODE=live, which errors out rather than quietly falling back to fixtures.

Question 3

How much does a full live run cost?

Accepted Answer

The README reports 28 requests, 29,674 input tokens and $0.00125 for the whole suite, with per-call latency between 1.2 and 1.7 seconds. Treat that as one shape of workload rather than a price quote: the same estimator is used elsewhere in the repo and the author notes it underestimates token usage for Chinese text and JSON by roughly 2.5x.

Question 4

Will the six demos transfer if my questions look nothing like the fixtures?

Accepted Answer

The patterns are meant to. One demo fans out seven questions in a single request and the code reads 13 of 28 branches, so discarding 54% of the output is the expected behaviour rather than a bug. One scores five resumes across four dimensions with five inferences and derives four leaderboards, so you can change weights without re-running inference. The fixtures are editable precisely so you can watch a threshold change the branch.

Question 5

What is the API key lookup order?

Accepted Answer

First the TYPESAFE_API_KEY process environment variable, then the project .env, then ~/.typesafe.env. The .env is gitignored with a committable .env.example as template, and check_setup.py reports where the key was found and how long it is without printing it.

Question 6

What does check_setup.py send on its first live request?

Accepted Answer

One minimal request with a question for each of the three primitives, chosen so the answer is obvious, and it prints Jev's raw response. That single call is a few dozen tokens, well under a thousandth of a cent.

Question 7

What did the live run reveal about Jev 1.13 distributions?

Accepted Answer

They are extremely peaked, with many 1.00 and 0.00 values and confidence frequently at 1.0, which leaves the officially suggested cautious band nearly empty. Real uncertainty shows up as choosing a fallback option, or a Noul landing between 0.4 and 0.6, rather than confidence dropping to 0.7.

TypeSafe AI Jev Example

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community