Question 1

How many API calls does a single sentence actually cost?

Accepted Answer

One per character, because each step is a single Choice request over the 28-option vocabulary. Defaults are a 60-character budget, three orderings averaged per step, which the README notes is free in requests, and beam width 1. Raising the beam keeps several candidates alive and costs one request per beam per step, measured at 25, 25 and 21 requests on the example prompt.

Question 2

Why do the options hold whole text fragments instead of single letters?

Accepted Answer

Because a classifier can only judge candidates that differ in meaning. The README calls this the single largest thing separating a working loop from a broken one: an option is the text you would have if you picked it, such as "my name is jev", never a description like "the next letter is v". Every request also carries the question, since with no target there is nothing to rank.

Question 3

Is there a temperature or sampling setting I can raise?

Accepted Answer

No, decoding is greedy with no temperature, because the README argues a character-level answer cannot recover from a wrong character, so randomness only relocates the failure. If greedy stalls on an attractive-looking character that leads nowhere, the lever is beam width instead: beam width 3 turned a 2-of-3 result into 3 of 3 on the example prompt, at roughly triple the requests.

Question 4

What failed attempts does the repo document?

Accepted Answer

archive/README.md holds what was tried and cut, with the numbers: the stop-mechanism comparison, why STOP shaping was dropped, and what temperature cost. The default test suite is offline. The one failure mode named in the main README is a character-level model locking onto a locally attractive character, such as the space after "capital was", which is what motivated beam search.

Question 5

How do I install and run it?

Accepted Answer

Create a uv venv and `uv pip install -e ".[dev,experiments]"`, then copy .env.example to .env and fill in TYPESAFE_API_KEY. Run `jev-lm` bare for a session, or `jev-lm --question "what is the capital of france"` to answer one and exit, adding --beam-width 3 for beam search.

Question 6

Which flags control generation cost and length?

Accepted Answer

--max-chars sets the 60-character budget and --window controls how much of the answer each option shows. --ensemble defaults to 3 shuffled orderings per step, which the README notes is free in requests. --beam-width defaults to 1 and costs one request per beam per step, --length-penalty ranks beams by logprob divided by length to that power, and --seed gives reproducibility.

Question 7

What did beam search change in the results?

Accepted Answer

On the largest planet prompt across three seeds, greedy was correct 2 of 3 times using 8, 8 and 4 requests, while beam width 3 was correct 3 of 3 using 25, 25 and 21 requests. Beam search rescues cases where an attractive character leads nowhere, such as the space after "capital was" scoring 0.87 against 0.05 for h.

Jev Lm — uhhfeef

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community