What does it actually cost to get one sentence out of this?

About 2.02 s and 147,329 input tokens per word, roughly $0.0062, across six calls: four carrying the dictionary in buckets, one for the completeness noul, and the run-off. A 60-word reply costs about 37 cents, and the recorded two-turn conversation cost $1.13 over 368 seconds and 1,092 calls.

How good is the next-word prediction, honestly?

On 10 transcripts with a known next word it gets 7 of 10 at top-1 and 8 of 10 at top-5. The README attributes both misses to the dictionary rather than the model: it answered four where the test wanted 4, and jupiter is absent from a 10,000-word frequency list, so it offered saturn. Read that way it is 9 of 10.

Why shuffle the vocabulary every step if order changes the answer?

Criteria order does bias the answer, so the dictionary is reshuffled, cut into 255-word buckets because that is the API's hard cap, and packed 20 to a call across four concurrent calls. Scores are normalised within a bucket only, so a nonsense bucket can still return a confident winner, which is why the run-off takes the top two from every bucket into one final choice.

Will rate limiting bite if I use it heavily?

The probed limits table records that sustained bursts can come back as a 401 authentication error, which clears on its own. Other probed limits: 255 criteria per choice, 32 questions of 255 criteria per call before max_tokens_exceeded, and a 12 KB state accepted. Decoding is otherwise deterministic, and --seed only drives the bucket shuffle.

uv run jevgpt starts an interactive REPL with /reset and /quit, and uv run jevgpt --say 'hi' runs a scripted turn, repeatable for a conversation. Flags include --max-words 150 (a hard stop per reply), --seed 0 (the bucket-shuffle seed) and --record PATH to save per-word distributions. TYPESAFE_API_KEY is read from a gitignored .env.

What are the three repetition penalties for?

Each was added after a loop was observed: local 1.5 to the power n over the last eight words (the last position counting triple), content 2.5 to the power n capped at four over the whole reply, and a gentler stopword 1.6 to the power n capped at six. They stop the model repeating what it just said and keep it off near-miss fragments.

Why must a reply run to sixty words?

Neither the end token nor the completeness noul (threshold 0.9) is eligible before 60 real words, and punctuation and newlines do not count toward that floor. Forced past the length it wants, the model runs out of things to classify and drifts into morphological neighbours, which is where the seam in the output comes from.

CommunityJev Practices & Patterns

Jevgpt — Bewinxed

Chat client over jev-latest, which cannot generate text: the transcript becomes the state and each step is a choice over a 20,000-word dictionary, so replies are built one word at a time.

See highlights

Open github.com

Stars: 30
Language: TypeScript
License: MIT
Published by: Community
Author: Bewinxed
Last verified: Oct 2026

Bewinxed/jevgpt

User repository on GitHub

View on GitHub

A chatbot built on a model that cannot generate text (TypeSafe AI's Jev, driven autoregressively)

Forks: 2
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

A chatbot built on a model that cannot generate text. TypeSafe AI's jev-latest is a decision model: it does not generate text, it only answers multiple-choice questions about a state. JevGPT is a chat client built on top of that — the state is the conversation transcript, the question is "Next word?", and the criteria are a 20,000-word dictionary. Ask it a question and it answers, one word at a time, several hundred choices deep. demo/jevgpt.mp4 is a Remotion render of JevGPT building a reply word by word.

Quoted from the project's README. Read the full README.

Tagscommunity jevbest catalog-import awesomejev

Works forJev for Classification

Used inJev for Software & DevTools

Languages: TypeScript

Highlights

Six calls per word, 2.02 s and 147,329 input tokens each, so a 60-word reply runs about 37 cents.
Bucket probabilities are not comparable across buckets, so the top two from every 20-word bucket compete in a final choice.
Three repetition penalties were each added after a loop was observed, from local to whole-reply to a gentler stopword pass.
On 10 transcripts with a known next word it gets 7 of 10 top-1 and 8 of 10 top-5, with both misses blamed on the dictionary.

Watch out

A reply may not end before 60 real words, and asked to stop at six the model runs out of things to classify, so replies drift into morphological neighbours such as end, ends, endlessly. The recorded two-turn conversation took 368 seconds, 1,092 API calls and $1.13. Bursts of traffic can return a 401 authentication error that clears on its own. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

Other listings of this project

Forks and copies of this project in the directory. This entry is the canonical one.

Jevgpt

Show HN for giga-james/jevgpt, an autoregressive chatbot built by repeatedly asking Jev to pick the next symbol from a tiny vocabulary.

Jev Guides & ArticlesPost#community#demos

Community

More Jev resources like this

jev-lm

Word-level language model experiment that uses Jev as its output layer, with explicit tokenizer, sampler, repetition mask, stop rules, and chunk verification.