Is TypeLLM just another name for the hosted Jev service?

No. It keeps the model's native thinking and free-form generation while constraining outputs to schema, and it targets open models you serve yourself rather than acting as an official Jev service.

Do I need a GPU server, or can I call this without serving a model?

Use the hosted cloud API with a key, or point the same client at your own SGLang server running a compatible model with prefix caching enabled. Both take the same generate requests.

Does turning on thinking mode raise my token bill substantially?

Yes for the fields that think. Thinking is off by default with a per-field budget cap, so only the fields you flag pay reasoning tokens while the rest answer at once.

What breaks if I ask for an open-ended answer outside the schema?

Choices stay within allowed values including enums capped at two dozen options, while free text stops after a bounded token limit and numerics use bounded decimal decoding.

How do I install it and choose between cloud and self-hosting?

pip install -U typellm, then either sign in and create a key, export it as TYPELLM_API_KEY with the tl-sk- prefix, and build TypeLLMClient with that key; or serve a compatible model with SGLang with prefix caching enabled and point TypeLLMClient at the server URL and model name, such as Qwen3.8-27B on 127.0.0.1:30000. Both clients take the same generate requests, and the hosted API includes $5 free credit.

What is the full set of output types?

String, integer, number, boolean, enum choice, object and array. Enum choices support string, integer and number types with 2 to 26 values, and the declared type validates them; objects group typed properties and arrays return a variable number of typed items. A string without an enum is free text, which stops after 128 tokens unless you raise text_max_tokens, and maxLength is not supported. Nullable fields add null to the type.

How does permutation averaging reduce option-order bias?

It is on by default and reduces option-order bias on enum and boolean probabilities using balanced, sampled or exhaustive orderings. The request can also declare depends_on dependency graphs with incremental prefix reuse, so independent fields run together while dependent fields are answered in order. There is also per-field thinking mode, off by default, with a budget you control.

CommunityJev Practices & Patterns

TypeAR

Type-safe generation layer over existing autoregressive models via JSON Schema on SGLang, returning constrained string, numeric, boolean, and enum answers with optional thinking.

See highlights

Open github.com

Stars: 941
Language: Python
License: APACHE-2.0
Published by: Community
Author: zmtomorrow
Last verified: Oct 2026

zmtomorrow/TypeAR

Organization repository on GitHub

View on GitHub

TypeLLM: LLMs with type-safe generation

Forks: 61
Last pushed: Updated Oct 2026

Repo stats from the GitHub API, cached Oct 2026.Homepage

About this project

Homepage   •   Blog   •   Docs   •   Examples   •   API   •   Contact   🚀 TypeLLM API is live: typed outputs without serving a model, $5 free credit. Try it · Quick start · Examples. [2026/10/08] Conditional fields can test how sure an answer is: "when": {"category": {"confidence": {"gte": 0.8}}}. [2026/10/05] Added objects and arrays: group typed properties into one value, or return a variable number of typed items.

Quoted from the project's README. Read the full README.

Tagscommunity jevbest catalog-import

Works forJev for Classification

Used inJev for Software & DevTools

Languages: Python

Highlights

Schema-guaranteed outputs from unchanged open weights, including enums, numerics, and nullable fields.
Dependency graphs with shared-prefix reuse so independent fields run together efficiently.
Per-field thinking mode plus permutation averaging to reduce option-order bias.
Image input for vision-language models alongside text context and conditional fields.

Watch out

TypeLLM constrains existing open models rather than replacing a calibrated decision service, and thinking, images, and long free text add tokens beyond the otherwise minimal single-token selections. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

jev-on-a-laptop

Unofficial laptop study of parallel constrained decoding: prefill once, share the KV cache across fields, and decide all schema fields in one batched forward pass without writing JSON.