System One Adapter (Python)

Comparing Jev against chat models on the same questions — cost, speed, and intelligence. Each call names a provider (openai, anthropic, or gemini) plus model, and responses add usage, n_retries, latency, and a debug attempt history.

With llm_answer_mode set to probabilities (per-label distribution) or discrete, and normalize_probabilities to rescale invalid distributions to sum to 1. Anthropic's 4,096-token output limit raises TypeSafeError instead of burning malformed-output retries.

Provider SDKs are optional extras, so you install only what you use: pip install 'system-one-adapter[openai]', 'system-one-adapter[anthropic]' or 'system-one-adapter[gemini]'. Provider and model may be set as constructor defaults, and provider is required unless model is a provider instance, for example OpenAIProvider('grok-4', base_url=...) pointing at a custom OpenAI-compatible endpoint.

OpenAI's own endpoint uses the Responses API with a strict JSON Schema for structured output and JSON mode for prompted output. Custom endpoints, including OPENAI_BASE_URL, default to Chat Completions, and you can force either with api='responses' or api='chat_completions' on OpenAIProvider, which is what you want behind a proxy. Gemini uses the Interactions API, with response_format set for structured mode and the schema left in the system prompt for prompted mode; both request store=False and resend the conversation on corrective retries.

Hitting Anthropic's output token limit raises TypeSafeError with instructions to raise max_tokens, whose default is 4,096, or ask fewer questions, without consuming a malformed-output retry. Provider-declared refusals and incomplete generations, Anthropic context overflow included, raise TypeSafeError too, naming the reason and keeping the full provider response in error.debug['llm_attempts']. Malformed structure is instead retried through n_retry_malformed_structure, and transient failures through a typesafe_sdk RetryPolicy.

AsyncSystemOneAdapterClient mirrors the sync client with await client.system_one(...) and async with. Treat clients as context managers, or call close() or await aclose() once all evaluations finish: the adapter closes providers it created itself, while a provider instance you passed as model stays caller-owned. Responses are Pydantic models in typesafe-sdk 0.7.0 and later, so model_dump() and model_dump_json() work for logging or serialization.

It subclasses typesafe_sdk.SystemOneResponse with the same answers and typed views, then adds usage: input_tokens_total, output_tokens_total, n_retries, n_retries_malformed_structure and latency. debug holds llm_attempts, retry_reasons and normalization diagnostics, and each attempt records messages, model_request_parameters, llm_response and debug_info, so an attempt can be replayed through the same provider. OpenAI-compatible endpoints may omit token usage, in which case the counts come back as None and the evaluation still succeeds.