Middleware is where a decision model earns its place in a TypeScript app. The AI SDK already routes model calls, tool calls, and streamed output through composable functions; a Jev call can sit in that path and return a typed judgment before the expensive or irreversible step runs. Vercel supports the pattern at two levels: a hosted Jev model on AI Gateway, and a provider package for the AI SDK. The hard part is not the call. It is deciding what the middleware does when the answer is uncertain.
Two integration surfaces
The Vercel AI SDK is the TypeScript toolkit for models and agents: 27,329 stars, a NOASSERTION license in GitHub metadata, and topics spanning OpenAI, Anthropic, Gemini, React, Svelte, Vue, and Next.js. It is not a Jev-specific project, and its README in the catalog is only a pointer to the package README. The Jev-specific packages are separate.
On the gateway, Jev is listed as typesafe-ai/jev on the AI Gateway model page. The card says evaluation type, 32,000-token context, 0 maximum output tokens, $0.042 per 1M input tokens, and no output charge. TypeSafe confirmed the listing in an announcement post. The evaluation documentation describes the call shape: experimental_evaluate from ai on AI SDK 7 or later, or POST /v1/evaluate on ai-gateway.vercel.sh for non-AI-SDK callers. Evaluation is not available through the OpenAI-compatible, Anthropic-compatible, or Cohere-compatible endpoints, so a chat-completions client will not reach it.
In the AI SDK itself, @ai-sdk/typesafe-ai exposes typeSafeAi.decisionModel('jev-latest') for experimental_decide. You pass a model, a state string, and a map of choice, score, or boolean questions; the typed answers arrive on result.answers. This is the more direct route when the decision is part of a TypeScript function rather than a gateway request.
What a decision middleware actually checks
A useful middleware has three outputs, not two: proceed, block, or ask. A binary gate collapses "I am not sure" into either allow or deny, and both are wrong some of the time. The confidence-gated routing pattern makes the distinction explicit. In the voice-banking example, intent below 0.6 confidence routes to a support agent before any action; check_balance can act at 0.6 because a wrong read-out is cheap; approve_transfer needs more than 0.85 or the user confirms.
For a tool-call middleware, that maps to a small policy table:
| Decision | Question shape | Middleware action |
|---|---|---|
| Is this tool call allowed? | Noul over the call plus policy | Allow, deny, or escalate |
| Which handler should run? | Choice over a fixed set | Dispatch to one function |
| How risky is this request? | Score over ordered levels | Set review level |
| Is the output grounded? | Noul per claim or passage | Pass, revise, or block |
The middleware should use the answer and the confidence together. A confidence gate with a per-action threshold is the pattern; a single global threshold is a policy bug waiting for the first high-stakes action. Confidence Gates in Production covers threshold design.
The fx safety reviewer is the cautionary example
Vercel's fx auto mode analyzes every command with a safety reviewer. Guillermo Rauch said on X that the reviewer, then on GPT Luna, is up to 18x faster at p95 and more accurate with Jev, and that Jev may become the default. A Vercel engineer's reply says the benchmark showed roughly 5 to 18x faster and more accurate than gpt-f-luna, the previous top choice.
Those are vendor claims about Vercel's own classifier. Neither post publishes the dataset, the labels, or how accuracy was scored. The reviewer judges a single shell command, not a whole agent trajectory, and neither post describes the low-confidence path or what happens when the reviewer call itself fails. That missing policy is the important part. A gate that is 18x faster but has no documented behavior when Jev is unavailable is not yet a production gate.
The reusable lesson is to keep the decision narrow. One command is a bounded yes-or-no judgment on one string. A whole trajectory is a different question, and the same model may not be the right tool for it. If you need trajectory-level review, decompose it into bounded questions or use a different mechanism.
Wiring it with the AI SDK
Vercel's Jev and AI SDK guide is the most concrete end-to-end example. It budgets 64,000 tokens per request with 32,000 for state, at $0.042 per 1M input tokens with no output charge. Choice confidence is at result.providerMetadata.typesafe.confidence, keyed by question id, and is not returned for boolean answers, where the probability always means P(true). TypeSafe rounds probabilities and scores to two decimals, so a choice distribution may sum to 0.99; assert on the selected option and the values, not exact arithmetic.
Two operational details are easy to miss. providerOptions.gateway.zeroDataRetention: true is supported per request, but evaluation calls still appear in Gateway logs and budgets, so retention and logging are separate settings. And the local VERCEL_OIDC_TOKEN from vercel env pull expires after 12 hours; a 401 usually means pull it again. Deployments on Vercel receive the token automatically.
The guide also uses Experimental_EvaluationMockModelV4 from ai/test to return supplied answers, so the threshold branches can be tested with no network call. That is the right test boundary: the middleware policy should be tested without the model, and the model should be tested separately against a labeled set.
MCP is the other middleware
Typesafe MCP puts the same judgment behind an MCP tool. It is a single-binary Go server exposing one evaluate tool with Noul, Choice, and Score questions, and one-command registration for Claude Code, Claude Desktop, and Codex. It retries 429 and 529 with exponential backoff, applies a 60-second timeout, caps responses at 16 MiB, and can evaluate up to 500 records in one call, with failed records not stopping the rest.
That is middleware in the agent-protocol sense: the agent calls a tool, and the server decides. It is useful when the agent environment already speaks MCP and you do not want to write SDK glue. It is not a gate on the agent's own tool calls unless you put it in that position. The same distinction applies to any MCP decision server: an available tool is not a policy.
Gateway trade-offs
| Route | Auth | Latency | Best fit |
|---|---|---|---|
| Direct TypeSafe API | TypeSafe key | Lowest | Tight loops, custom retries |
| Vercel Gateway | Vercel credentials | One hop | Next.js apps already on AI SDK |
| AI SDK provider | TypeSafe key | Direct or gateway | Typed decisions in TypeScript |
| MCP server | TypeSafe or OpenRouter key | Local process plus API | Agent environments |
The gateway's only: ['typesafe-ai'] option is worth setting. The model page lists both TypeSafe AI and DigitalOcean, and which provider wins is not specified. If residency, retention, or contracting depends on the provider, pin it. The same page warns that detailed capability metadata has not been reported, and the 32,000-token window is the whole budget, so check before sending a long document as state.
A typed decision is the payoff: the next function receives a value from a known set, not prose. The middleware can then be tested like any other branch. The remaining risk is the one the fx example shows: an undocumented uncertainty path. Define what happens at low confidence, on timeout, and on a malformed answer before the gate goes anywhere near production. Jev on the Gateways compares the other surfaces, and Decision Gates in Agent Frameworks covers the framework-level equivalents.
