How does Jev control the aircraft in FlightBench?

Each control step sends Jev a state snapshot covering flight, runway, current controls, motion and phase, and asks four Choice questions - aileron, elevator, rudder and throttle - each with seven to ten textual levels. The controller maps the selected labels to fixed absolute commands, and JSBSim computes the resulting physics.

Can I try landing the plane myself?

Yes. Run the FastAPI backend and the Vite frontend locally, then fly from the browser in human-pilot mode with no API key or Codex account. The author's record is 70.1/100; recordings can be submitted to the project, and baseline and Jev flights are watchable as published previews.

How good is Jev at landing, honestly?

Not great. Its headline run scored 56 and landed, clearing the 50-point success threshold, but across attempts it landed only 14% of the time, ran off the runway 74% of the time, hit nose-first 9% and timed out 2%. The baseline controller scored 69.9/100 against the author's human record of 70.1/100.

What did a full Jev flight cost?

168 calls at an effective rate of about 2.4 Hz, a mean latency of 0.33 s per call, roughly 291k input tokens and an estimated $0.0122. The 71 attempts that actually landed took 71 seconds of simulation and wall time.

How do the Codex models compare?

No GPT-6 variant landed, even at 2 Hz lockstep. GPT-6 Luna at low effort crashed nose-first across 39 of 202 calls at about 0.4 Hz and 2.53 s per call, and GPT-6 Astra ran off the runway on 49 of 61 calls at 4.50 s per call and an estimated $1.758, roughly 143 times Jev's cost for that flight.

What do I need installed to run it?

Python 3.11 to 3.13 with uv, and Node.js 20.19+ or 22.12+ with npm. `uv sync` then `uv run uvicorn flightbench.server:app` on 127.0.0.1:8000, and `npm ci && npm run dev` in the frontend folder. Jev needs `uv sync --extra jev` plus TYPESAFE_API_KEY; Codex needs the codex extra, the Codex CLI and `codex login` against a ChatGPT plan rather than an API key.

What else does the benchmark cover?

Fixed-wing landing under JSBSim physics with Three.js visuals, a sunny preset, TypeScript and Vite on the front end and FastAPI behind it, MIT licensed. The simulation state sent per step includes runway geometry such as altitude AGL, centerline offset, heading error and glideslope error.

CommunityJev Demos & Experiments

FlightBench

Fixed-wing landing simulator where a human can compete against Jev: four Jev Choice decisions map to aileron, elevator, rudder and throttle commands in JSBSim, run in real time. The benchmark also compares Jev, classical controllers and Codex models on landing score and API cost.

See highlights

Open github.com

Stars: 2
Language: Python
License: MIT
Published by: Community
Last verified: Oct 2026

AlperKartkaya/FlightBench

User repository on GitHub

View on GitHub

FlightBench is a fixed-wing landing simulator and benchmark for decision models, LLMs, and classical controllers. You can run the simulator, land the plane, and compete with Jev and more.

Forks: 1
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

FlightBench Can you land a plane better than Jev? FlightBench is a landing simulator and benchmark for decision models, LLMs, and classical controllers. Try landing it yourself first, and compete with Jev, Codex, or your friends! Jev is TypeSafe's System One decision model. Given a state and typed questions, it returns structured choices and probabilities. Here, the right panel shows the state sent to Jev and the four actuator commands that Jev chose. Watch the full flight!

Quoted from the project's README. Read the full README.

Tagscommunity simulation aviation benchmarking

Works forJev for Realtime control Jev for Evaluation

Used inJev for Gaming

Languages: Python TypeScript
Platforms: Browser
Capabilities: Open sourceRequires API key

Highlights

Jev landed with a score of 56, above the 50-point success threshold, in realtime.
168 calls at about 2.4 Hz, mean latency 0.33 s, roughly 291k input tokens and $0.0122 estimated.
Landing outcomes: 14 percent success, 74 percent runway excursion, 9 percent nose-first, 2 percent timeout.
No Codex model landed; GPT-6 variants ran in 2 Hz lockstep at 2.53-4.50 s per call.
The author's human record is 70.1/100 and the baseline controller scores 69.9/100.

Watch out

MIT licensed. Jev's landing record is not strong at 14 percent success, and the numbers cover one constrained scenario. Jev mode needs a TypeSafe API key and the jev extra; Codex runs need a Codex CLI login; human and baseline modes need neither.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

jev-plays

A Craftax (Crafter) survival harness where deterministic code lists every feasible action with its facts and Jev picks one Choice per step, optionally guided by an LLM-written objective and standing rules; three-seed ablations are included.

Jev Demos & Experiments#community#craftax#games

Community

mansicer/jev-playsPython

jev-goal-reflex

Three.js simulation where an OpenAI model turns each plain-language instruction into a plan and Jev makes every reflex decision: two Score questions for move and turn plus one Noul for jump, with code acting only past a 0.33 dead zone and sending no Jev request while no step is active.

Jev Demos & Experiments#community#threejs#simulation

Community

blakeandrewwood/jev-goal-reflexTypeScript

Cua

29.3k

The Cua computer-use platform documents a jev-use recipe where TypeSafe Jev picks bounded action IDs and Cua Driver executes clicks and fills in the browser.

Jev Demos & Experiments#community#browser#computer-use

Community

trycua/cuaRust

Pacman AI Race

Browser Pac-Man race where a deterministic three-junction simulation filters fatal routes, then Jev makes one typed Choice among the surviving route IDs while the server rejects any answer outside the supplied set. Self-hosted Laya runs the same contract for comparison.

Jev Demos & Experiments#community#gaming#simulation

Community

MaryNfs/pacman-ai-raceJavaScript

clash-jev

A Clash Royale bot that turns a phone screenshot into JSON once a second and asks Jev for a strategy, a card, and a square.

Jev Demos & Experiments#community#python#gaming

Community

bytelabs-oss/clash-jevPython

System-One Control Bench

A benchmark that steers decision models through 100 key-and-door grid puzzles with one Jev Choice per move, scored against an exact BFS solver; it found Jev picks an optimal move about 90% of the time yet finishes only 57 of 100 games.

Jev Benchmarks & Evaluations#community#benchmark#gridworld

Community

Exemocaro/System-One-Control-BenchPython

From the Jev guides

Original write-ups on what this entry is for.

Per-Tick Decisions in Games

Sep 28, 2026

A game tick is a state plus a legal action list. How Clash Royale, Minecraft, and the earlier Mario, Pokémon, StarCraft, and drone loops ask Jev one step at a time.

Jev Demos & Experiments#gaming#agents#control

Read article

Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai