Skip to content

Jev Demos & Experiments

Robotics and industrial control: perception-to-action gates

Perception-to-action gates are safety-critical decisions. How robotics projects use decision models and where deterministic control still wins.
Published
Category
Jev Demos & Experiments
Resources cited
10
Reading time
7 min
Tags#robotics#control#safety#demos

Robotics adds two constraints that change how a decision model should be used: a wrong answer can damage something physical, and the loop has a hard latency budget. The result is a different architecture from a chat assistant. The model evaluates a bounded set of actions, and a controller that knows the machine's limits decides whether to act.

Perception-to-action is a gate

Robo-Jev 2B is a single-pass model for robots' physical judgment. It reads a short text of what a robot senses and answers questions such as whether the glass will tip if it brakes now, with a probability for every answer. The merged weights are Qwen3.5-2B with the robo-jev v0.7 LoRA, saved in bfloat16 as a text-only model.

The self-reported benchmark is a specialization story. Physical judgment is 0.81 versus TypeSafe Jev 0.28, Liquid d1 0.17 and untrained Qwen3.5-2B minus 0.15 on 1,100 held-out situations, but general reading on BoolQ and MultiNLI is 0.71 against Jev's 0.91. On one L4 GPU a short request answers in about 50 ms and a dozen questions in about 0.1 seconds. It supports 1 to 12 questions, 2 to 466 choice options and state text up to 16,000 characters.

One documented limit matters: options past the 26th get two-letter codes that were not part of training. Other models were queried through OpenRouter while robo-jev ran locally, and transfer beyond the authors' benchmark is unproven.

Driving decisions are distributions over behaviours

DriveJev is an open-source System I decision model for autonomous driving. Instead of generating text it assigns a probability to every offered driving behaviour, and the vehicle acts on the result. The demo shows the front camera and every behaviour probability across six clips: an oncoming left turn, a four-way stop with three cars, a red-light runner, a hidden pedestrian, exit-crosswalk pedestrians and cut-ins.

The project states that every demo drive reached its destination with no collision and no violation. That describes six selected demo clips in the JevPilot simulator, not an independent benchmark or a road test. The README is truncated mid-caption, so install steps and evaluation results are not in the digest. It is MIT licensed, and the weights are in a separate DriveJev-4B repository.

The architectural lesson is the same as the ground robots: the model ranks behaviours, and the vehicle's controller keeps the final authority. A distribution over behaviours is inspectable, but it is not a safety case.

NavJev reformulates online vision-language navigation from repeated multimodal generation into visual compression plus typed action selection. It integrates waypoint geometry, BLIP captions and RAM semantic tags into compact candidate-action representations, and a Discriminative Action-Semantic Memory filters shared semantics and maintains action-specific evidence across steps. Jev then performs structured probabilistic decisions over the available action set.

On R2R-CE it reaches 27.0% success rate and 22.4% SPL with 0.65 seconds per navigation step, substantially reducing inference latency and cost compared with MLLM-based vision-language navigation methods. The results come from the authors' own pipeline, and the comparison against MLLM methods is at the system level, so the gains are not decomposed per component.

The useful idea is that the model should not re-read the whole world every step. Compress the scene into candidate actions, carry the action-specific evidence forward, and ask for a choice. That is how a decision model stays inside a control loop.

Industrial alarms are a four-way triage

HydroJEV screens SCADA alarms into four causes: cyberattack, physical fault, normal transient or faulty sensor. The task is not to stop the plant; it is to tell an operator what needs deliberate review. The authors built a four-class cause-attribution benchmark on the C-Town network in EPANET and later transferred the unchanged gated cascade to two further networks.

With only a label-free prior correction, Jev matched the hand-written rule tree at macro-F1 0.62 to 0.64 against 0.56 to 0.61 in distribution, in all four rounds. It beat the supervised classifier by 0.36 to 0.42 macro-F1 on event subtypes absent from its labels, in all four sealed pre-registered rounds, and decided 20 to 40 times faster than frontier LLMs. It outperformed the classifier below about four labeled events per class. A rule-tree-gated cascade spared an LLM reviewer 35 to 38% of windows on fresh sealed sets without losing macro-F1.

The caveat is that the results cover one simulated EPANET network plus two transfer networks in the authors' sealed rounds. The cascade stayed within a non-inferiority margin, but the abstract reports no real utility SCADA deployments.

Network and affordance judgments

Jev-IDS asks Jev two questions per serialized flow: a binary attack probability and a finite-choice traffic category. Across 5,400 decisions on a 300-flow NSL-KDD pilot split it reports F1 0.859, precision 0.941, recall 0.790 and novel-attack recall 0.838. It was 4.8 times faster and 3.8 times cheaper than GPT-5.6 Luna, with 1.5 times higher novel-attack recall, and produced 15 times fewer false alarms than a low-data Random Forest.

The limits are severe: a 300-flow NSL-KDD pilot is small and dated, increasing k to 2 reduced F1 to 0.839, and the paper calls the system open but names no repository. It is evidence of a shape, not a network security product. The precision and recall tradeoff still has to be set per deployment.

SystemOneAffordance is an exploratory workspace for 2D affordance tasks, meaning predicting interaction regions tied to robot execution. Its five Chinese-language documents cover an affordance primer, visual System One models, prior work, a minimal decision test and method D analysis. The minimal test compares independent region judgments against random-point foreground/background judgments; methods A and D have finished first-round debugging on small samples, and independent check-set evaluation has not been done.

That is a useful counterweight to the polished demos. An affordance model is only useful if its regions survive an independent check set, and this project is honest that it has not run one yet.

When to stop and think

System Switch studies exactly the robotics question: when should a fast decision model stop and hand control to a slower reasoner? On 900 held-out questions, 0.15B to 9B decision models choose to collect items 1.6 to 1.8 times more often than chance among their errors. Accuracy, calibration and sensitivity are distinct; models of similar accuracy differ widely in AUROC. Deferring the least confident 30% gains over random deferral in proportion to the actor's AUROC, with a rank correlation of 0.87.

Offline, the gain with actor and rate chosen on held-out games is +0.13 with a 95% interval of [0.08, 0.18], and +0.08 [0.02, 0.14] with shuffled options. In closed loop over 33 games and three seeds, no variant reaches the exit. The negative closed-loop result is as important as the offline gain.

Pick Verify shows the same discipline at a smaller scale: it adds a description-level judgment on top of a scan code, returns accept or re-scan, never commands robot motion, and reports a 96 ms median against a 150 ms scan-cycle budget on a 5,000-pick replay. It is a station-level aid to be evaluated on the included replay before it gates live picks.

The use-case map, JEV Sees and confidence gates in production fill in the rest. JEV Sees finds objects in images, video and RGB-D cameras, keeps their IDs consistent across frames, and asks typed Jev questions about each object; detection and tracking come from Florence-2 and CLIP, and the project publishes no accuracy or latency benchmarks. The designing for rejection and browser GUI action gating pieces cover the control-side patterns.

The rule is simple: the model should answer a bounded question with a probability, the controller should enforce the physical limits, and a person should own anything irreversible. Fail-closed is the default, and every result in this area should be read as evidence for a design, not as a safety certification.

From the Jev directory

The resources behind this article.

Steerix's robo-jev v0.7 merges a robotics LoRA into Qwen3.5-2B as a 2B System 1 that answers physical-judgment questions from a robot's scene; the card reports 0.81 physical judgment against TypeSafe Jev's 0.28.
Jev Models & Reimplementations#community#open-models#robotics
Community
94GitHub stars
An open-source System I decision model for autonomous driving: DriveJev scores every offered driving behaviour with a probability instead of generating text, and the demo clips show it holding, yielding, and proceeding in the JevPilot simulator.
Jev Models & Reimplementations#community#github#autonomous-driving
A vision-language navigation framework that replaces multimodal generation with visual compression and typed action selection: Action-Centric Visual Compression builds action representations, Discriminative Action-Semantic Memory keeps them distinct. On R2R-CE it reaches 27.0% SR and 22.4% SPL at 0.65 s per step.
Jev Models & ReimplementationsArticle#community#research#arxiv
Community
A pre-registered evaluation of Jev as a one-second, training-free screen for SCADA alarm triage in water distribution networks. On a four-class EPANET C-Town benchmark it matched a hand-written rule tree, and a gated cascade spared an LLM reviewer 35-38% of windows.
Jev Practices & PatternsArticle#community#research#arxiv
Community
JEV-IDS is an open experimental network intrusion detector that serializes one flow per request and asks Jev for a binary attack probability and a finite-choice traffic category. On a 300-flow NSL-KDD pilot it reports F1 0.859 and novel-attack recall 0.838.
Jev Tools & CLIsArticle#community#research#arxiv
Community
0GitHub stars
A research workspace exploring visual System One models on 2D affordance tasks, predicting interaction regions for robot execution, with a five-document primer covering concepts, candidate models, prior work and first-round debugging of methods A and D.
Jev Practices & Patterns#community#github#robotics
Studies a fast learned actor in closed-loop Doom that hands control to a reasoning vision-language model only when a gate opens. On 900 held-out questions, deferring the least confident 30% of decisions gains +0.13 with doomLaya's option order, but no closed-loop variant reaches the exit.
Jev Practices & PatternsArticle#community#research#arxiv
A warehouse pick-verification step that compares a scanned item, the bin's expected contents, and the order line to decide between accept and re-scan, catching confusable picks a barcode alone would pass.
Jev Integrations & Adapters#community#python#robotics
Community
Explore TypeSafe use cases by industry and turn promising ideas into concrete software workflows.
Jev Practices & PatternsDocs#official#use-cases
Official
224GitHub stars
An independent open-source pipeline that gives Jev vision: it detects and tracks objects in images, video, and RGB-D camera feeds, then asks Jev a typed question about each relevant object and prints the chosen answer with its confidence.
Jev Tools & CLIs#community#github#python
Back to all articles

Related Jev articles

Per-tick decisions, NPC behaviour, and matchmaking run at a different clock. What the game demos show about latency and action gating.
Jev Demos & Experiments#gaming#demos#latency
Read article
A game tick is a state plus a legal action list. How Clash Royale, Minecraft, and the earlier Mario, Pokémon, StarCraft, and drone loops ask Jev one step at a time.
Jev Demos & Experiments#gaming#agents#control
Read article

From the Jev AI model community

Posts from builders shipping with Jev right now.

Follow @typesafeai