What does this project test?

A post with roughly 400k views claims TypeSafe Jev is the fastest AI model ever built for trading, making calibrated buy/sell decisions under 100 ms. The article behind it contains no backtest, no P&L and no hit rate, so this project runs the tests itself.

What was the headline trading result?

On 298,549 trades across six Binance majors over 2.5 years, the model was right 48.4% of the time and grossed minus 0.30 bp per trade, slightly worse than a coin flip before fees; a replication on rebuilt data measured 48.6% and minus 0.16 bp.

How fast is Jev really?

Sequential calls on a warm connection from the EU measured a median of 294 ms and p99 of 447 ms. Under load across 35,685 calls the median was 295 ms, and 0.00% of decisions landed under the claimed 100 ms.

How can I verify the numbers without an API key?

Every model response used is stored in data/jev_cache.sqlite, keyed by a hash of the exact request. The scripts read that cache and make zero API calls, so check_headline_numbers.py re-runs everything and checks each published number in about 8 minutes.

Is the model bad at everything?

No. The author notes it is a good text classifier: 99.3% accurate on exchange announcement titles when its confidence is at least 0.8 (1,145 of 1,313 labelled titles), and several of its apparent errors were the author's regex being wrong.

What does it cost to replicate?

The original tests cost $0.97 of API credit. If you run the scripts against the committed cache you pay nothing, because no API call is made; the cache holds all 35,685 responses from the reported experiments.

Who wrote it and why?

It is siim's independent reproduction repository. The README frames the goal as testing a viral hype claim with everything needed to check the author: scripts, cached model responses and a per-number verification pass.

Jev directory

CommunityJev Benchmarks & Evaluations

Jev for trading: claim vs. measured

An independent test of viral claims that Jev is an HFT trading model: 298,549 intraday trades came out at a 48.4% hit rate before fees, latency measured 294 ms median, and all 35,685 model responses ship in a SQLite cache so anyone can verify without an API key.

See highlights Open github.com

Stars: 3
Language: Python
Reading time: 3 min
Published by: Community
Author: siim
Last verified: Oct 2026

siim/jev-claim-vs-measured

User repository on GitHub

View on GitHub

I tested the 'AI model for HFT' hype: TypeSafe Jev on 298,549 intraday trades. 48.4% hit rate, worse than a coin flip before fees. Every model response shipped, so anyone can verify without an API key.

Forks: 1
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

Tagscommunity catalog-import logicrw trading backtesting reproducible-research

Works forJev for Evaluation

Used inJev for Finance & Fintech

Languages: Python
Platforms: SQLite
Capabilities: Open source

Highlights

Runs on the raw tape at one decision per second and at 15-60 minute horizons; the published replication cost $0.97 of API credit.
Latency from the EU was 294 ms median and 447 ms p99 across 35,685 calls, with 0.00% under the claimed 100 ms.
On symmetric inputs the model says SELL far more often than BUY: 7,013 versus 3,312 states in the 15-minute test, 4,665 versus 827 on the tape.
data/jev_cache.sqlite holds all 35,685 responses keyed by request hash, so check_headline_numbers.py re-verifies every published number with zero API calls.
On text the model scores 99.3% on exchange announcement titles when confidence is at least 0.8: a solid cheap classifier, not a price oracle.

Quickstart

bash

git clone https://github.com/Siim/jev-claim-vs-measured && cd jev-claim-vs-measured
pip install -r requirements.txt
python check_headline_numbers.py

Watch out

Grounded in the truncated repository README and GitHub metadata reviewed on 2026-10-15; the license shows as NOASSERTION on GitHub. The measurements come from the author's own scripts and cached responses, so they are reproducible evidence against the claim under test rather than a peer-reviewed study.

Questions about this entry

Short answers drawn from the linked project or page.

Reactions & coverage

Posts, threads, and videos about this entry from around the web.

X: A trading loop with Jev and GPT-6 Astra

X: News sentiment in a trading terminal

X: A Jev trading bot on Telegram

X: A Jev and Grok copy-trading bot

This post couldn't be loaded here — read it on X.https://x.com/Mnilax/status/2102760642872508510

From the Jev guides

Original write-ups on what this entry is for.

The trust ledger: measuring confidence your domain has earned

Oct 16, 2026

ASSAY-001 calibrated Jev on CLINC150 but found it overconfident on Banking77 — one audit, two verdicts. How calibration research measures the model-and-domain pair, discounts stated confidence, and signs the evidence.

Jev Benchmarks & Evaluations#calibration#evaluation#open-models

Read article

From the Jev AI model community

Posts from builders shipping with Jev right now.

Follow @typesafeai