Is that 5 ms number a real end-to-end decision?

It is a single-question result on an M3 Max with ANE FP16: a 91-token question padded to 96, including prompt preparation, tokenization, array building, synchronous inference, calibration, and formatting, with loading and warmup excluded. The README states the requested 10x improvement was not achieved.

Will it fit my documents, or is it short-prompt only?

The ANE bundle has a 96-token total limit covering question, options, and state; longer requests raise a capacity error rather than truncating. For general work, point at the 1024-token multilingual bundle instead.

Is the Neural Engine export the same model as the plain Core ML one?

No, they are separate implementations. The ordinary SDPA export defaults to CPU plus GPU after unrestricted GPU shapes failed local fidelity checks, and changing its device setting alone does not reproduce the ANE numbers. The ANE rewrite uses BC1L activations and per-head attention, with CPU still handling input and output.

Is the port faithful to upstream Laya?

On conversion fidelity, yes: three general-purpose FP16 checkpoints match upstream selected answers on 189 of 189 validation questions and survive 100 repeated calls each; ANE FP16 at 96 tokens passes 59 of 59 with maximum calibrated-probability drift of 0.002925. Those are porting fixtures, not evidence of task accuracy.

What hardware and OS does Laya-CoreML require?

Apple Silicon, macOS 15+, Python 3.11-3.13; the demo runs offline after a one-time download and needs no PyTorch, Transformers or MLX for inference.

What is the token limit on the ANE bundle?

The ANE bundle has a 96-token total limit including question, options and state; use aac6fef/laya-multilingual-coreml for the general-purpose 1024-token model.

How does the W8 variant compare with FP16 on energy?

The W8 palette variant reached 4.88 ms P50 and a 3.19x whole-system energy improvement per decision, versus 4.98 ms and 2.78x for the FP16 ANE bundle; it is approximate.

CommunityJev Models & Reimplementations

Laya-CoreML

Laya typed decisions through Apple Core ML and the Neural Engine, with the author's speed and energy benchmarks for short decisions on an M3 Max.

See highlights

Open github.com

Stars: 1.6k
Language: Python
License: APACHE-2.0
Published by: Community
Author: mizorewww
Last verified: Oct 2026

mizorewww/laya-coreml

User repository on GitHub

View on GitHub

Local Laya typed decisions on Apple Core ML and Neural Engine. Validated ports, ~5 ms short decisions on M3 Max, reproducible speed and energy benchmarks.

Forks: 131
Last pushed: Updated Oct 2026

Repo stats from the GitHub API, cached Oct 2026.Homepage

About this project

Version 0.4.0 aligns the host inference contract directly with official Laya v0.4.1 (1adc59f), replacing the earlier selective port. Batch/long-text prediction, hooks, structured decisions, calibration, HTTP/MCP, evaluation and Router behavior now share the official host semantics. See the migration guide for changed defaults, validation evidence and the remaining Core ML boundaries. Open-weight typed decisions on Apple Silicon. Core ML, Neural Engine, zero generated tokens. PyPI · Hugging Face weights · 中文.

Quoted from the project's README. Read the full README.

Tagscommunity python open-models local

Works forJev for Classification

Used inJev for Software & DevTools

Languages: Python
Platforms: macOS
Capabilities: Open sourceRuns locally

Highlights

Apache-2.0. 1,540 stars on 3 October 2026.
The description says short decisions are about 5 ms on an M3 Max.
Targets Core ML and the Neural Engine, not the MLX runtime in laya-mlx.

Watch out

The 5 ms figure is the repo's own claim. Same author as laya-mlx, and not a TypeSafe or Convai project.

Questions about this entry

Short answers drawn from the linked project or page.

Reactions & coverage

Posts, threads, and videos about this entry from around the web.

LinkedIn: What multimodal decision models actually change, and what they do not.

LinkedInJevDirectory.org

Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.

jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker News

Hacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.

1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker News

Hacker News: A one-token logprob wrapper that also classifies webcam frames.

allanrbo151 pointsA single function Jev-like wrapper for LLMs, including vision modelsRead the thread on Hacker News

Related terms

Glossary definitions related to this entry.

Open weights

More Jev resources like this

Laya-MLX

6.9k

A native MLX runtime for Laya's open typed-decision weights: short decisions on Apple Silicon, with no generated tokens, PyTorch, or cloud API.

Jev Models & Reimplementations#community#python#open-models

Community

mizorewww/laya-mlxPython

SemIf-OpenJev

4.8k

An open-model semantic-if engine positioned as a local Jev-style decision stack, separate from the original SemIf repo but sharing the same author.

Jev Models & Reimplementations#community#python#open-models

Community

TheoLeeCJ/SemIf-OpenJevPython

jevos

1.3k

A local yes/no decision model that speaks Jev's wire format on a laptop CPU, refusing Choice and Score until those question types ship.

Jev Models & Reimplementations#community#python#open-models

Community

feder-cr/jevC++

Rizzo Flow

877

A local server that reads next-token probabilities from a fine-tuned Spark model and returns Choice, Score, and Noul answers with zero generated tokens.

Jev Models & Reimplementations#community#python#open-models

Community

Rizzo-AI-Academy/rizzo-flowPython

mini-Jev

A preregistered measurement of a Jev-style interface on frozen Qwen3-4B: read the option letter's logits instead of generating grammar-constrained JSON.

Jev Models & Reimplementations#community#python#local

Community

r-ms/mini-jevPython

OpenDecision

An Apache-2.0 decision engine with a TypeSafe-compatible endpoint, document answers with source passages, and a ViZDoom demo that picks actions from structured game state.

Jev Models & Reimplementations#community#python#local

Community

deepanwadhwa/OpenDecisionPython

From the Jev guides

Original write-ups on what this entry is for.

Exporting Decision Models: GGUF, ONNX, and Device Runtimes

Oct 10, 2026

Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.

Jev Tools & CLIs#open-models#local#onnx

Read article

Choosing an Open Decision Model

Oct 10, 2026

Interface, size, license, claims — in that order. A selection guide across the system-one catalog, from a 706K form scorer to 27B Jev students.

Jev Practices & Patterns#open-models#evaluation#benchmarks

Read article

What Is Actually on Hugging Face

Oct 8, 2026

Hundreds of repositories carry the system-one tag, but the ecosystem is a dozen ideas in different sizes. A tour of the families, the licenses, and what a download does not include.

Jev Models & Reimplementations#open-models#hugging-face#local

Read article

Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai