Snapshot
Scroll sideways to see all columns. With a keyboard, focus this table and use the arrow keys.
| Item | Detail |
|---|---|
Deployment | Local: the app and CLI share one server on 127.0.0.1:8000, with OpenAI routes (/v1/chat/completions, /v1/responses, /v1/completions, /v1/models) and an Anthropic-compatible /v1/messages, plus /health and /metrics. |
Requirements | Apple Silicon (M1 or newer), macOS 14 or later. |
Install | The Mac app as a 60 MB DMG with everything included, or the CLI alone with brew install youssofal/mtplx/mtplx or python3 -m pip install mtplx. |
Cost basis | Published as free and open source; the models run on hardware you already own. |
License | Apache-2.0, commercial shipping included, but attribution is required: a product built on MTPLX must say so inside the product, and a mention in your repo or website does not cover it. |
Clients | OpenCode, Pi, Hermes, Claude Code, Cline, Cursor, Continue, Open WebUI, curl and the openai and anthropic Python clients. |
Retrieval | The same daemon serves MLX embedding and reranker models on /v1/embeddings and /v1/rerank, loaded on first request and capped by --retrieval-max-resident (default 2), so a RAG setup needs no second inference server. |
Models | Official packs on Hugging Face under Youssofal: Qwen 3.8 Flash Next, Qwen 3.8 27B in three packs each with an FP16 build for M1 and M2, Ternary Bonsai 2 27B, Qwen 3.6, Qwen 3.5, MiMo V2.6 Qwen 9B and Gemma 4. |
Our take
The buying question is unified memory, and it is settled before you install anything. On M3, M4 and M5 Macs the app and CLI recommend Qwen 3.5 4B Optimized Speed below 16 GB, Ternary Bonsai 2 27B from 16 GB, Qwen 3.8 27B Optimized Speed from 32 GB and Flash Next Optimized Speed from 256 GB. With 16 GB you get Bonsai 2 in an 8.85 GB pack with vision, but an 8,192-token window and a measured 11.80 GiB peak, and agent clients such as OpenCode and Hermes need 18 GB or more — so 32 GB is the entry point for the coding pack MTPLX itself recommends.
Evaluate MTPLX
Comparisons
Where it stands out
Exactness is tested, not asserted. For 2.11.3 the project compared a thousand four-token samples from the fast path with a thousand from the plain path at temperature 1, top-p 0.95 and top-k 20, by token id; they matched within the plain path's own noise on Flash Next and the 27B Quality pack.
Published speeds carry their conditions: 125.8 tok/s on one OpenCode request means 1,301 tokens generated against an 18,539-token prompt with 18,364 tokens served from cache at depth 3, on an M5 Max with 128 GB and fans verified. You can check whether your workload looks like that one.
Tuning is measured on your own Mac: mtplx tune runs the real model at each depth with fans pinned, keeps autoregressive decoding as the baseline, and saves a depth only when it beats that baseline. On a 16 GB M4 Mac mini the 9B lands on depth 1, 14.4 tok/s to 23.0.
No silent fallbacks. mtplx inspect classifies a model before weights load — verified, family- or architecture-compatible but unverified, AR-only, incompatible, or no MTP heads at all — and unverified models load with an explicit label.
A default-on SSD session cache restores sessions near-instantly across restarts; a 96,760-token conversation restored in 8 ms in the app. That is the difference between resuming a long agent thread and re-reading its prompt.
Forge converts a Hugging Face repo to MLX, trains an MTP adapter, and reports the measured before-and-after verdict instead of assuming the adapter helped.
Product screenshot

Trade-offs
Apple Silicon only. MTPLX is MLX-native and not a CUDA project; for Linux the project points you to vLLM, so it covers one half of a mixed fleet.
Throughput falls as the prompt grows: on an M5 Max with 2.11.3, Flash Next decoded 79.3 tok/s on a 9k-token code prompt, 61.8 on a 109k-token OpenCode turn and 50.3 at 200k. Budget by the context length you actually work at.
From 2.12.0 image requests also use the compiled verifier, and the sampled fast-path against plain-path comparison has not yet been repeated for images. Vision work sits outside the check that backs the text claim.
Flash Next Optimized Quality is a 169.96 GB download whose weights need about 128.5 GiB, and the project states it has not yet been run on a 256 GB Mac and its speed has not been measured.
The Responses route is stateless, text-only and runs with hosted tools disabled: Codex 0.146 sends web_search by default and gets a precise 400 until that hosted tool is disabled or removed.
You cannot attach a separately supplied MTP sidecar to an arbitrary MLX trunk; a model has to ship its own matching MTP weights or be rebuilt and verified through Forge.
Public signals
About these signals
Verified factual signals from public sources. They indicate observable activity or interest, not total adoption, product quality, or cost.
1.5k GitHub commits 90d2.7k GitHub stars0 vulnerabilities across 1 package
See all signals from 3 sources
PyPI · mtplx@2.12.3
Alternatives to MTPLX
The reviewed substitutes for MTPLX among the model runtimes, and what would make each one the better answer.
Direct alternatives
Reviewed substitutes: products bought for the same job, where a team picks one.
- Ollama
- Ollama pulls and serves many open-weight models on macOS and other platforms behind OpenAI- and Anthropic-compatible APIs, and it aims for quick setup more than top speed. MTPLX serves a small set of Qwen and Bonsai models on Apple Silicon with built-in MTP speculative decoding, so the choice comes down to wanting Ollama's wide model library or MTPLX's higher tokens per second and its ability to fit a 27B model on a 16 GB Mac.Applies to: local coding-agent model serving on a Mac
Other approaches
A different approach to the same problem. Each substitutes only for the workload named beside it.
- LocalAI
- LocalAI is a self-hosted runtime that puts text, voice, vision and image models behind OpenAI, Anthropic and Ollama-compatible APIs, and it runs on CPU-only hardware too. MTPLX does one job, fast text generation for a few models on Apple Silicon, so LocalAI wins when a buyer needs many kinds of models through one endpoint and MTPLX wins when decoding speed on a Mac matters most.Applies to: self-hosted OpenAI/Anthropic-compatible local endpoint