Local LLMs & Inference

MTPLX

by youssofal · Python

MTPLX is an open-source LLM inference tool written in Python, released under the Apache-2.0 license. The repository has collected 2.6k stars and 202 forks; the latest push was in October 2026. Topics include anthropic-compatible, apple-silicon and claude-code.

“The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.”

— youssofal, project description on GitHub

Details

CategoryLocal LLMs & Inference
GitHub stars2,600
Forks202
LanguagePython
LicenseApache-2.0
Last updated
Owneryoussofal
Websitemtplx.com
Repositoryyoussofal/MTPLX

Tags

anthropic-compatibleapple-siliconclaude-codeflash-nextinference-enginellm-inferencelocal-ailocal-llm

Repository facts from the GitHub public API, refreshed October 18, 2026. The quoted tagline is written by the project owner; the summary above is generated from public facts.

All Local LLMs & Inference

EAGLE

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

Local LLMs & Inference2.6k stars

nanocoder

An open coding agent for your terminal, built by a community collective rather than a company. Bring your…

Local LLMs & Inference2.5k stars

neuron-ai

The Agentic Framework of the PHP ecosystem to build production-ready AI driven applications. Connect…

Local LLMs & Inference2.1k stars

aici

AICI: Prompts as (Wasm) Programs

Local LLMs & Inference2.1k stars

HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127…

Local LLMs & Inference2.1k stars

llama2-webui

Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper`…

Local LLMs & Inference1.9k stars