Local LLMs & Inference

litgpt

by Lightning-AI · Python

litgpt is an open-source LLM inference tool written in Python, released under the Apache-2.0 license. With 14k stars on GitHub, it was last updated in October 2026. Topics include ai, artificial-intelligence and deep-learning.

“20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.”

— Lightning-AI, project description on GitHub

Details

CategoryLocal LLMs & Inference
GitHub stars13,717
Forks1,634
LanguagePython
LicenseApache-2.0
Last updated
OwnerLightning-AI
Websitelightning.ai
RepositoryLightning-AI/litgpt

Tags

aiartificial-intelligencedeep-learninglarge-language-modelsllmllm-inferencellms

Repository facts from the GitHub public API, refreshed October 18, 2026. The quoted tagline is written by the project owner; the summary above is generated from public facts.

All Local LLMs & Inference

OpenLLM

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Local LLMs & Inference13k stars

openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Local LLMs & Inference11k stars

PowerInfer

High-speed Large Language Model Serving for Local Deployment

Local LLMs & Inference9.8k stars

kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no…

Local LLMs & Inference9.1k stars

BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model…

Local LLMs & Inference8.9k stars

dynamo

A Datacenter Scale Distributed Inference Serving Framework

Local LLMs & Inference8.3k stars