Fast, flexible LLM inference
Quick read
Latest capture 2026-09-08 03:07
Archive status reflects the last catalog sync, not a maintenance assessment. Check recent releases, issue responses, and fit for your project with the repository shortlisting checklist.
4 paths
Agent instructions and tool configuration found in this repository.
Agent instructions 3
Claude Code
7 observed captures since 2026-05-25. Observed captures are shown by default.
Stars from first capture +483
Observed captures only
All tracked data
Observed snapshots
Observed snapshots
Nearest indexed repositories by embedding similarity.
Official inference library for Mistral models
Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
Go ahead and axolotl questions
Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
