A fast inference library for running LLMs locally on modern consumer-class GPUs
Appears on
Quick read
Latest capture 2026-09-11 03:06
Archive status reflects the last catalog sync, not a maintenance assessment. Check recent releases, issue responses, and fit for your project with the repository shortlisting checklist.
0 paths
Agent instructions and tool configuration found in this repository.
No config files detected.
7 observed captures since 2026-05-25. Observed captures are shown by default.
Stars from first capture +85
Observed captures only
All tracked data
Observed snapshots
Observed snapshots
Nearest indexed repositories by embedding similarity.
Inference code for Llama models
llama.cpp fork with additional SOTA quants and improved performance
Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models
