Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
Appears on
Quick read
Latest capture 2026-09-10 10:55
Archive status reflects the last catalog sync, not a maintenance assessment. Check recent releases, issue responses, and fit for your project with the repository shortlisting checklist.
16 paths
Agent instructions and tool configuration found in this repository.
Claude Code 15
CodeRabbit
1 observed capture since 2026-09-10. Observed captures are shown by default.
Stars from first capture 0
Observed captures only
All tracked data
Observed snapshots
Observed snapshots
Nearest indexed repositories by embedding similarity.
AirLLM 70B inference with single 4GB GPU
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
Accessible large language models via k-bit quantization for PyTorch.
A high-throughput and memory-efficient inference and serving engine for LLMs
