TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
Appears on
Quick read
Latest capture 2026-09-09 03:10
Archive status reflects the last catalog sync, not a maintenance assessment. Check recent releases, issue responses, and fit for your project with the repository shortlisting checklist.
273 paths
Agent instructions and tool configuration found in this repository.
Agent instructions 5
Claude Code 267
CodeRabbit
249 more paths detected.
7 observed captures since 2026-05-25. Observed captures are shown by default.
Stars from first capture +850
Observed captures only
All tracked data
Observed snapshots
Observed snapshots
Nearest indexed repositories by embedding similarity.
Achieve state of the art inference performance with modern accelerators on Kubernetes
llama.cpp fork with additional SOTA quants and improved performance
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
