Awesome Llm

No description.

A high-throughput and memory-efficient inference and serving engine for LLMs

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

Making large AI models cheaper, faster and more accessible

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain

DSPy: The framework for programming—not prompting—language models

SGLang is a high-performance serving framework for large language models and multimodal models.

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Ongoing research training transformer models at scale

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

A framework for few-shot evaluation of language models.

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Minimal reproduction of DeepSeek R1-Zero

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Access large language models from the command-line

Go ahead and axolotl questions

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Efficient Triton Kernels for LLM Training

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

A PyTorch native platform for training generative AI models

🌟100+ 原创 LLM / RL 原理图📚，《大模型算法》作者巨献！💥（100+ LLM/RL Algorithm Maps ）

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

Prompt Engineering | Prompt Versioning | Use GPT or other prompt based models to get structured output. Join our discord for Prompt-Engineering, LLMs and other latest research

No description.

AdalFlow: The library to build & auto-optimize LLM applications.

Harness LLMs with Multi-Agent Programming

Easiest and laziest way for building multi-agent LLMs applications.

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

Minimalistic large language model 3D-parallelism training

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

A simple, performant, and scalable Jax LLM!

Seamlessly integrate LLMs as Python functions

OpenAGI: When LLM Meets Domain Experts

Mesh TensorFlow: Model Parallelism Made Easier

A security scanner for your LLM agentic workflows

Make your LLM agent and chat with it simple and fast!